Hierarchical Transformer for Task Oriented Dialog Systems (2021)

Abstract: Generative models for dialog systems have gained much interest because of the recent success of RNN and Transformer based models in tasks like question answering and summarization. Although the task of dialog response generation is generally seen as a sequence to sequence (Seq2Seq) problem, researchers in the past have found it challenging to train dialog systems using the standard Seq2Seq models. Therefore, to help the model learn meaningful utterance and conversation level features, Sordoni et al. (2015b), Serban et al. (2016) proposed Hierarchical RNN architecture, which was later adopted by several other RNN based dialog systems. With the transformer-based models dominating the seq2seq problems lately, the natural question to ask is the applicability of the notion of hierarchy in transformer-based dialog systems. In this paper, we propose a generalized framework for Hierarchical Transformer Encoders and show how a standard transformer can be morphed into any hierarchical encoder, including HRED and HIBERT like models, by using specially designed attention masks and positional encodings. We demonstrate that Hierarchical Encoding helps achieve better natural language understanding of the contexts in transformer-based models for task-oriented dialog systems through a wide range of experiments.

## introduction dialog systems are concerned with replicating the human ability to make conversation. in a generative dialog system, the model aims at generating coherent and informative responses given a dialog context and, optionally, some external information through knowledge bases @xcite or annotations e.g. belief states, dialog acts etc. @xcite . a dialog is usually represented as a series of utterances. however, it is not sufficient to view each utterance independently for engaging in a conversation. in a dialogue between humans, the speakers communicate both utterance level and dialog level information. e.g., dialog intent often cannot be detected by looking at a single utterance, whereas dialog acts are specific to each utterance and change throughout a conversation. intuitively, we can instruct the model to achieve both utterance level and dialog level understanding separately through a hierarchical encoder @xcite . there has been a lot of interest in the past towards using the hierarchical encoder-decoder (hred) model for encoding utterances in many rnn based dialog systems. however, since the rise of transformers and self-attention @xcite , the use of hierarchy has not been explored further for transformer-based dialog models. past research and user-studies have also shown that hierarchy is an important aspect of human conversation @xcite . but, most previous works based on transformer have focused on training models either as language models @xcite or as standard (non-hierarchical) seq2seq models @xcite @xcite with certain task specific extensions. although arguably, the self-attention mechanism might automatically learn such a scheme during the training process, our empirical results show that forcing this inductive bias by manual design as proposed here leads to better performing models. this paper bridges these two popular approaches of transformers and hierarchical encoding for dialogs systems to propose a family of hierarchical transformer encoders. although arguably, the self-attention mechanism of standard encoders might automatically learn such a scheme during the training process, our empirical results show that forcing this inductive bias by manual design as proposed here leads to better performing models. our contributions in this paper include: ## models formally, the task of a dialog system is to predict a coherent response, r, given a dialog context c. in case of a goal oriented dialog system, context c might consist of dialog history, c @xmath0, s 1 , ..., u t ], and optionally a belief state (dialog act, slot values, intent etc.) b t , when available. here, u i , s i represent the user and system utterances at turn i, respectively. the actual target response following c t is the system utterance s t . ## experimenal framework our implementation is based on the pytorch library. all the models use a vocabulary of size 1,505. we generate responses using beam search foot_1 with beam width 5. the model optimizes a cross entropy loss. full details of model parameters are given in suplementary material. ## results for the four different experimental settings discussed in section 3.1, we showcase results from those experiments in tables 2 through 5 . in the context-to-response generation task with oracle policy ( further in while the above experiments focus on proving the base performance of the proposed response generation models (hier, hier++, hier-cls, and ablations), ht-encoder can be applied to any model that uses a standard transformer encoder. hence, in a final experiment ( young et al. (2013) modeled this problem as a markov decision process whose state comprised of various utterance and dialog features detected by an nlu module. however, such models had the usual drawback of any pipelined approaches, error propagation. wen et al. (2017) proposed using neural networks for extracting features like intent, belief states, etc. and training the nlu and nlg modules end-to-end using a single loss function. marco (wang et al., 2020) and hdsa (chen et al., 2019) used a finetuned bert model as their act predictor as it often triumphs other ways to train the dialog policy network (even joint learning). hdsa is a transformer seq2seq model with act-controllable self-attention heads (in the decoder) to disentangle the individual tasks and domains within the network. marco uses a soft-attention over the act sequence during the response generation process. hierarchical encoders the concept of hierarchical encoders have been used in many different context in the past. it has been most well known in the area of dialog response generation as the hred model. many open domain dialog systems have used the hierarchical recurrent encoding scheme of hred for various tasks and architectures. hierarchical encoder was first proposed by (sordoni et al., 2015a) for using in a query suggestion system. they used it encode the user history comprising multiple queries using an hierarchical lstm network. serban et al. (2016) extended this work to open domain dialog generation problems and proposed the hred network. hred captures the high level features of the conversation in a context rnn. several models have adopted this approach later on, e.g. vhred @xcite , cvae @xcite , dialogwae @xcite , etc. another area in which researchers have proposed the use of hierarchical encoder is for processing of paragraph or long documents. @xcite used a hierarchical lstm network for training an autoencoder that can encode and decode long paragraphs and documents. @xcite proposed hibert where they introduced hierarchy into the bert architecture to remove the limitation on length of input sequence. hibert samples a single vector for each sentence or document segment (usually contextual embedding of cls or eos token) from the sentence encoder to be passed onto the higher level transformer encoder. @xcite applies a similar approach for encoding documents in a multi-document summarization task. ## conclusion this paper explored the use of hierarchy in transformer-based models for task-oriented dialog system. we started by proposing a generalized framework for hierarchical transformer encoders (ht-encoders). using that, we implemented two models, one new model called hier, and another hier-cls model by adapting the existing hib-ert architecture into our framework. we thoroughly experimented with these models in four different response generation tasks of the multi-woz dataset. we compared the proposed models with an exhaustive set of recent state-of-the-art models to thoroughly analyze the effectiveness of ht-encoders. we empirically show that the basic transformer seq2seq architecture, when equipped with an ht-encoder, outperforms many of the stateof-the-art models in each experiment. we further prove its usefulness by applying it to an existing model marco. this work opens up a new direction on hierarchical transformers in dialogue systems where complex dependencies exist between the utterances. it would also be beneficial to explore the effectiveness of the proposed ht-encoder when applied for various other tasks.