On Task-Level Dialogue Composition of Generative Transformer Model (2020)

Abstract: Task-oriented dialogue systems help users accomplish tasks such as booking a movie ticket and ordering food via conversation. Generative models parameterized by a deep neural network are widely used for next turn response generation in such systems. It is natural for users of the system to want to accomplish multiple tasks within the same conversation, but the ability of generative models to compose multiple tasks is not well studied. In this work, we begin by studying the effect of training human-human task-oriented dialogues towards improving the ability to compose multiple tasks on Transformer generative models. To that end, we propose and explore two solutions: (1) creating synthetic multiple task dialogue data for training from human-human single task dialogue and (2) forcing the encoder representation to be invariant to single and multiple task dialogues using an auxiliary loss. The results from our experiments highlight the difficulty of even the sophisticated variant of transformer model in learning to compose multiple tasks from single task dialogues.

## introduction recent years have seen a tremendous surge in the application of deep learning methods for dialogue in general @xcite @xcite and task-oriented dialogue @xcite @xcite specifically. task-oriented dialogue systems help users accomplish tasks such as booking a movie ticket and ordering food via conversation. generative models are a popular choice for next turn response generation in such systems @xcite @xcite . these models are typically learned using large amounts of dialogue data for every task @xcite . it is natural for users of the task-oriented dialogue system to want to accomplish multiple tasks within the same conversation, e.g. booking a movie ticket and ordering a taxi to the movie theater within the same conversation. the brute-force solution would require collecting dialogue data for every task combination which might be practically infeasible given the combinatorially many possibilities. while the ability of generative dialogue models to compose multiple tasks has not yet been studied in the literature, there has been some investigation on the compositionality skills of deep neural networks. lake and @xcite propose a suite of tasks to evaluate a method's compositionality skills and find that deep neural networks generalize to unseen compositions only in a limited way. @xcite analyze whether the language emerged when multiple generative models interact with each other is compositional and conclude that compositionality arises only with strong regularization. motivated by the practical infeasibility of collecting data for combinatorially many task compositions, we focus on task-level compositionality of text response generation models. we begin by studying the effect of training data size of humanhuman multiple task dialogues on the performance of transformer @xcite generative models. next, we explore two solutions to improve task-level compositionality. first, we propose a data augmentation approach @xcite @xcite @xcite where we create synthetic multiple task dialogues for training from human-human single task dialogue; we add a portion of one dialogue as a prefix to another to simulate multiple task dialogues during training. as a second solution, we draw inspiration from the domain adaptation literature @xcite @xcite @xcite and encourage the model to learn domain invariant representations with an auxiliary loss to learn representations that are invariant to single and multiple task dialogues. we conduct our experiments on the multiwoz dataset @xcite . the dataset contains both single and multiple task dialogues for training and evaluation. in multiwoz, the tasks in multiple task dialogues are only the combinations of tasks in single task dialogues. this allows the dataset to be an appropriate benchmark for our experiments. to summarize, our key findings are: the response generation model is trained to generate each turn of the assistant response given the conversation history. the generative model learns a probability distribution given by p (a i | (u 1 , a 1 ), . . . , (u i-1 , a i-1 ), u i ). we drop the symbol m that denotes a particular training example for simplicity. the assistant turn a i consists of a sequence of word tokens, a @xmath0, w i 2 , . . . , w i l i ). the response generation model factorizes the joint distribution left-to-right given by, ## data augmentation the first solution we explore for task compositionality generates synthetic multiple task dialogues for training from human-human single task dialogues 1 . here, we sample two dialogues from the training set, and add a portion of one dialogue as a prefix to another. while this procedure might not create dialogues of the quality equivalent to human-human multiple task dialogue, it is an unsupervised way to create approximate multiple task dialogues that the model could theoretically benefit from. concretely, we randomly sample two single task dialogues d i and d j from the training set and create a noisy multiple task dialogue by adding a fraction of the dialogue d j as a prefix to dialogue d i . the fraction of dialogue taken from dialogue d j is given by the hyperparameter augment f raction. the number of times dialogue d i is augmented by a randomly sampled dialogue is given by the hyperparameter augment f old. we consider two strategies for sampling the dialogue d j . in random augment, the dialogue is uniformly randomly sampled from the remainder of the training set. a potential issue with the random strategy is that it might create spurious task combinations and the model might fit to this noise. motivated by the spurious task combination phenomenon, we consider another sampling strategy t argeted augment where we create synthetic multiple task dialogues only for task combinations that exist in the development set. here, d j is sampled from a set of dialogues whose task is compatible with the task of dialogue d i . the transformer model is now trained on the augmented training set using the objective function given in equation 1. the effect of the sampling strategy and the hyperparameters on the model performance is discussed in the experiments section (section 5). ## domain invariant transformer we propose domain invariant transformer model (figure 1 ) to maintain a domain invariant representation of the encoder by training the encoder representation for an auxiliary task. here, the auxiliary task for the network is to predict the label , i l, denoting the type of task (single or multi-task) in the encoded conversation history. the model takes as input the sequence of byte pair encoded tokens that are represented at the encoder hidden state as a set of attention weights from the multi-head multiple layer attention mechanism of transformer. the conditional language model (equation 1 ) is learnt by a transformer decoder on top that attends over the encoder states. the discriminator task network is trained with average pooling of the encoder summary over the attention heads (h j )as shown in equation 2 . the average pooled encoder summary is passed as input to a two-layer feed forward discriminator. the discriminator network has a dropout @xcite layer in-between the two fully connected layers (f 1 and f 2 ) (equation 3 ). the binary cross-entropy loss, l disc , for the predicted label, ŷi , an input context i is computed as in equation 4 . the language model loss makes sure that the model learns to generate the next utterance while the discriminator loss makes sure the model is aware of the nature of task. to understand the effect of the auxiliary loss we experiment with different values for α (ref appendix). ## conclusion we studied the problem of composing multiple dialogue tasks to predict next utterance in a single multiple-task dialogue. we found that even powerful transformer models do not naturally compose multiple tasks and the performance is severely relied on multiple task dialogues. in this paper, we explored two solutions that only further showed the difficulty of composing multiple dialogue tasks.