Venue: ACL 2020

# Multi-Domain Dialogue Acts and Response Co-Generation

Kai Wang1, Junfeng Tian2, Rui Wang2, Xiaojun Quan1, Jianxing Yu1

1School of Data and Computer Science, Sun Yat-sen University, China

2Alibaba Group, China wangk73@mail2.sysu.edu.cn, {quanzj3, yujx26}@mail.sysu.edu.cn

{tjlf141457, masi.wr}@alibaba-inc.com

Xiaojun Quan is the corresponding author of this paper. Most of this work was done when Kai Wang was working as an intern at Alibaba DAMO Academy.

###### Abstract

Generating fluent and informative responses is of critical importance for task-oriented dialogue systems. Existing pipeline approaches generally predict multiple dialogue acts first and use them to assist response generation. There are at least two shortcomings with such approaches. First, the inherent structures of multi-domain dialogue acts are neglected. Second, the semantic associations between acts and responses are not taken into account for response generation. To address these issues, we propose a neural co-generation model that generates dialogue acts and responses concurrently. Unlike those pipeline approaches, our act generation module preserves the semantic structures of multi-domain dialogue acts and our response generation module dynamically attends to different acts as needed. We train the two modules jointly using an uncertainty loss to adjust their task weights adaptively. Extensive experiments are conducted on the large-scale MultiWOZ dataset and the results show that our model achieves very favorable improvement over several state-of-the-art models in both automatic and human evaluations.

+
Footnote †: _Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics_, pages 7125–7134

+
Footnote †: _Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics_, pages 7125–7134

## 1 Introduction

Task-oriented dialogue systems aim to facilitate people with such services as hotel reservation and ticket booking through natural language conversations. Recent years have seen a rapid proliferation of interests in this task from both academia and industry Bordes et al. (2017); Budzianowski et al. (2018); Wu et al. (2019). A standard architecture of these systems generally decomposes this task into several subtasks, including natural language understanding Gupta et al. (2018), dialogue state tracking Zhong et al. (2018) and natural language generation Su et al. (2018). They can be modeled separately and combined into a pipeline system.

Figure 1 shows a dialogue example, from which we can notice that the natural language generation subtask can be further divided into dialogue act prediction and response generation Chen et al. (2019); Zhao et al. (2019); Wen et al. (2017). While the former is intended to predict the next action(s) based on current conversational state and database information, response generation is used to produce a natural language response based on the action(s).

In order for dialogues to be natural and effective, responses should be fluent, informative, and relevant. Nevertheless, current sequence-to-sequence models often generate uninformative responses like "I don't know" Li et al. (2016a), hindering the dialogues to continue or even leading to a failure. Some researchers Pei et al. (2019); Mehri et al. (

Figure 1: An example of dialogue from the MultiWOZ dataset, where the dialogue system needs to generate a natural language response according to current belief state and related database records.



2019) sought to combine multiple decoders into a stronger one to avoid such responses, while others (Chen et al., 2019; Wen et al., 2015; Zhao et al., 2019; Wen et al., 2017) represent dialogue acts in a global, static vector to assist response generation.

As pointed out by Chen et al. (2019), dialogue acts can be naturally organized in hierarchical structures, which has yet to be explored seriously. Take two acts _station-request-stars_ and _restaurant-inform-address_ as an example. While the first act rarely appears in real-world dialogues, the second is more often. Moreover, there can be multiple dialogue acts mentioned in a single dialogue turn, which requires the model to attend to different acts for different sub-sequences. Thus, a global vector is unable to capture the inter-relationships among acts, nor is it flexible for response generation especially when more than one act is mentioned.

To overcome the above issues, we treat dialogue act prediction as another sequence generation problem like response generation and propose a co-generation model to generate them concurrently. Unlike those classification approaches, act sequence generation not only preserves the interrelations among dialogue acts but also allows close interactions with response generation. By attending to different acts, the response generation module can dynamically capture salient acts and produce higher-quality responses. Figure 2 demonstrates the difference between the classification and the generation approaches for act prediction.

As for training, most joint learning models rely on hand-crafted or tunable weights on development sets (Liu and Lane, 2017; Mrksic et al., 2017; Rastogi et al., 2018). The challenge here is to combine two sequence generators with varied vocabularies and sequence lengths. The model is sensitive during training and nontrivial to generate an optimal weight. To address this issue, we opt for an uncertainty loss (Kendall et al., 2018) to adaptively adjust the weight according to task-specific uncertainty. We conduct extensive studies on a large-scale task-oriented dataset to evaluate the model. The experimental results confirm the effectiveness of our model with very favorable performance over several state-of-the-art methods.

The contributions of this work include:

* We model dialogue act prediction as a sequence generation problem that allows to exploit act structures for the prediction.
* We propose a co-generation model to generate act and response sequences jointly, with an uncertainty loss used for adaptive weighting.
* Experiments on MultiWOZ verify that our model outperforms several state-of-the-art methods in automatic and human evaluations.

## 2 Related Work

Dialogue act prediction and response generation are closely related in general in the research of dialogue systems (Chen et al., 2019; Zhao et al., 2019; Wen et al., 2017), where dialogue act prediction is first conducted and used for response generation. Each dialogue act can be treated as a triple (domain-action-slot) and all acts together are represented in a one-hot vector (Wen et al., 2015; Budzianowski et al., 2018). Such sparse representation makes the act space very large. To overcome this issue, Chen et al. (2019) took into account act structures and proposed to represent the dialogue acts with level-specific one-hot vectors. Each dimension of the vectors is predicted by a binary classifier.

To improve response generation, Pei et al. (2019) proposed to learn different _expert_ decoders for different domains and acts, and combined them with a _chair_ decoder. Mehri et al. (2019) applied a cold-fusion method (Sriram et al., 2018) to combine their response decoder with a language model. Zhao et al. (2019) treated dialogue acts as latent variables and used reinforcement learning to optimize them. Reinforcement learning was also applied to find optimal dialogue policies in task-oriented dialogue systems (Su et al., 2017;

Figure 2: Demonstration of hierarchical dialogue act structures (top) and different approaches (bottom) for dialogue act prediction. Classification approaches separately predict each act item (domain, action and slot), while generation approaches treat each act as a token that can be generated sequentially.

 

Williams et al. (2017) or obtain higher dialog-level rewards in chatting Li et al. (2016, 2016); Serban et al. (2017). Besides, Chen et al. (2019) proposed to predict the acts explicitly with a compact act graph representation and employed hierarchical disentangled self-attention to control response text generation.

Unlike those pipeline architectures, joint learning approaches try to explore the interactions between act prediction and response generation. A large body of research in this direction uses a shared user utterance encoder and train natural language understanding jointly with dialogue state tracking Mrksic et al. (2017); Rastogi et al. (2018). Liu and Lane (2017) proposed to train a unified network for two subtasks of dialogue state tracking, i.e., knowledge base operation and response candidate selection. Jiang et al. (2019) showed that joint learning of dialogue act and response benefits representation learning. These works generally demonstrate that joint learning of the subtasks of dialogue systems is able to improve each other and the overall system performance.

## 3 Architecture

Let \(T=\{U_{1},R_{1},\ldots,U_{t-1},R_{t-1},U_{t}\}\) denote the dialogue history in a multi-turn conversational setting, where \(U_{i}\) and \(R_{i}\) are the \(i\)-th user utterance and system response, respectively. \(D=\{d_{1},d_{2},\ldots,d_{n}\}\) includes the attributes of related database records for current turn. The objective of a dialogue system is to generate a natural language response \(R_{t}=y_{1}y_{2}\ldots y_{n}\) of \(n\) words based on the current belief state and database attributes.

In our framework, dialogue acts and response are co-generated based on the transformer encoder-decoder architecture Vaswani et al. (2017). A standard transformer includes a multi-head attention layer that encodes a value \(V\) according to the attention weights from query \(Q\) to key \(K\), followed by a position-wise feed-forward network (\(\mathcal{G}_{f}\)):

\[O=V+\mathcal{G}_{f}(MultiHead(Q,K,V)) \tag{1}\]

where \(Q,K,V,O\in\mathbb{R}^{n\times d}\). In what follows we use \(\mathcal{F}(Q,K,V)\) to denote the transformer.

EncoderWe use \(E=\text{Emb}([T;D])\) to represent the concatenated word embeddings of dialogue history \(T\) and database attributes \(D\). The transformer \(\mathcal{F}(Q,K,V)\) is then used to encode \(E\) and output its hidden state \(H^{e}\):

\[H^{e}=\mathcal{F}(E,E,E) \tag{2}\]

DecoderAt each time step \(t\) of response generation, the decoder first computes a self-attention \(h_{t}^{r}\) over already-generated words \(y_{1:t-1}\):

\[h_{t}^{r}=\mathcal{F}(e_{t-1}^{r},e_{1:t-1}^{r},e_{1:t-1}^{r}) \tag{3}\]

where \(e_{t-1}^{r}\) is the embedding of the \((t-1)\)-th generated word and \(e_{1:t-1}^{r}\) is an embedding matrix of \(e_{1}^{r}\) to \(e_{t-1}^{r}\). Cross-attention from \(h_{t}^{r}\) to dialogue history \(T\) is then executed:

\[c_{t}^{r}=\mathcal{F}(h_{t}^{r},H^{e},H^{e}) \tag{4}\]

The resulting vectors of Equations 3 and 4, \(h_{t}^{r}\) and \(c_{t}^{r}\), are concatenated and mapped to a distribution of vocabulary size to predict next word:

\[p(y_{t}|y_{1:t-1})=\text{softmax}(W_{r}[c_{t}^{r};h_{t}^{r}]) \tag{5}\]

## 4 The MarCo Approach

Based on the above encoder-decoder architecture, our model is designed to consist of three components, namely, a shared encoder, a dialogue act generator, and a response generator. As shown in Figure 3, instead of predicting each act token individually and separately from response generation, our model aims to generate act sequence and response concurrently in a joint model which is optimized by the uncertainty loss Kendall et al. (2018).

### Dialogue Acts Generation

Dialogue acts can be viewed as a semantic plan for response generation. As shown in Figure 2, they can be naturally organized in hierarchical structures, including domain level, action level, and slot level. Most existing methods treat dialogue acts as triples represented in one-hot vectors and predict the vector values with binary classifiers Wen et al. (2015); Budzianowski et al. (2018). Such representations ignore the inter-relationships and associations among acts, domains, actions and slots. For example, the slot _area_ may appear in more than one domain. Unlike them, we model the prediction of acts as a sequence generation problem, which takes into consideration the structures of acts and generates each act token conditioned on its previously-generated tokens. In this approach, different domains are allowed to share common slots and the search space of dialogue act is greatly reduced.

The act generation starts from a special token "\(\langle\text{SOS}\rangle\)" and produces dialogue acts \(A=a_{1}a_{2}\ldots a_{n}\) sequentially. During training, the act 

sequence is organized by domain, action and slot, while items at each level are arranged in dictionary order, where identical items are merged. When decoding each act token, we first represent the current belief state with an embedding vector \(v_{b}\) and add it to each act word embedding \(e^{a}_{t}\) as:

\[u^{a}_{t}=W_{b}v_{b}+e^{a}_{t}. \tag{6}\]

Finally, the decoder of Section 3.2 is used to generate hidden states \(H^{a}\) and act tokens accordingly.

### Acts and Response Co-Generation

Dialogue acts and responses are closely related in dialogue systems. On one hand, system responses are generated based on dialogue acts. On the other, their shared information can improve each other through joint learning.

Shared EncoderOur dialogue act generator and response generator share one same encoder and input, but having different masking strategies for the input to focus on different information. In particular, only the current utterance is kept for act generation, while the entire history utterances are used for response generation.1

Footnote 1: Empirical evidences show that act generation is more related to the current utterance, while response generation benefits more from long dialogue history.

Dynamic Act AttentionA response usually corresponds to more than one dialogue act in multi-domain dialogue systems. Nevertheless, existing methods mostly use a static act vector to represent all the acts, and add the vector to each response token representation. They ignore the fact that different subsequences of a response may need to attend to different acts. To address this issue, we compute dynamic act attention \(o^{r}_{t}\) from the response to acts when generating a response word:

\[o^{r}_{t}=\mathcal{F}(h^{r}_{t},H^{a},H^{a}) \tag{7}\]

where \(h^{r}_{t}\) is the current hidden state produced by Equation 3. Then, we combine \(o^{r}_{t}\) and \(h^{r}_{t}\) with response-to-history attention \(c^{r}_{t}\) (by Equation 4) to estimate the probabilities of next word:

\[p(y_{t}|y_{1:t-1})=\text{softmax}(W_{r}[h^{r}_{t};c^{r}_{t};o^{r}_{t}]) \tag{8}\]

Uncertainty LossThe cross-entropy function is used to measure the generation losses, \(\mathcal{L}_{a}(\theta)\) and \(\mathcal{L}_{r}(\theta)\), of dialogue acts and responses, respectively:

\[\mathcal{L}_{a}(\theta) =-\sum_{j=1}^{T_{a}}\log p(a^{*(i)}_{j}|a^{(i)}_{1:j-1},T,D,v_{b}) \tag{9}\] \[\mathcal{L}_{r}(\theta) =-\sum_{j=1}^{T_{r}}\log p(y^{*(i)}_{j}|y^{(i)}_{1:j-1},T,D,A) \tag{10}\]

Figure 3: Architecture of the proposed model for act and response co-generation, where act and response generators share the same encoder. The response generator is allowed to attend to different act hidden states as needed using dynamic act attention. The two generators are trained jointly and optimized by the uncertainty loss.

 

where the ground-truth tokens of acts and response of each turn are represented by \(A^{*}\) and \(Y^{*}\), while the predicted tokens by \(A\) and \(Y\).

To optimize the above functions jointly, a general approach is to compute a weighted sum like:

\[\mathcal{L}(\theta)=\alpha\mathcal{L}_{a}(\theta)+(1-\alpha)\mathcal{L}_{r}(\theta) \tag{11}\]

However, dialogue acts and responses vary seriously in sequence length and vocabulary size, making the weight \(\alpha\) unstable to tune. Instead, we opt for an uncertainty loss Kendall et al. (2018) to adjust it adaptively:

\[\mathcal{L}(\theta,\sigma_{1},\sigma_{2})=\frac{1}{2\sigma_{1}^{2}}\mathcal{ L}_{a}(\theta)+\frac{1}{2\sigma_{2}^{2}}\mathcal{L}_{r}(\theta)+\log\sigma_{1}^{2} \sigma_{2}^{2} \tag{12}\]

where \(\sigma_{1}\) and \(\sigma_{2}\) are two learnable parameters. The advantage of this uncertainty loss is that it models the homoscedastic uncertainty of each task and provides task-dependent weight for multi-task learning Kendall et al. (2018). Our experiments also confirm that it leads to more stable weighting than the traditional approach (Section 6.3).

## 5 Experiments

### Dataset and Metrics

MultiWOZ 2.0 Budzianowski et al. (2018) is a large-scale multi-domain conversational datatset consisting of thousands of dialogues in seven domains. For fair comparison, we use the same validation set and test set as previous studies Chen et al. (2019); Zhao et al. (2019); Budzianowski et al. (2018), each set including 1000 dialogues.2 We use the _Inform Rate_ and _Request Success_ metrics to evaluate dialog completion, with one measuring whether a system has provided an appropriate entity and the other assessing if it has answered all requested attributes. Besides, we use BLEU Papineni et al. (2002) to measure the fluency of generated responses. To measure the overall system performance, we compute a combined score: \((\textit{Inform Rate}+\textit{Request Success})\times 0.5+\textit{BLEU}\) as before Budzianowski et al. (2018); Mehri et al. (2019); Pei et al. (2019).

Footnote 2: There are only five domains (_restaurant_, _hotel_, _attract_, _taxi_, _train_) of dialogues in the test set as the other two (_hospital_, _police_) have insufficient dialogues.

### Implementation Details

The implementation3 is on a single Tesla P100 GPU with a batch size of 512. The dimension of word embeddings and hidden size are both set to 128. We use a 3-layer transformer with 4 heads for the multi-head attention layer. For decoding, we use a beam size of 2 to search for optimal results, and apply trigram avoidance Paulus et al. (2018) to fight trigram-level repetition. During training, we first train the act generator for 10 epochs for warm-up and then optimize the uncertainty loss with the Adam optimizer Kingma and Ba (2015).

Footnote 3: [https://github.com/InitialBug/MarCo-Dialog](https://github.com/InitialBug/MarCo-Dialog)

### Baselines

A few mainstream models are used as baselines for comparison with our neural co-generation model (MarCo), being categorized into three categories:

* **Without Act**. Models in this category directly generate responses without act prediction, including LSTM Budzianowski et al. (2018), Transformer Vaswani et al. (2017), TokenMoE Pei et al. (2019) and Structured Fusion Mehri et al. (2019).
* **One-Hot Act**. In SC-LSTM Wen et al. (2015), dialogue acts are treated as triples and information flow from acts to response generation is controlled by gates. HDSA Chen et al. (2019) is a strong two-stage model, which relies on BERT Devlin et al. (2019) to predict a one-hot act vector for response generation.
* **Sequential Act**. Since our model does not rely on BERT, to make a fair comparison with HDSA, we design the experiments from two aspects to ensure they have the same dialogue act inputs for response generation. First, the act sequences produced by our co-generation model are converted into one-hot vectors and fed to HDSA. Second, the predicted one-hot act vectors by BERT are transformed into act sequences and passed to our model as inputs.

### Overall Results

The overall results are shown in Table 1, in which HDSA (MarCo) means HDSA using MarCo's dialogue act information, and MarCo (BERT) means MarCo based on BERT's act prediction. From the table we can notice that our co-generation model (MarCo) outperforms all the baselines in _Inform Rate_, _Request Success_, and especially in _combined score_ which is an overall metric. By comparing the two HDSA models, we can find HDSA derives its main performance from the external BERT, which can also be used to improve our MarCo considerably (MarCo (BERT)). These 

results confirm the success of MarCo by modeling act prediction as a generation problem and training it jointly with response generation.

Another observation is that despite its strong overall performance, MarCo shows inferior BLEU performance to the two HDSA models. The reason behind this is studied and analyzed in human evaluation (Section 7), showing that our model often generates responses inconsistent with references but favored by human judges.

The performance of our model across different domains is also compared against HDSA. The average number of turns is 8.93 for single-domain dialogues and 15.39 for multi-domain dialogues Budzianowski et al. (2018). As in Figure 4, our model shows superior performance to HDSA across all domains. The results suggest that MarCo is good at dealing with long dialogues.

Results on MultiWOZ 2.1We also conducted experiments on MultiWOZ 2.1 Eric et al. (2019), which is an updated version of MultiWOZ 2.0. As shown in Table 2, the overall results are consistent with that on MultiWOZ 2.0.

## 6 Further Analysis

More thorough studies and analysis are conducted in this section, trying to answer three questions: (1) How is the performance of our act generator in comparison with existing classification methods? (2) Can our joint model successfully build semantic associations between acts and responses? (3) How does the uncertainty loss contribute to our co-generation model?

### Dialogue Act Prediction

To evaluate the performance of our act generator, we compare it with several baseline methods mentioned in Chen et al. (2019), including BiLSTM, Word-CNN, and 3-layer Transformer. We use MarCo to represent our act generator which is trained jointly with the response generator, and use Transformer (GEN) to denote our act generator without joint training. From Table 3, we notice that the separate generator, Transformer (GEN), performs much better than BiLSTM and Word-CNN, but comparable with Transformer. But after trained jointly with the response generator, MarCo manages to show the best performance, confirming the effect of the co-generation.

\begin{table}
\begin{tabular}{l l c c c c} \hline \hline Dialog Act & Model & Inform & Success & BLEU & Combined Score \\ \hline  & LSTM & 71.29 & 60.96 & 18.80 & 84.93 \\ Without Act & Transformer & 71.10 & 59.90 & 19.10 & 84.60 \\  & TokenMoE & 75.30 & 59.70 & 16.81 & 84.31 \\  & Structured Fusion & 82.70 & 72.10 & 16.34 & 93.74 \\ \hline  & SC-LSTM & 74.50 & 62.50 & 20.50 & 89.00 \\ One-hot Act & HDSA (MarCo) & 76.50 & 62.30 & 21.85 & 91.25 \\  & HDSA & 82.90 & 68.90 & **23.60** & 99.50 \\ \hline Sequential Act & MarCo & 90.30 & 75.20 & 19.45 & 102.20 \\  & MarCo (BERT) & **92.30** & **78.60** & 20.02 & **105.47** \\ \hline \hline \end{tabular}
\end{table}
Table 1: Overall results on the MultiWOZ 2.0 dataset.

Figure 4: Combined score of MarCo vs. HDSA across different domains. If a dialogue involves more than one domain, it is copied into each domain. Single-domain includes dialogues with only one domain mentioned, while the rest belongs to the multi-domain.

 



### Joint vs. Pipeline

To study the influence of the joint training and the dynamic act attention on response generation, we implement two pipeline approaches for comparison. We first train our act generator separately from response generation. Then, we keep its parameters fixed and train the response generator. The first baseline is created by replacing the dynamic act attention (Equation 7) with an average of the act hidden states, while the second baseline uses the dynamic act attention. As shown in Table 4, Pipeline\({}_{2}\) with dynamic act attention is superior to Pipeline\({}_{1}\) without it in all metrics, but inferior to the joint approach. Our joint model also surpasses the currently state-of-the-art pipeline system HDSA, even HDSA uses BERT. We find that by utilizing sequential acts, the dynamic act attention mechanism helps the response generator capture the local information by attending to different acts.

An illustrative example is shown in Figure 5, where the response generator can attend to the local information such as "day" and "stay" as needed when generating a response asking about picking a different day or shorter stay. We reckon that by utilizing sequential acts, response generation benefits in two ways. First, the dynamic act attention allows the generator to attend to different acts when generating a subsequence. Second, the joint training makes the two stages interact with each other, easing error propagation of pipeline systems.

### Uncertainty Loss

We opt for an uncertainty loss to optimize our joint model, rather than a traditional weighted-sum loss. To illustrate their difference, we conduct an experiment on the development set. For the traditional loss (Equation 11), we run for each weight from 0 to 1 stepped by 0.1. Note that since the weights, \(\sigma_{1}\) and \(\sigma_{2}\), in the uncertainty loss are not hyperparameters but learned internally to each batch, we only record the best score within each round without giving the values of \(\sigma_{1}\) and \(\sigma_{2}\). As shown in Figure 6, the uncertainty loss can learn adaptive weights with consistently superior performance.

## 7 Human Evaluation

We conduct a human study to evaluate our model by crowd-sourcing.4 For this purpose we randomly selected 100 sample dialogues (742 turns in total) from the test dataset and constructed two groups of systems for comparison: MarCo vs. HDSA and

Figure 5: An illustrative example of the dynamic act attention mechanism. Response (row) subsequence can attend to the act (column) token “day” or “stay” as needed when generating a response asking about picking a different day or shorter stay.

\begin{table}
\begin{tabular}{l c c c c} \hline \hline Model & Inform & Succ & BLEU & Combined \\ \hline HDSA & 82.9 & 68.9 & 23.60 & 99.50 \\ Pipeline\({}_{1}\) & 84.3 & 54.4 & 16.00 & 85.35 \\ Pipeline\({}_{2}\) & 86.6 & 66.0 & 18.31 & 94.61 \\ Joint & **90.3** & **75.2** & **19.45** & **102.20** \\ \hline \hline \end{tabular}
\end{table}
Table 4: Results of response generation by joint and pipeline models, where Pipeline\({}_{1}\) and Pipeline\({}_{2}\) represent two pipeline approaches with or without using dynamic act attention. The performance of HDSA, as the best pipeline model, is provided for comparison.

Figure 6: Performance of the uncertainty loss and the weighted-sum loss on the development dataset.



MarCo vs. Human Response, where Human Response means the reference responses. Responses generated by each group were randomly assigned in pairs to 3 judges, who ranked them according to their completion and readability Chen et al. (2019); Zhang et al. (2019). _Completion_ measures if the response correctly answers a user query, including relevance and informativeness. _Readability_ reflects how fluent, natural and consistent the response is.

The results of this study are shown in Figure 7, where "Win", "Tie" or "Lose" mean our MarCo system wins over, ties with or loses to its counterpart, respectively. From the results we note that MarCo outperforms HDSA and Human Response in completion, and ties 94% with HDSA in readability while underperforming Human Response. Overall speaking, MarCo is superior to HDSA and comparable with Human Response. We further analyzed the bad cases of our model in readability and found that our model slightly suffers from token level repetition, a problem that can be solved by methods like the coverage mechanism Mi et al. (2016); Tu et al. (2016). In completion, our model can understand the users' need and tends to provides them more relevant information, so that they can finish their goals in shorter turns.

We present two examples in Figure 8. In the first example, the user requests the hotel type while HDSA ignores it. The user requests to book one ticket in the second example, yet both HDSA and Human Response ask about the number once again. In contrast, our model directly answers the questions with correct information. To sum up, MarCo successfully improves the dialogue system by generating relevant and informative responses.

## 8 Conclusion

In this paper, we presented a novel co-generation model for dialogue act prediction and response generation in task-oriented dialogue systems. Unlike previous approaches, we modeled act prediction as a sequence generation problem to exploit the semantic structures of acts and trained it jointly with response generation via dynamic attention from response generation to act prediction. To train this joint model, we applied an uncertainty loss for adaptive weighting of the two tasks. Extensive studies were conducted on a large-scale task-oriented dataset to evaluate the proposed model, and the results confirm its effectiveness with very favorable performance over several state-of-the-art methods.

#### Acknowledgments

We thank the anonymous reviewers for their constructive reviews. This work was partially supported by the Fundamental Research Funds for the Central Universities (No.19lgpy220 and No.19lgpy219), the Program for Guangdong Introducing Innovative and Entrepreneurial Teams (No.2017ZT07X355), and the National Natural Science Foundation of China (No.61906217).

Figure 8: Two examples to show that MarCo successfully improves the dialogue system by generating relevant and informative responses.

Figure 7: Results of human study in response quality. Two groups of systems are studied, where the top figure corresponds to results of MarCo vs. HDSA and the bottom figure represents MarCo vs. Human Response (ground-truth). “Win”, “Tie” or “Lose” respectively indicate the proportions that our MarCo system wins over, ties with or loses to its counterpart.

 



## References

* Bordes et al. (2017) Antoine Bordes, Y-Lan Boureau, and Jason Weston. 2017. Learning end-to-end goal-oriented dialog. In _5th International Conference on Learning Representations, ICLR 2017, Toulon, France, April 24-26, 2017, Conference Track Proceedings_.
* a large-scale multi-domain wizard-of-oz dataset for task-oriented dialogue modelling. In _Proceedings of the 2018 Conference on Empirical Methods in Natural Language Processing (EMNLP)_.
* Chen et al. (2019) Wenhu Chen, Jianshu Chen, Pengda Qin, Xifeng Yan, and William Yang Wang. 2019. Semantically conditioned dialog response generation via hierarchical disentangled self-attention. In _Proceedings of the 57th Annual Meeting of the Association for Computational Linguistics_, pages 3696-3709, Florence, Italy. Association for Computational Linguistics.
* Devlin et al. (2019) Jacob Devlin, Ming-Wei Chang, Kenton Lee, and Kristina Toutanova. 2019. Bert: Pre-training of deep bidirectional transformers for language understanding. In _Proceedings of the 2019 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, Volume 1 (Long and Short Papers)_, pages 4171-4186.
* Eric et al. (2019) Mihail Eric, Rahul Goel, Shachi Paul, Abhishek Sethi, Sanchit Agarwal, Shuyag Gao, and Dile Hakkanir Tur. 2019. MultiwoZ 2.1: Multi-domain dialogue state corrections and state tracking baselines. _arXiv preprint arXiv:1907.01669_.
* Gupta et al. (2018) Sonal Gupta, Rushin Shah, Mrinal Mohit, Anuj Kumar, and Mike Lewis. 2018. Semantic parsing for task oriented dialog using hierarchical representations. In _Proceedings of the 2018 Conference on Empirical Methods in Natural Language Processing_, pages 2787-2792, Brussels, Belgium. Association for Computational Linguistics.
* Jiang et al. (2019) Zhuoxuan Jiang, Ziming Huang, Dong Sheng Li, and Xiauling Mao. 2019. Dialogact2vec: Towards end-to-end dialogue agent by multi-task representation learning. _arXiv preprint arXiv:1911.04088_.
* Kendall et al. (2018) Alex Kendall, Yarin Gal, and Roberto Cipolla. 2018. Multi-task learning using uncertainty to weigh losses for scene geometry and semantics. In _Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition_, pages 7482-7491.
* Kingma and Ba (2015) Diederik P. Kingma and Jimmy Ba. 2015. Adam: A method for stochastic optimization. In _3rd International Conference on Learning Representations, ICLR 2015, San Diego, CA, USA, May 7-9, 2015, Conference Track Proceedings_.
* Li et al. (2016a) Jiwei Li, Will Monroe, Alan Ritter, Dan Jurafsky, Michel Galley, and Jianfeng Gao. 2016a. Deep reinforcement learning for dialogue generation. In _Proceedings of the 2016 Conference on Empirical Methods in Natural Language Processing, EMNLP 2016_, pages 1192-1202.
* Liu et al. (2016b) Bing Liu and Ian Lane. 2017. An end-to-end trainable neural network model with belief tracking for task-oriented dialog. In _Interspeech 2017, 18th Annual Conference of the International Speech Communication Association, Stockholm, Sweden, August 20-24, 2017_, pages 2506-2510. ISCA.
* Mehri et al. (2019) Shikhib Mehri, Tejas Srinivasan, and Maxine Eskenazi. 2019. Structured fusion networks for dialog. _arXiv preprint arXiv:1907.10016_.
* Mi et al. (2016) Haitao Mi, Baskaran Sankaran, Zhiguo Wang, and Abe Ittycheriah. 2016. Coverage embedding models for neural machine translation. _arXiv preprint arXiv:1605.03148_.
* Mrksic et al. (2017) Nikola Mrksic, Diarmid O Seaghdha, Tsung-Hsien Wen, Blaise Thomson, and Steve Young. 2017. Neural belief tracker: Data-driven dialogue state tracking. In _Proceedings of the 55th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers)_, pages 1777-1788, Vancouver, Canada. Association for Computational Linguistics.
* Papineni et al. (2002) Kishore Papineni, Salim Roukos, Todd Ward, and Wei-Jing Zhu. 2002. Bleu: a method for automatic evaluation of machine translation. In _Proceedings of the 40th annual meeting on association for computational linguistics_, pages 311-318. Association for Computational Linguistics.
* Paulus et al. (2018) Romain Paulus, Caiming Xiong, and Richard Socher. 2018. A deep reinforced model for abstractive summarization. In _International Conference on Learning Representations_.
* Pei et al. (2019) Jiahuan Pei, Pengjie Ren, and Maarten de Rijke. 2019. A modular task-oriented dialogue system using a neural mixture-of-experts. _arXiv preprint arXiv:1907.05346_.
* Rastogi et al. (2018) Abhinav Rastogi, Raghav Gupta, and Dilek Hakkanir-Tur. 2018. Multi-task learning for joint language understanding and dialogue state tracking. In _Proceedings of the 19th Annual SIGdial Meeting on Discourse and Dialogue_, pages 376-384.
* Serban et al. (2017) Iulian V Serban, Chinnadhurai Sankar, Mathieu Germain, Saizheng Zhang, Zhouhan Lin, Sandeep Subramanian, Taesup Kim, Michael Pieper, Sarath Chandar, Nan Rosemary Ke, et al. 2017. A deep reinforcement learning chatbot. _arXiv preprint arXiv:1709.02349_

Anuroop Sriram, Heewoo Jun, Sanjeev Satheesh, and Adam Coates. 2018. Cold fusion: Training seq2seq models together with language models. In _6th International Conference on Learning Representations, ICLR 2018, Vancouver, BC, Canada, April 30 - May 3, 2018, Workshop Track Proceedings_.
* Su et al. (2017) Pei-Hao Su, Pawel Budzianowski, Stefan Ultes, Milica Gasic, and Steve Young. 2017. Sample-efficient actor-critic reinforcement learning with supervised data for dialogue management. In _Proceedings of the 18th Annual SIGdial Meeting on Discourse and Dialogue_, pages 147-157.
* Su et al. (2018) Shang-Yu Su, Kai-Ling Lo, Yi-Ting Yeh, and Yun-Nung Chen. 2018. Natural language generation by hierarchical decoding with linguistic patterns. In _Proceedings of the 2018 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, Volume 2 (Short Papers)_, pages 61-66, New Orleans, Louisiana. Association for Computational Linguistics.
* Tu et al. (2016) Zhaopeng Tu, Zhengdong Lu, Yang Liu, Xiaohua Liu, and Hang Li. 2016. Modeling coverage for neural machine translation. _arXiv preprint arXiv:1601.04811_.
* Vaswani et al. (2017) Ashish Vaswani, Noam Shazeer, Niki Parmar, Jakob Uszkoreit, Llion Jones, Aidan N Gomez, Lukasz Kaiser, and Illia Polosukhin. 2017. Attention is all you need. In _Advances in neural information processing systems_, pages 5998-6008.
* Wen et al. (2015) Tsung-Hsien Wen, Milica Gasic, Nikola Mrksic, Pei-Hao Su, David Vandyke, and Steve Young. 2015. Semantically conditioned LSTM-based natural language generation for spoken dialogue systems. In _Proceedings of the 2015 Conference on Empirical Methods in Natural Language Processing_, pages 1711-1721, Lisbon, Portugal. Association for Computational Linguistics.
* Wen et al. (2017) Tsung-Hsien Wen, Yishu Miao, Phil Blunsom, and Steve Young. 2017. Latent intention dialogue models. In _Proceedings of the 34th International Conference on Machine Learning-Volume 70_, pages 3732-3741. JMLR. org.
* Williams et al. (2017) Jason D Williams, Kavosh Asadi, and Geoffrey Zweig. 2017. Hybrid code networks: practical and efficient end-to-end dialog control with supervised and reinforcement learning. In _Proceedings of the 55th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers)_, pages 665-677.
* Wu et al. (2019) Chien-Sheng Wu, Andrea Madotto, Ehsan Hosseini-Asl, Caiming Xiong, Richard Socher, and Pascale Fung. 2019. Transferable multi-domain state generator for task-oriented dialogue systems. In _Proceedings of the 57th Conference of the Association for Computational Linguistics, ACL 2019, Florence, Italy, July 28- August 2, 2019, Volume 1: Long Papers_, pages 808-819.
* Zhang et al. (2019) Yizhe Zhang, Siqi Sun, Michel Galley, Yen-Chun Chen, Chris Brockett, Xiang Gao, Jianfeng Gao, Jingjing Liu, and Bill Dolan. 2019. Dialogopt: Large-scale generative pre-training for conversational response generation. _arXiv preprint arXiv:1911.00536_.
* Zhao et al. (2019) Tiancheng Zhao, Kaige Xie, and Maxine Eskenazi. 2019. Rethinking action spaces for reinforcement learning in end-to-end dialog agents with latent variable models. In _Proceedings of the 2019 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, Volume 1 (Long and Short Papers)_, pages 1208-1218.
* Zhong et al. (2018) Victor Zhong, Caiming Xiong, and Richard Socher. 2018. Global-locally self-attentive dialogue state tracker. _arXiv preprint arXiv:1805.09655_.