SlowGuess/ACL_2020_OCR huggingface dataset:
To study the influence of the joint training and the dynamic act attention on response generation, we implement two pipeline approaches for comparison. We first train our act generator separately from response generation. Then, we keep its parameters fixed and train the response generator. The first baseline is created by replacing the dynamic act attention (Equation 7) with an average of the act hidden states, while the second baseline uses the dynamic act attention. As shown in Table 4,  $\mathrm{Pipeline}_2$  with dynamic act attention is superior to  $\mathrm{Pipeline}_1$  without it in all metrics, but inferior to the joint approach. Our joint model also surpasses the currently state-of-the-art pipeline system HDSA, even HDSA uses BERT. We find that by utilizing sequential acts, the dynamic act attention mechanism helps the response generator capture the local information by attending to different acts.
An illustrative example is shown in Figure 5, where the response generator can attend to the local information such as "day" and "stay" as needed when generating a response asking about picking a different day or shorter stay. We reckon that by utilizing sequential acts, response generation benefits in two ways. First, the dynamic act attention allows the generator to attend to different acts when
Figure 5: An illustrative example of the dynamic act attention mechanism. Response (row) subsequence can attend to the act (column) token "day" or "stay" as needed when generating a response asking about picking a different day or shorter stay.
Figure 6: Performance of the uncertainty loss and the weighted-sum loss on the development dataset.
generating a subsequence. Second, the joint training makes the two stages interact with each other, easing error propagation of pipeline systems.
6.3 Uncertainty Loss
We opt for an uncertainty loss to optimize our joint model, rather than a traditional weighted-sum loss. To illustrate their difference, we conduct an experiment on the development set. For the traditional loss (Equation 11), we run for each weight from 0 to 1 stepped by 0.1. Note that since the weights,  $\sigma_{1}$  and  $\sigma_{2}$ , in the uncertainty loss are not hyperparameters but learned internally to each batch, we only record the best score within each round without giving the values of  $\sigma_{1}$  and  $\sigma_{2}$ . As shown in Figure 6, the uncertainty loss can learn adaptive weights with consistently superior performance.
7 Human Evaluation
We conduct a human study to evaluate our model by crowd-sourcing. For this purpose we randomly selected 100 sample dialogues (742 turns in total) from the test dataset and constructed two groups of systems for comparison: MARCo vs. HDSA and
4The annotation results are available at https: //github.com/InitialBug/MarCo-Dialog/ tree/master/human.evaluation
7131
Figure 7: Results of human study in response quality. Two groups of systems are studied, where the top figure corresponds to results of MARCO vs. HDSA and the bottom figure represents MARCO vs. Human Response (ground-truth). "Win", "Tie" or "Lose" respectively indicate the proportions that our MARCO system wins over, ties with or loses to its counterpart.
MARCO vs. Human Response, where Human Response means the reference responses. Responses generated by each group were randomly assigned in pairs to 3 judges, who ranked them according to their completion and readability (Chen et al., 2019; Zhang et al., 2019). Completion measures if the response correctly answers a user query, including relevance and informativeness. Readability reflects how fluent, natural and consistent the response is.
The results of this study are shown in Figure 7, where "Win", "Tie" or "Lose" mean our MARCo system wins over, ties with or loses to its counterpart, respectively. From the results we note that MARCo outperforms HDSA and Human Response in completion, and ties  $94\%$  with HDSA in readability while underperforming Human Response. Overall speaking, MARCo is superior to HDSA and comparable with Human Response. We further analyzed the bad cases of our model in readability and found that our model slightly suffers from token level repetition, a problem that can be solved by methods like the coverage mechanism (Mi et al., 2016; Tu et al., 2016). In completion, our model can understand the users' need and tends to provide them more relevant information, so that they can finish their goals in shorter turns.
We present two examples in Figure 8. In the first example, the user requests the hotel type while HDSA ignores it. The user requests to book one ticket in the second example, yet both HDSA and Human Response ask about the number once again. In contrast, our model directly answers the questions with correct information. To sum up, MARCo successfully improves the dialogue system by generating relevant and informative responses.
Figure 8: Two examples to show that MARCo successfully improves the dialogue system by generating relevant and informative responses.
In this paper, we presented a novel co-generation model for dialogue act prediction and response generation in task-oriented dialogue systems. Unlike previous approaches, we modeled act prediction as a sequence generation problem to exploit the semantic structures of acts and trained it jointly with response generation via dynamic attention from response generation to act prediction. To train this joint model, we applied an uncertainty loss for adaptive weighting of the two tasks. Extensive studies were conducted on a large-scale task-oriented dataset to evaluate the proposed model, and the results confirm its effectiveness with very favorable performance over several state-of-the-art methods.
We thank the anonymous reviewers for their constructive reviews. This work was partially supported by the Fundamental Research Funds for the Central Universities (No.19lgpy220 and No.19lgpy219), the Program for Guangdong Introducing Innovative and Entrepreneurial Teams (No.2017ZT07X355), and the National Natural Science Foundation of China (No.61906217).
7132
Antoine Bordes, Y-Lan Boureau, and Jason Weston. 2017. Learning end-to-end goal-oriented dialog. In 5th International Conference on Learning Representations, ICLR 2017, Toulon, France, April 24-26, 2017, Conference Track Proceedings.
Pawel Budzianowski, Tsung-Hsien Wen, Bo-Hsiang Tseng, Inigo Casanueva, Ultes Stefan, Ramadan Osman, and Milica Gašić. 2018. Multiwoz - a large-scale multi-domain wizard-of-oz dataset for task-oriented dialogue modelling. In Proceedings of the 2018 Conference on Empirical Methods in Natural Language Processing (EMNLP).
Wenhu Chen, Jianshu Chen, Pengda Qin, Xifeng Yan, and William Yang Wang. 2019. Semantically conditioned dialog response generation via hierarchical disentangled self-attention. In Proceedings of the 57th Annual Meeting of the Association for Computational Linguistics, pages 3696-3709, Florence, Italy. Association for Computational Linguistics.
Jacob Devlin, Ming-Wei Chang, Kenton Lee, and Kristina Toutanova. 2019. Bert: Pre-training of deep bidirectional transformers for language understanding. In Proceedings of the 2019 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, Volume 1 (Long and Short Papers), pages 4171-4186.
Mihail Eric, Rahul Goel, Shachi Paul, Abhishek Sethi, Sanchit Agarwal, Shuyag Gao, and Dilek Hakkani-Tur. 2019. Multiwoz 2.1: Multi-domain dialogue state corrections and state tracking baselines. arXiv preprint arXiv:1907.01669.
Sonal Gupta, Rushin Shah, Mrinal Mohit, Anuj Kumar, and Mike Lewis. 2018. Semantic parsing for task oriented dialog using hierarchical representations. In Proceedings of the 2018 Conference on Empirical Methods in Natural Language Processing, pages 2787-2792, Brussels, Belgium. Association for Computational Linguistics.
Zhuoxuan Jiang, Ziming Huang, Dong Sheng Li, and Xianling Mao. 2019. Dialogact2vec: Towards end-to-end dialogue agent by multi-task representation learning. arXiv preprint arXiv:1911.04088.
Alex Kendall, Yarin Gal, and Roberto Cipolla. 2018. Multi-task learning using uncertainty to weigh losses for scene geometry and semantics. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pages 7482-7491.
Diederik P. Kingma and Jimmy Ba. 2015. Adam: A method for stochastic optimization. In 3rd International Conference on Learning Representations, ICLR 2015, San Diego, CA, USA, May 7-9, 2015, Conference Track Proceedings.
Jiwei Li, Will Monroe, Alan Ritter, Dan Jurafsky, Michel Galley, and Jianfeng Gao. 2016a. Deep reinforcement learning for dialogue generation. In Proceedings of the 2016 Conference on Empirical Methods in Natural Language Processing, EMNLP 2016, pages 1192-1202.
Jiwei Li, Will Monroe, Alan Ritter, Dan Jurafsky, Michel Galley, and Jianfeng Gao. 2016b. Deep reinforcement learning for dialogue generation. In Proceedings of the 2016 Conference on Empirical Methods in Natural Language Processing, pages 1192-1202.
Bing Liu and Ian Lane. 2017. An end-to-end trainable neural network model with belief tracking for task-oriented dialog. In *Interspeech* 2017, 18th Annual Conference of the International Speech Communication Association, Stockholm, Sweden, August 20-24, 2017, pages 2506-2510. ISCA.
Shikib Mehri, Tejas Srinivasan, and Maxine Eskenazi. 2019. Structured fusion networks for dialog. arXiv preprint arXiv:1907.10016.
Haitao Mi, Baskaran Sankaran, Zhiguo Wang, and Abe Ittycheriah. 2016. Coverage embedding models for neural machine translation. arXiv preprint arXiv:1605.03148.
Nikola Mrkšić, Diarmuid Žáeghdha, Tsung-Hsien Wen, Blaise Thomson, and Steve Young. 2017. Neural belief tracker: Data-driven dialogue state tracking. In Proceedings of the 55th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), pages 1777-1788, Vancouver, Canada. Association for Computational Linguistics.
Kishore Papineni, Salim Roukos, Todd Ward, and Wei-Jing Zhu. 2002. Bleu: a method for automatic evaluation of machine translation. In Proceedings of the 40th annual meeting on association for computational linguistics, pages 311-318. Association for Computational Linguistics.
Romain Paulus, Caiming Xiong, and Richard Socher. 2018. A deep reinforced model for abstractive summarization. In International Conference on Learning Representations.
Jiahuan Pei, Pengjie Ren, and Maarten de Rijke. 2019. A modular task-oriented dialogue system using a neural mixture-of-experts. arXiv preprint arXiv:1907.05346.
Abhinav Rastogi, Raghav Gupta, and Dilek Hakkani-Tur. 2018. Multi-task learning for joint language understanding and dialogue state tracking. In Proceedings of the 19th Annual SIGdial Meeting on Discourse and Dialogue, pages 376-384.
Iulian V Serban, Chinnadhurai Sankar, Mathieu Germain, Saizheng Zhang, Zhouhan Lin, Sandeep Subramanian, Taesup Kim, Michael Pieper, Sarath Chandar, Nan Rosemary Ke, et al. 2017. A deep reinforcement learning chatbot. arXiv preprint arXiv:1709.02349.
7133
Anuroop Sriram, Heewoo Jun, Sanjeev Satheesh, and Adam Coates. 2018. Cold fusion: Training seq2seq models together with language models. In 6th International Conference on Learning Representations, ICLR 2018, Vancouver, BC, Canada, April 30 - May 3, 2018, Workshop Track Proceedings.
Pei-Hao Su, Paweł Budzianowski, Stefan Ultes, Milica Gasic, and Steve Young. 2017. Sample-efficient actor-critic reinforcement learning with supervised data for dialogue management. In Proceedings of the 18th Annual SIGdial Meeting on Discourse and Dialogue, pages 147-157.
Shang-Yu Su, Kai-Ling Lo, Yi-Ting Yeh, and YunNung Chen. 2018. Natural language generation by hierarchical decoding with linguistic patterns. In Proceedings of the 2018 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, Volume 2 (Short Papers), pages 61-66, New Orleans, Louisiana. Association for Computational Linguistics.
Zhaopeng Tu, Zhengdong Lu, Yang Liu, Xiaohua Liu, and Hang Li. 2016. Modeling coverage for neural machine translation. arXiv preprint arXiv:1601.04811.
Ashish Vaswani, Noam Shazeer, Niki Parmar, Jakob Uszkoreit, Llion Jones, Aidan N Gomez, Lukasz Kaiser, and Illia Polosukhin. 2017. Attention is all you need. In Advances in neural information processing systems, pages 5998-6008.
Tsung-Hsien Wen, Milica Gašić, Nikola Mrkšić, PeiHao Su, David Vandyke, and Steve Young. 2015. Semantically conditioned LSTM-based natural language generation for spoken dialogue systems. In Proceedings of the 2015 Conference on Empirical Methods in Natural Language Processing, pages 1711-1721, Lisbon, Portugal. Association for Computational Linguistics.
Tsung-Hsien Wen, Yishu Miao, Phil Blunsom, and Steve Young. 2017. Latent intention dialogue models. In Proceedings of the 34th International Conference on Machine Learning-Volume 70, pages 3732-3741. JMLR.org.
Jason D Williams, Kavosh Asadi, and Geoffrey Zweig. 2017. Hybrid code networks: practical and efficient end-to-end dialog control with supervised and reinforcement learning. In Proceedings of the 55th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), pages 665-677.
Chien-Sheng Wu, Andrea Madotto, Ehsan Hosseini-Asl, Caiming Xiong, Richard Socher, and Pascale Fung. 2019. Transferable multi-domain state generator for task-oriented dialogue systems. In Proceedings of the 57th Conference of the Association for Computational Linguistics, ACL 2019, Florence, Italy, July 28-August 2, 2019, Volume 1: Long Papers, pages 808-819.
Yizhe Zhang, Siqi Sun, Michel Galley, Yen-Chun Chen, Chris Brockett, Xiang Gao, Jianfeng Gao, Jingjing Liu, and Bill Dolan. 2019. Dialogpt: Large-scale generative pre-training for conversational response generation. arXiv preprint arXiv:1911.00536.
Tiancheng Zhao, Kaige Xie, and Maxine Eskenazi. 2019. Rethinking action spaces for reinforcement learning in end-to-end dialog agents with latent variable models. In Proceedings of the 2019 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, Volume 1 (Long and Short Papers), pages 1208-1218.
Victor Zhong, Caiming Xiong, and Richard Socher. 2018. Global-locally self-attentive dialogue state tracker. arXiv preprint arXiv:1805.09655.
7134