Where to Go for the Holidays: Towards Mixed-Type Dialogs for Clarification of User Goals (2022)

Abstract: Most dialog systems posit that users have figured out clear and specific goals before starting an interaction. For example, users have determined the departure, the destination, and the travel time for booking a flight. However, in many scenarios, limited by experience and knowledge, users may know what they need, but still struggle to figure out clear and specific goals by determining all the necessary slots. In this paper, we identify this challenge, and make a step forward by collecting a new human-to-human mixed-type dialog corpus. It contains 5k dialog sessions and 168k utterances for 4 dialog types and 5 domains. Within each session, an agent first provides user-goal-related knowledge to help figure out clear and specific goals, and then help achieve them. Furthermore, we propose a mixed-type dialog model with a novel Prompt-based continual learning mechanism. Specifically, the mechanism enables the model to continually strengthen its ability on any specific type by utilizing existing dialog corpora effectively.

## introduction one of the overarching goals of artificial intelligence is to build an intelligent agent that can generate coherent multi-turn dialogs to meet user needs/goals. recently, multiple dialog agents have been launched, such as echo and siri. these agents usually position themselves as some kind of "do engines" that act under users' clear instructions. specifically, they posit users have figured out clear and specific goals by determining all the necessary aspects or slots of their goals. for example, before booking a flight, a user has determined the departure, the destination and the travel time. however, such assumption can not hold in many real-world scenarios. for example, a user wants to plan a trip to beijing for relaxing, but he or she only has limited knowledge about beijing. thus it is difficult for him or her to decide which slots are needed to achieve this goal. obviously, in this scene, the user needs additional consultant services from an agent to help figure out clear and specific goals. however, the aforementioned assumption hinders providing these services effectively. in this paper, we make a step towards solving the challenge. in order to facilitate the study of how to help users clarifying their goals, we construct a new dialog corpus at baidu, denoted as duclar-ifydial. 1 as shown in figure 1 , a user chats about "feels anxious" because of work pressure, and wants to relax himself or herself but have no clear idea about the trip. in the scenario, the agent conducts knowledge-grounded dialogs and question answering conversations to help the user learn more about goal-related knowledge, which helps figure out clear and specific goals. finally, the user determines to visit "wangfujing catholic church" and books a restaurant nearby. specifically, in duclarifydial, besides basic social chitchat, an agent should help users figure out clear and specific goals by providing goal-related knowledge through coherent knowledge-grounded dialogs and question answering (qa) conversations. then, upon request, it should also conduct task-oriented dialogs to help achieve user goals. to this end, we first collect a human-to-human mixed-type dialog dataset. it contains 5k dialog sessions and 168k utterances for 4 dialog types and 5 domains. specifically, each session contains at least two of following four dialog types, i.e., social chitchat, question answering, knowledge-grounded dialog, and task-oriented dialog. furthermore, in duclarifydial bot 最近咋样啊？ [how are you doing?] user 工作压力有点大，情绪也有点焦虑 [i get stressed out at work and feel anxious. ] bot 我建议你出去旅游，可能对你有帮助 [how about go on a trip for relaxing?] user 好主意，想去北京转转，有没有什么还不错的地方可以玩玩？ [good idea. i want to go to beijing. is there any good place for fun?] bot 当然有了。最近很多人去王府井天主教堂玩。[of course. recently, many people visit the wangfujing catholic church.] user 要不你帮我在简单介绍介绍这个教堂呗 [please give me a brief introduction to this church.] bot 这个教堂始建于公元1655年... [the church was founded in 1655 …] user 这个王府井天主教堂经历不少啊 [wow! what a colorful history this church has.] bot 不仅如此，整个建筑为砖木结构... [in addition, the whole building is of brick and wood structure.] user 听起来很有意思哦 [sounds great] ... bot 顺便说一下，景区附近的美食驴打滚也很不错哦 [by the way, there are delicious food nearby, e.g. lvdagunr.] user 为什么叫驴打滚呢 [what a strange name it is!] bot 因其最后制作工序中… [this name comes from its final production process …] user 听着都挺香的，过去了一定得尝一下！先帮我定下餐厅吧 [it sounds delicious. i want it! please book a restaurant for me.] ... bot 好的，已经帮你预订好了[ok, i've made a reservation for you.] ... social chitchat knowledge grounded dialog question answering task-oriented dialog multiwoz user i am looking for a attraction in the east of town. bot sure, i can help you with that. there are plenty of attracts to the east. is there a specific type of attraction you are interested in? … multiwoz-beyond user i need a train from king's lynn to cambridge arriving by 17:45 on sunday. bot i have train tr6003 leaving cambridge at 5:11 and arriving at 5:58 on sunday. user i also need to bring my dog. do they allow pets? bot yes, you can travel with your dog up to 20 pounds. user that sounds great. i also need help finding a place to stay in the south. ## the duclarifydial dataset duclarifydial is designed to collect a high quality mixed-type dialog dataset for helping figure out clear and specific goals. in duclarifydial, one person serves as the user and the other as the wizard (agent). in order to help figure out clear and specific goals, besides social chitchat, the agent provides user-goal-related information through knowledge grounded dialogs and qa conversations, and then help achieve the goals through task-oriented dialogs. specifically, in order to effectively weave together multi types of dialogs for achieving this purpose, it is essential for different types of dialogs to share the same state space and action space. thus, in section 3.4, we utilize a unified dialog state schema and dialog act schema for the aforementioned four types of dialogs. in the following, we will introduce the four steps of duclarifydial collection: (1) building knowledge base to provide goal-related information; (2) constructing dialog templates to assist dialog collection; (3) collecting conversation utterances by crowdsourcing; (4) annotating dialog states and dialog acts. ## the mixed-type dialog model with prompt-based mechanism recently, large scale pre-trained dialog models have achieved impressive performance, both in task-oriented dialog @xcite and open-domain chitchat @xcite @xcite . meanwhile, the methodologies for different types of dialogs have gradually shifted to generative and end-to-end modelling. following these trends, we propose a pre-trained mixed-type dialog model based on @xcite , denoted as plato-mt. furthermore, we equip our model with a novel promptbased continual learning mechanism to strengthen the model ability by continually utilizing external existed different types of dialog corpora. ## duclarifydial as a new benchmark we break down the mixed-type dialog modelling task into three sub-tasks: dialog state tracking, dialog act planning, and dialog-act-to-text generation. besides, in order to facilitate end-to-end dialog modelling, we define an end-to-end dialogcontext-to-text generation sub-task. for each of the four sub-tasks, we report benchmark results on the following dialog models, which have achieved promising performance in the popular multiwoz dataset @xcite . specifically, we use the original codes released by the authors. ubar @xcite ubar is a fully endto-end task-oriented dialog model that takes a pretrained model as backbone. here, since duclari-fydial is a chinese dataset, we utilize a chinese large-scale pre-trained model, ernie @xcite , to initialize ubar. mintl @xcite mintl is a strong model that utilizes effective transfer learning to plug-andplay pre-trained models. here, instead of utilizing bart @xcite as in the original paper, we utilize the multi-lingual version, mbart @xcite , for initialization. plato @xcite ) plato is the state-ofthe-art chinese pre-trained dialog model. we use the released parameters. foot_9 plato-mt it is the proposed unified mixed-type dialog model with prompt-based continual learning mechanism. here, the prompt-related parameters are random initialized. plato-mt w/o prompt it is the plato-mt model without prompting. we first fine-tune it on the same set of existing dialog corpus as in plato-mt, and then fine-tune it on duclarifydial. ## conclusion in this paper, we first identify the challenge that users may struggle to figure out clear and specific goals in many real scenarios. then, we make a step forward by collecting a new human-to-human mixed-type dialog corpus, which contains 5k dialog sessions and 168k utterances for 4 dialog types and 5 domains. furthermore, we setup benchmarks based on the corpus. moreover, we propose a mixed-type dialog generation model with a novel prompt-based continual learning mechanism. finally, experimental results demonstrate the effectiveness of the mechanism.