System-Initiated Transitions from Chit-Chat to Task-Oriented Dialogues with Transition Info Extractor and Transition Sentence Generator (2023)

Abstract: In this work, we study dialogue scenarios that start from chit-chat but eventually switch to task-related services, and investigate how a unified dialogue model, which can engage in both chit-chat and task-oriented dialogues, takes the initiative during the dialogue mode transition from chit-chat to task-oriented in a coherent and cooperative manner. We firstly build a transition info extractor (TIE) that keeps track of the preceding chit-chat interaction and detects the potential user intention to switch to a task-oriented service. Meanwhile, in the unified model, a transition sentence generator (TSG) is extended through efficient Adapter tuning and transition prompt learning. When the TIE successfully finds task-related information from the preceding chit-chat, such as a transition domain (“train” in Figure fig: system-initiated transition from chit-chat to task-oriented.), then the TSG is activated automatically in the unified model to initiate this transition by generating a transition sentence under the guidance of transition information extracted by TIE. The experimental results show promising performance regarding the proactive transitions. We achieve an additional large improvement on TIE model by utilizing Conditional Random Fields (CRF). The TSG can flexibly generate transition sentences while maintaining the unified capabilities of normal chit-chat and task-oriented response generation.

## introduction spoken dialogue systems (sdss) have usually been developed targeting only one out of two different categories, task-oriented or chit-chat (aka opendomain). the former focuses on achieving functional goals and the latter aims at creating engaging social conversations without special goals. in recent years, several previous works @xcite @xcite have studied unified conversational models that can engage in both chit-chat and task-oriented dialogue. however, the system-initiated transitions that emerge during switchover between these two dialogue modes have rarely been explored. especially when a user chats casually with the dialogue system, but implicitly expresses a need for a specific task-related service, it is desired that the dialogue system is able to capture this hidden information and proactively ask the user if they require such a task-oriented service (like booking a train ticket in figure 1 ). it has been proven to be beneficial for commercial sdss to proactively offer or sell their task-related services @xcite . furthermore, these transitions smoothly initiated by the dialogue system are regarded as a proactive feature @xcite and can greatly improve the user interaction experience @xcite . the goal of this work is to develop the initiative capabilities of a unified conversational model that is capable of detecting the implicit user intention of using some task-related services, even if they are talking casually, and to proactively bridge the connection from chit-chat to task-oriented dialogue through generating a transition sentence (red in figure 1 ). as the dialogue example in figure 1 shows, the original response at the transition turn is only "i see". if the agent can anticipate in advance that the user wants to visit the "london kings cross" through the preceding chit-chat, it can then proactively establish a connection with the task-oriented "train" service that the user needs by saying "if you want, i can look for a train to london kings cross for you.". to enable the initiative capabilities in a unified model, the main contributions of this paper are as follows: the overall architecture flow of this work is shown in figure 2 . when the tie successfully extracts the transition information from the preceding chit-chat, the tsg in the unified nlg is activated to generate a transition sentence besides the normal response to proactively guide this switch. the combined flow is highlighted in red. otherwise, the tie continually tracks the chit-chat, and unified nlg works as usual to generate a normal chit-chat or task-oriented response without (w/o) a transition sentence. the remainder of this paper is structured as follows: section 2 shows related work of our research. section 3 presents the transition sentence augmentation and templates for the tsg training. section 4 introduces the proposed tie model for detecting the task-related transition information from the preceding chit-chat interaction. section 5 presents the unified nlg extended with tsg through transition prompt and adapter tuning. section 6 elaborates on the performance evaluation of this work. section 7 concludes this work and outlines future research. ## related works nlu is generally a crucial component in taskoriented sdss and responsible for parsing an utterance into a semantic frame to identify the user's intention @xcite . with the development of deep learning methods, rnn, cnn, as well as their variations or combinations have been widely for the nlu task @xcite @xcite . @xcite proposed a attentionbased encoder-decoder, cnn-blstm, for joint intent detection and slot filling. @xcite proposed a slot gate that focused on capturing the relationship between slot and intent. @xcite and @xcite both used the pre-trained bert for the joint intent classification and slot filling. the proposed tie is inspired by nlu modeling. beyond that, @xcite and @xcite both utilized the traditional approach, conditional random fields (crf) @xcite , for sequence labelling with the combination of lstm and cnn. we also leverage the crf technology to further improve the performance of the tie model. @xcite introduced the dodecadialogue task, to assemble important aspects of an engaging conversational agent into a single collection by leveraging 12 tasks. adapter-bot @xcite utilized multiple adapter layers with the pretrained dialogpt model to activate new response skills and styles. @xcite proposed a dialogue model for training chit-chat and task-oriented in a unified data schema, which both include belief states, representation of dataset results, and system acts. however, these models simply fuse chit-chat dialogue and task-oriented dialogue into ## transition sentence augmentation and templates this section introduces the details of human augmentation of transition sentences and template collection for unannotated dialogues. we mainly utilize the prepended fusedchat @xcite dataset for initiative transitions from chit-chat to task-oriented in this work. fused-chat is a public available dataset, where human augmented open-domain dialogues are prepended and appended to the dialogues of the popular taskoriented dataset multiwoz @xcite . in the prepended fusedchat, each dialogue starts with chit-chat interaction and eventually switch to task-oriented requests. to task-oriented services. 2 however, our goal is to build a proactive dialogue system that can establish a smooth transition from chit-chat to task-oriented by itself. to achieve this, we hire one master student with computational linguistics background to augment a domain guided transition sentence and a domainslot-value guided transition sentence (red sentence in figure 1 ) for 215 prepended fusedchat dialogues respectively. the domain guided transition sentence must explicitly include the domain information. the domain-slot-value guided transition sentence must contain the specific value extracted from the preceding chit-chat dialogue aside from the domain, as the transition sentence in figure 1 , "if you want, i can look for a train to london kings cross for you." with "train" domain and "london kings cross" value. after the human augmentation, we collect the templates for transition sentences in different domains and different domain-slot pairs from the augmented 215 dialogues respectively. for the domain-slot-value guided transition sentences, we use "[value]" to replace the specific value to collect the domain-slot templates. ## transition info extractor (tie) this section presents our tie model that can detect potential user intention to switch to task-oriented services. as shown in figure 3 , tie is adapted from pre-trained roberta @xcite and has three components, a transition domain classifier, a transition slot classifier and a slot filling layer. when the interaction starts from chit-chat, the tie keeps track of the preceding chit-chat to predict the potential transition domain and slot, while extracting the specific value through slot filling. for instance, the transition domain-slot-value extracted in figure 3 , is "restaurant-food-korean restaurant". ## unified nlg extended with transition sentence generator (tsg) this section firstly introduces the unified nlg model that can reply to both chit-chat and taskoriented requests. then we mainly elaborate on the tsg integrated in unified nlg through efficient adapter tuning and transition prompt technologies. the extended nlg with tsg can generate a transition sentence given the transition information extracted by tie to enable the system-initiated transition. the details of unified nlg and the extension with tsg are shown in figure 4 . ## results comparison this section evaluates this work and provides detailed performance comparison from different perspectives. we firstly evaluate different tie models and different generation models separately with test prepended fusedchat. then we further evaluate the combined performance of the best tie model and generation model only at transition turns. ## conclusion this work investigates the dialogue transition from chit-chat to task-oriented initiated by a dialogue agent. we build a tie model adapted from pretrained roberta to keep track of the preceding chit-chat and predict transition domain, slot, while extracting the specific value from the chit-chat history via slot filling task. a unified generation model adapted from the pre-trained gpt-2 is built and extended its proactive capability for transition sentence generation through efficient adapter tuning and transition prompt learning. our proposed work shows promising performance both on transition information extraction and transition sentence generation. we will continue working on system-initiated transitions in other dialogue scenarios in the future.