Context-Aware Module Selection in Modular Dialog Systems (2023)

Abstract: In modular dialog systems, a dialog system consists of multiple conversational agents. The task “module selection” selects the appropriate sub-dialog system for an incoming user utterance. Current models for module selection use features derived from the current user turn only, such as the utterances text or confidence values of the natural language understanding systems of the individual conversational agents, or they perform text classification on the user utterance. However, dialogs often span multiple turns, and turns are embedded into a context. Therefore, looking at the current user turn only is a source of error in certain situations. This work proposes four models for module selection that include the dialog history and the current user turn into module selection. We show that these models surpass the current state of the art in module selection.

## introduction dialog systems (ds) often consist of multiple subdialog systems or modules. there are multiple reasons for such a combination: the designer of a ds might want to combine several existing ds without a reimplementation. sometimes a ds spans multiple departments and cannot be merged into a single, unified system. a hybrid system is a possible solution when a ds consists of multiple incompatible subsystems, e.g., a task-oriented ds and a questionanswering system. although this architecture is frequently used in practical applications, it is a gap in scientific research. the modular dialog system (mds) @xcite describes a framework to combine several dialog systems. in an mds, a central component called "module selection" (ms) selects the appropriate sub-ds that generates the answer for an incoming user utterance @xcite . ms is a classification task to choose one sub-ds from a list of sub-ds for a given user utterance. figure 1 : example dialog between a user and a modular dialog system that consists of two chatbots. based on the current utterance, the module selection can easily select the proper agent for the first user utterance. however, the module selection requires dialog context to classify the second user utterance correctly. current solutions for ms, such as @xcite or @xcite , focus on the current user utterance only. they use the text of the user utterance, confidence values of the models nlus, or additional features such as detected named entities for the models. however, both works showed that the text of the user utterance is the essential feature for high performance in ms @xcite . in some cases, more than the current user utterance and other derived features are needed to find the appropriate sub-ds. figure 1 shows an example mds that consists of a hotel reservation bot and a taxi reservation bot. the ms can easily categorize the first user utterance of the example dialog "i am looking for a hotel in the north of the city". however, the second user utterance "yes" alone does not transport enough information for the ms. therefore, we propose models to include the dialog history and the current user utterance into ms. we show that these models surpass state of the art in ms. ## background in this work, we use task-oriented dialog systems which "use conversation with users to help complete tasks" @xcite . @xcite define a turn as a "single contribution from one speaker of the dialog". the length of a turn is not fixed but can consist of a single utterance or up to multiple sentences. let u i be the ith turn of the user and s i the ith turn of the system. a dialog is a sequence of alternating user and system turns u 1 s 1 ... u n s n . @xcite describe a typical architecture for task-oriented dialog systems: each incoming user turn is first processed by natural language understanding (nlu), which converts the unstructured textual information of the user turn into structured information. most notable is intent detection, which classifies the user turn to a list of predefined intents. another standard function of the nlu is slot filling, which extracts slots from the user turn. slots are entities such as dates, names, or places. so, for example, for the user turn "i want to book a table for friday, 8 pm" the nlu can detect the intent "book table" and the slot "@xmath0. dialog state tracking processes the results of the nlu and keeps track of the slot values across the dialog. so in the restaurant booking domain, we might define slots time and number of people. during the dialog, dialog state tracking fills these slots with values. a dialog manager keeps track of the various states of the dialog. dialog managers can be hand-crafted or machine-learned. finally, the answer generation generates the system turn, which is shown to the user. mds and ms are similar to multidomain dialog systems (mdds) (see, e.g., @xcite ), in which a dialog system encompasses different domains. the multiwoz dataset was originally a dataset for mdds. however, the essential difference between mds and mdds is the motivation: in mdds, the goal is a dialog system with maximal performance, which can be implemented in a single, monolithic system. on the other hand, in mds, we want to distribute the system across several ds, which often results in a decreased per-formance @xcite . ## results ## discussion all three proposed models surpass state of the art (see at the same time, their results are very similar. we conclude that the most important contributions of the dialog history to the model's performance stem from the last turn (l2t model). including longer parts of the history (models l4t and fh) improves the performance only marginally. the f1-scores for the individual domains ( generally, the amount of training data is huge compared to the small number of domains and the limited range of the domains. the amount of generated training data would be lower in a practical use case due to the cost of training data generation. also, in a real-world scenario, the test data will be more diverse. so although we reached almost 100% f1-score in our experiments on this dataset, we do not believe that the task ms is solved. ## related works here we give an overview of ms. other approaches used features derived from the current user utterance only; @xcite and @xcite used a text classification on the current user utterance, which serves as the baseline model in our paper. @xcite compared various features for ms with each other, such as confidence values or slot values of the dialog systems nlu. however, these works do not utilize the dialog history for ms. the scientific literature proposes several approaches to combine multiple dialog systems. some authors @xcite use domain classification, which is similar to our framework, although their work stems from the mdds tradition and not from the mds tradition. another strategy is to let every dialog system generate a response and rank them to find the most suitable response @xcite . the very successful and feature-rich chatbot xiaoice uses a framework based on options over markov decision processes to decide which of his modules can answer the user utterance @xcite . the clara dialog system lets the user decide which of the sub-agents he wants to talk to @xcite . ## conclusion we proposed new models for ms that include the dialog history as a feature. these models show a high performance in our experiments, surpassing the state of the art. moreover, the performance is close to 100%, meaning that, at least on our dataset, the ms works almost perfectly. in the future, we plan to experiment more with our approach with less training data and on more diverse and challenging datasets.