XQA - DST : Multi-Domain and Multi-Lingual Dialogue State Tracking (2023)

Abstract: Dialogue State Tracking (DST), a crucial component of task-oriented dialogue (ToD) systems, keeps track of all important information pertaining to dialogue history: filling slots with the most probable values throughout the conversation. Existing methods generally rely on a predefined set of values and struggle to generalise to previously unseen slots in new domains. To overcome these challenges, we propose a domain-agnostic extractive question answering (QA) approach with shared weights across domains. To disentangle the complex domain information in ToDs, we train our DST with a novel domain filtering strategy by excluding out-of-domain question samples. With an independent classifier that predicts the presence of multiple domains given the context, our model tackles DST by extracting spans in active domains. Empirical results demonstrate that our model can efficiently leverage domain-agnostic QA datasets by two-stage fine-tuning while being both domain-scalable and open vocabulary in DST. It shows strong transferability by achieving zero-shot domain-adaptation results on MultiWOZ 2.1 with an average JGA of 36.7%. It further achieves cross-lingual transfer with state-of-the-art zero-shot results, 66.2% JGA from English to German and 75.7% JGA from English to Italian on WOZ 2.0.

## introduction task-oriented dialogue systems are designed to provide natural conversation with users and assist them in achieving daily goals. with the growth of task-oriented dialogue systems, there is an increasing interest in supporting dialogues among many domains and languages to fit the users' demands. however, either modelling a multi-domain or multilingual dialogue system requires substantial data collected in real scenarios. this data acquisition procedure is extremely expensive, and it motivates us to resolve this challenge by leveraging dialogue data in rich-resource domains and languages via zero-shot transfer learning. dialogue state tracking (dst) is crucial for accurately extracting user intents and goals over multiple turns within the dialogue. based on the tracked dialogue states, the dialogue manager makes corresponding next actions with back-end results, where the accuracy of the dst becomes absolutely vital. with a fully predefined ontology, traditional approaches tackle the dst as a classification problem by enumerating every combination of slot-value pairs @xcite . those approaches are strongly limited by their scalability, as some slots (e.g. name) have an unbounded set of slot values. secondly, they are generally not flexible to unseen slot-value pairs, making them more difficult to adapt to zero-shot transfer learning. moreover, a completely predefined ontology is hard to acquire and not scalable for tod systems in real applications. to overcome those challenges, we take inspiration from @xcite and investigate how dst can be tackled by extracting slot values from user utterances directly. in this paper, we propose a domain-independent and transferable dialogue state tracker within an extractive question answering architecture. our model is responsible for filling the slot value by recognising specially designed domain-slot prompts by span prediction, which extracts answers from the input utterance by predicting the token positions. in addition, we introduce a novel domain filtering strategy in training and an independent multi-domain classifier in evaluation such that we only ask slot questions that appear in predicted domains. for example, given hotel as the current turn domain, all questions under the train domain are filtered out as there is no overlap between them. this simple but effective filtering strategy significantly reduces the noise from unnec-essary questions in both the training and evaluation phases. furthermore, we study unexplored impacts of two-stage fine-tuning on dst transfer learning with mono-lingual and multi-lingual question answering datasets. we call the final model xqa-dst: xlm-r based dialogue state tracker in question answering. our main contributions are summarised below: ## related work traditional dialogue state tracking approaches mostly rely on a predefined ontology. @xcite implement a slot-utterance matching module that computes the similarity between the utterance and each slot-value pair. lai et al. (2020) use bert @xcite as the context encoder and generate the relevance score for every pair. recently, @xcite and @xcite include schema graph networks to utilise inter-slot relationships. however, their scalability is strongly limited by the availability of the predefined ontology and schema graphs. to improve efficiency, span prediction methods have been proposed to tackle dst so that the slot can be filled by directly addressing values in the context. @xcite implement copy mechanisms, but they use independent span projection layers for each slot, which make their model incapable of inference in new domains. @xcite and @xcite formulate the dst as a question answering problem, and they prepare questions for asking the model to answer values for every slot. we differentiate from these approaches by disentangling the complex domain information from domain filtering and domain classification strategies. generative approaches @xcite provide an alternative way to handle dst. @xcite introduce a generative question answering approach, gpt2-m, that leverages an autoregressive language model. similarly, @xcite propose t5dst, and they study the impacts of slot descriptions and cross-task transfer on domain adaptation. @xcite reformulate dst as prompting states via schema descriptions from language models. recent end-to-end dialogue models @xcite also show strong supervised performance on dst. cross-lingual transfer learning for dst aims to leverage the labelled data in rich-resource languages and transfer learned knowledge to lowresource languages. @xcite study this problem and propose the xl-nbt teacherstudent framework. @xcite introduce an attention-informed mixed-language training (amlt) method to build code-switching training sentences. they study the effectiveness of multilingual pretrained language models, xlm (conneau and lample, 2019) and mbert @xcite , with their amlt approach. @xcite propose a data augmentation framework, which encourages cross-lingual alignment by fine-tuning mbert on generated code-switching data. @xcite introduce intermediate fine-tuning on parallel sentences to improve the cross-lingual dst. to the best of our knowledge, we are the first work that studies the effectiveness of a multi-lingual pretrained language model, xlm-r @xcite , on dst without implementing additional cross-lingual alignment strategies. ## conclusion we introduce a new multi-domain and multi-lingual dialogue state tracker, xqa-dst, within an extractive question answering framework. it gives distinct advantages for avoiding relying on any predefined ontology and being open-vocabulary to new slots with unseen values. we have shown a strong domain and cross-lingual transferable ability of our model by outperforming famous baselines. we have demonstrated its competitive performance in multi-domain dst with a novel turn-domain filtering strategy and a multi-domain classifier in parallel. with the design of an xlm-r based multidomain classifier, our approach is feasible for track-ing states in multi-domain and multi-lingual scenarios. therefore, it holds a strong potential to overcome the challenging data scarcity problem for either domains or languages in the real application of task-oriented dialogue systems.