From Cambridge to Pisa: A Journey into Cross-Lingual Dialogue Domain Adaptation for Conversational Agents
Authors: Tiziano Labruna, Bernardo Magnini (Fondazione Bruno Kessler, Libera Università di Bolzano, 2021)
Topic: Chatbots and Dialogue Systems

Abstract
Domain and language shift are still major bottlenecks for a vast range of task-oriented dialogue systems. This paper focuses on data-driven models for dialogue state tracking, and builds on top of recent work on dialogue domain adaptation, showing that state-of-the-art models are very sensible to language shift obtained through automatic translation. Experiments show that combining training data for the two languages (English and Italian) is always beneficial, while combining domains does not increase performance. As a relevant side effect of our work, we present a new dataset for dialogue state tracking available for Italian, derived from MultiWOZ 2.3.

Introduction
This paper is mainly motivated by the interest of exploring, and improving, the capacity of current data-driven task-oriented conversational systems to address shifts of domain and changes of language. Our starting point is the dialogue domain adaptation (DDA) approach proposed by  <cit.>, which allows to adapt training dialogues collected for a source domain knowledge (e.g., restaurants in Cambridge) to a domain where certain changes (e.g., a new restaurant opens, a restaurants changes its food , etc.) have occurred. The idea behind DDA is, that, rather then trying to improve the model robustness, it is worth to generate new training dialogues that are consistent with the domain changes. In this paper we extend and experiment DDA, so that also changes of language are included, particularly moving from English to Italian.   A further motivation for our work is related to the scarcity of annotated data for task-oriented dialogues systems for the Italian language.  Among the dialogic resources originally collected in Italian, we mention the recent JILDA dataset <cit.> <cit.>, a corpus of dialogues in the domain of job offers, semantically annotated. NESPOLE <cit.> is a corpus of human-human dialogues, which is not annotated for training purposes. The LUNA dataset <cit.>, although annotated with frame information, is not publicly available. Finally, the SUGAR dataset was presented at Evalita 2018 <cit.> allowing to train systems able to interpret spoken commands in Italian for the preparation of recipes.     A second type of data is derived through automatic translation from English. This is the case of the Italian version of the SNIPS dataset <cit.>, which provides annotations for spoken language understanding and can be used for slot filling and intent detection. Similarly, an Italian version of the SQuAD question-answer pair corpus is available <cit.> for Italian through translation. It has to be remarked that the Italian translations of both SNIPS and SQuAD do not rely on task-oriented dialogues, rather just on question-answer pairs. In addition, they are produced through simple translation from English, i.e., preserving domain information (e.g., names of places, restaurants, hotels, etc. reflect the English domain). We also notice that, unfortunately, the recent MultiATIS++ dataset <cit.>, does not include Italian among the languages.  We are particularly interested in data-driven approach of dialogue state tracking (DST) <cit.> for the Italian language. DST captures the capacity of a model to predict the correct dialogue state at each turn in a dialogue, representing both the communicative goals (dialogue acts) of the user and the portion of domain knowledge involved in such goals (slot-value pairs). To the best of our knowledge, the only dataset in Italian that can be used to model dialogue state tracking is JILDA <cit.>, where dialogue state annotations were carried on following the MultiWOZ style. However, being concluded very recently, still there are no available DST baselines for JILDA, and, for this reason, we have developed an Italian version of the MultiWOZ dataset <cit.>.  Starting from MultiWOZ 2.3, a popular dataset in English developed for booking traveling facilities (e.g., restaurants, hotels, trains, attractions) in the area of Cambridge, we incrementally operated both language and domain shifts. We provide three experimental configurations: (i) a translation of the Cambridge data set into Italian; (ii) a domain shift from Cambridge to Pisa, maintaining English as language; and, finally, (3) a configuration where both the initial domain and the language are changed. As a relevant side effect, the datasets for the three configurations are now available for further research on dialogue state tracking for Italian[https://github.com/tLabruna/DDA].  In the paper we first introduce the relavant background in dialogue domain adaptation (Section 2), then we explain how dialogue domain adaptation is concretely applied to domain changes, and finally we report the experiments we have conducted (Section 4 and 5).

Conclusion
We have investigated domain and language shift for data-driven task-oriented dialogue systems. We have extended recent work on dialogue domain adaptation to a cross-language setting, where both the domain and the language are changed. We showed that: (i) state-of-the-art models are very sensible to language shift obtained through automatic translation; (ii) combining training data for the two languages is always beneficial; on the contrary, combining data of different domains does not produce any improvement in all of our settings. Finally, as a relevant side effect of our work, we present a new dataset for dialogue state tracking available for Italian, derived from MultiWOZ 2.3. All the data are made available for further research on dialogue domain adaptation.