G lobal W o Z : Globalizing M ulti W o Z to Develop Multilingual Task-Oriented Dialogue Systems (2022)

Abstract: Over the last few years, there has been a move towards data curation for multilingual task-oriented dialogue (ToD) systems that can serve people speaking different languages. However, existing multilingual ToD datasets either have a limited coverage of languages due to the high cost of data curation, or ignore the fact that dialogue entities barely exist in countries speaking these languages. To tackle these limitations, we introduce a novel data curation method that generates GlobalWoZ — a large-scale multilingual ToD dataset globalized from an English ToD dataset for three unexplored use cases of multilingual ToD systems. Our method is based on translating dialogue templates and filling them with local entities in the target-language countries. Besides, we extend the coverage of target languages to 20 languages. We will release our dataset and a set of strong baselines to encourage research on multilingual ToD systems for real use cases.

## introduction one of the fundamental objectives in pursuit of artificial intelligence is to enable machines with the ability to intelligently communicate with human in natural languages, with one of the widely-heralded applications being the task-oriented dialogue (tod) systems @xcite . recently, tod systems have been successfully deployed to assist users with accomplishing certain domain-specific tasks such as hotel booking, alarm setting or weather query @xcite @xcite , thanks to the joint advent of neural networks and availability of domain-specific data. however, most existing tod systems are predominately built for english, limiting their service for all of the world's citizens. the reason of this limitation lies in the stark lack of high-quality multilingual tod datasets due to the high expense and challenges of human annotation @xcite . one solution to this is annotating conversations in other languages from scratch, e.g., crosswoz @xcite and bitod @xcite . however, these methods involve expensive human efforts for dialogue collection in the other languages, resulting in a limited language/domain coverage. the other major line of work focused on translating an existing english tod dataset into target languages by professional human translators @xcite @xcite . despite the increasing language coverage, these methods simply translated english named entities (e.g., location, restaurant name) into the target languages, while ignored the fact that these entities barely exist in countries speaking these languages. this hinders a trained tod system from supporting the real use cases where a user looks for local entities in a target-language country. for example in figure 1 , a user may look for the british museum when traveling to london (a.), while look for the oriental pearl tower when traveling to shanghai (b.). in addition, prior studies @xcite have shown that code-switching phenomena frequently occurs in a dialogue when a speaker cannot express an entity immediately and has to alternate between two languages to convey information more accurately. such phenomena could be ubiquitous during the cross-lingual and cross-country task-oriented conversations. one of the reasons for code-switching is that there are no exact translations for many local entities in the other languages. even though we have the translations, they are rarely used by local people. for example in figure 1 (c.), after obtaining the recommendation from a tod system, a chinese speaker traveling to london would rather use the english c. use case f&e: a foreign language speaker travels to a country of english. d. use case e&f: a english speaker travels to a country of a foreign language. to bridge the aforementioned gap between existing data curation methods and the real use cases, we propose a novel data curation method that globalizes an existing multi-domain tod dataset beyond english for the three unexplored use cases. specifically, building on top of multiwoz @xcite ) -an english tod dataset for dialogue state tracking (dst), we create globalwoz, a new multilingual tod dataset in three new targetlanguages via machine translation and crawled ontologies in the target-language countries. our method only requires minor human efforts to post-edit a few hundred machine-translated dialogue templates in the target languages for evaluation. besides, as cross-lingual transfer via pre-2 see comparisons of these use cases in appendix a trained multilingual models @xcite @xcite has proven effective in many cross-lingual tasks, we further investigate another question: how do these multilingual models trained on the english tod dataset transfer knowledge to our globalized dataset? to answer this question, we prepare a few baselines by evaluating popular tod systems on our created test datasets in a zero-shot cross-lingual transfer setting as well as a few-shot setting. our contributions include the following: ## data curation methodology in order to globalize an existing english tod dataset for the three aforementioned use cases, we propose an approach consisting of four steps as shown in figure 2: (1) we first extract dialogue templates from the english tod dataset by replacing english-specific entities with a set of generalpurpose placeholders ( §2.1); (2) we then translate the templates to a target language for both training and test data, with one key distinction that we only post-edit the test data by professional translators to ensure the data quality for evaluation ( §2.2); (3) next, we collect ontologies @xcite containing the definitions of dialogue acts, local entities and their attributes in the target-language countries ( §2.3); (4) finally, we tailor the translated templates by automatically substituting the placeholders with entities in the extracted ontologies to construct data for the three use cases ( §2.4). ## proposed baselines we prepare a base model for globalwoz in the zero-shot and few-shot cross-lingual transfer settings. we select transformer-dst @xcite ) as our base model as it is one of the state-ofthe-art models on both multiwoz 2.0 and multi-woz 2.1 foot_4 . in our paper, we replace its bert encoder with an mbert encoder @xcite for our base model and propose a series of training methods for globalwoz. as detailed below, we propose several data augmentation baselines that create different training and validation data for training a base model. note that all the proposed baselines are model agnostic and the base model can be easily substituted with other popular models @xcite . for each baseline, we first train a base model on its training data for 20 epochs and use its validation set to select the best model during training. finally we evaluate the best model of each baseline on the same test set from globalwoz. we will release globalwoz and our pre-trained models to encourage faster adaptation to future research. we refer the readers to ## experiment results in this section, we show the results of all methods in the zero-shot ( §5.1) and few-shot ( §5.2) settings. ## conclusions in this paper, we provide an analysis on three unexplored use cases for multilingual task-oriented dialogue systems. we propose a new data curation method that leverages a machine translation system and local entities in target languages to create a new multilingual tod dataset, globalwoz. we propose a series of strong baseline methods and conduct extensive experiments on globalwoz to encourage research for multilingual tod systems. besides, we extend the coverage of languages on multilingual tod to 20 languages, marking the one step further towards building a globalized multilingual tod system for all of the world's citizen.