HCLD : A Hierarchical Framework for Zero-shot Cross-lingual Dialogue System (2022)

Abstract: Recently, many task-oriented dialogue systems need to serve users in different languages. However, it is time-consuming to collect enough data of each language for training. Thus, zero-shot adaptation of cross-lingual task-oriented dialog systems has been studied. Most of existing methods consider the word-level alignments to conduct two main tasks for task-oriented dialogue system, i.e., intent detection and slot filling, and they rarely explore the dependency relations among these two tasks. In this paper, we propose a hierarchical framework to classify the pre-defined intents in the high-level and fulfill slot filling under the guidance of intent in the low-level. Particularly, we incorporate sentence-level alignment among different languages to enhance the performance of intent detection. The extensive experiments report that our proposed method achieves the SOTA performance on a public task-oriented dialog dataset.

## introduction natural language understanding (nlu) plays a significant role in task-oriented dialogue systems, which is aimed to parse dialog utterances by identifying user's intent and the arguments of the intent @xcite . these two tasks are known as intent detection and slot filling respectively. for instance, in the utterance "next week's forecast" as shown in however, most works focus on monolingual datasets which are expensive to build. furthermore, some dialogue systems, e.g., google home and apple siri, need to serve numerous users around the world in different languages and might be faced with the scarcity of dialogue data in certain languages. thus, it's motivated to build cross-lingual dialogue systems that enable zero-shot adaptation from a high-resource language to a low-resource language without any training data in the target language. specifically, some models @xcite @xcite used cross-lingual pretrained embeddings to bridge different languages. some approaches like @xcite adopted a small vocabulary of translated word-pairs to align cross-lingual embeddings by bilingual projection and alleviate the inherent discrepancies among different languages. many existing methods @xcite utilized pretrained cross-lingual language model (pxlm), e.g., xlm @xcite , mbert @xcite and xlm-r @xcite , to derive contextual embeddings of words in different languages. nevertheless, most of existing methods consider embedding alignments among source and target languages at the word-level to fulfill the crosslingual adaptation, but they ignore that intent detection is a sentence classification task @xcite , which requires higher level alignments among different languages. furthermore, these two tasks are closely related and the slots highly depend on the intent @xcite , but their deep relations are not fully discovered by existing works. in light of these observations, we propose a model named hcld (a hierarchical framework for zero-shot cross-lingual dialogue system) to alleviate these defects. our approach is built upon a hierarchical framework to jointly accomplish intent detection and slot filling. it learns to classify pre-defined intents in the high-level of our model and fill the semantic slots under the guidance of the predicted intent in the low-level, which can help to find more related arguments of the intent to enhance the performance of slot filling. particularly, we also adopt a pretrained language agnostic bil-stm encoder laser @xcite to derive the sentence embeddings and directly use them for later classification task of intent detection, where the embeddings of sentences in different languages are aligned in the sentence-level. we conduct experiments on a public task-oriented dialogue dataset @xcite . the results show that our proposed method achieves state-ofthe-art performance on zero-shot adaptation. code-switching is used as data augmentation in an alternative data alignment method known as cosda @xcite . to make model training highly multilingual, random words in the input are translated and substituted, leading to increased cross-lingual transfer ability. there were additional attempts to learn how to code-switch automatically @xcite . our work combines the cross-lingual sentence embedding and word embedding to handle intent detection and slot filling respectively. intent detection and slot filling intent detection and slot filling are key tasks in the natural language understanding (nlu) of task-oriented dialogue systems. recently, some models @xcite consider to implement intent detection and slot filling jointly. the recent work @xcite first proposed a joint model to learn the correlation between intent and slots by rnn. co-gat @xcite investigated a non-autoregressive model for joint multiple intent detection and slot filling. recently, information of intent has been incorporated for slot filling. the prior work @xcite utilizes a slot-gated mechanism to model the relationship between two tasks. some approaches like @xcite propose the bi-model to consider the cross-impact between the intent and slots based on atis datasets @xcite . some models @xcite predict the intent based on each word and then feed into slot filling as input. compared with previous works, our approach firstly predicts the intent given by multi-lingual sentence representation, and directly incorporates the intent information of sentence for slot filling. furthermore, our model is handling simultaneously intent detection and slot filling at the cross-lingual setting. ## conclusion in this paper, we propose a hierarchical framework to classify the pre-defined intents in the high-level and fulfill slot filling under the guidance of intent in the low-level. particularly, we adopt sentence-level alignments to improve the performance of intent detection, and further enhance the performance of slot filling. the experiments conducted on the public dataset demonstrate the effectiveness of our proposed method and our model achieves state-ofthe-art performance in the zero-shot cross-lingual scenario. the research work is supported by national key r&d program of china (no.2017yfb1302400), key research and development program of in shandong province (2019jzzy020102), industrial internet innovation and development project in 2021 (tc210a02m, tc210804d), key research and development program of jiangsu province (no.be2018084), opening project of beijing key laboratory of mobile computing and pervasive device.