Simple Data Augmentation for Multilingual NLU in Task Oriented Dialogue Systems
Authors: Samuel Louvan, Bernardo Magnini (Università di Trento, Fondazione Bruno Kessler, 2020)
Topic: Chatbots and Dialogue Systems

Abstract
Data augmentation has shown potential in alleviating data scarcity for Natural Language Understanding (e.g. slot filling and intent classification) in task-oriented dialogue systems. As prior work has been mostly experimented on English datasets, we focus on five different languages, and consider a setting where limited data are available. We investigate the effectiveness of non-gradient based augmentation methods, involving simple text span substitutions and syntactic manipulations. Our experiments show that (i) augmentation is effective in all cases, particularly for slot filling; and (ii) it is beneficial for a joint intent-slot model based on multilingual BERT, both for limited data settings and when full training data is used.

Introduction
Natural Language Understanding (NLU) in task-oriented dialogue systems is responsible for parsing user utterances to extract the intent of the user and the arguments of the intent (i.e. slots) into a semantic representation, typically a semantic frame AUTHOR. For example, the utterance ``Play Jeff Pilson on Youtube"" has the intent PlayMusic and Youtube as value for the slot Service. As more skills are added to the dialogue system, the NLU model frequently needs to be updated to scale to new domains and languages, a situation which typically becomes problematic when labeled data are limited (data scarcity). One way to combat data scarcity is through data augmentation (DA) techniques performing label preserving operations to produce auxiliary training data. Recently, DA has shown potential in tasks such as machine translation AUTHOR, constituency and dependency parsing AUTHOR, and text classification AUTHOR. As for slot filling (SF) and intent classification (IC), a number of DA methods have been proposed to generate synthetic utterances using sequence to sequence models AUTHOR, Conditional Variational Auto Encoder AUTHOR, or pre-trained NLG models AUTHOR. To date, most of the DA methods are evaluated on English and it is not clear whether the same finding apply to other languages. In this paper, we study the effectiveness of DA on several non-English datasets for NLU in task-oriented dialogue systems. We experiment with existing lightweight, non-gradient based, DA methods from AUTHOR that produces varying slot values through substitution and sentence structure manipulation by leveraging syntactic information from a dependency parser. We evaluate the DA methods on NLU datasets from five languages: Italian, Hindi, Turkish, Spanish, and Thai. The contributions of our paper are as follows: i) We assess the applicability of DA methods for NLU in task-oriented dialogue systems in five languages. ii) We demonstrate that simple DA can improve performance on all languages despite different characteristic of the languages. iii) We show that a large pre-trained multilingual BERT (M-Bert) AUTHOR can still benefit from DA, in particular for slot filling.

Conclusion
We evaluate the effectiveness of data augmentation for slot filling and intent classification tasks in five typologically diverse languages. Our results show that by applying simple augmentation, namely slot values substitutions and dependency tree manipulations, we can obtain substantial improvement in most cases when only small amount of training data is available. We also show that a large pre-trained multilingual BERT benefits from data augmentation.