HIT - SCIR at MMNLU -22: Consistency Regularization for Multilingual Spoken Language Understanding (2022)

Abstract: Multilingual spoken language understanding (SLU) consists of two sub-tasks, namely intent detection and slot filling. To improve the performance of these two sub-tasks, we propose to use consistency regularization based on a hybrid data augmentation strategy. The consistency regularization enforces the predicted distributions for an example and its semantically equivalent augmentation to be consistent. We conduct experiments on the MASSIVE dataset under both full-dataset and zero-shot settings. Experimental results demonstrate that our proposed method improves the performance on both intent detection and slot filling tasks. Our system ranked 1st in the MMNLU-22 competition under the full-dataset setting.

## introduction the mmnlu-22 evaluation focuses on the problem of multilingual natural language understanding. it is based on the massive dataset @xcite , a multilingual spoken language understanding (slu) dataset with two sub-tasks, including intent detection and slot filling. specifically, given a virtual assistant utterance in an arbitrary language, the model is designed to predict the corresponding intent label and extract the slot results. an english example is illustrated in figure 1 . fine-tuning pre-trained cross-lingual language models allows task-specific supervision to be shared and transferred across languages @xcite @xcite . this motivates the two setting for the mmnlu-22 evaluation, namely the full-dataset setting and the zero-shot setting. participants are allowed to use training data in all languages under the full-dataset setting, while they can only access the english training data under the zero-shot setting. the latter is also called zero-shot cross-lingual slu in previous work @xcite . cross-lingual data augmentation methods have been proven effective to improve cross-lingual transferability, e.g., code-switch substitution @xcite and machine translation @xcite . most previous work directly utilizes the data augmentations as additional training data for fine-tuning. however, they ignore the inherent correlation between the original example and its semantically equivalent augmentation, which can be fully exploited with the consistency regularization @xcite . the consistency regularization enforces the model predictions to be more consistent for semantic-preserving augmentations. motivated by this, we propose to apply consistency regularization based on a hybrid data augmentation strategy, including data augmentation of machine translation and subword sampling @xcite . we use machine translation augmentation to align the model predictions of the intent detection task. meanwhile, subword sampling augmentation is used to align the model predictions of both intent detection and slot filling tasks. the proposed method consistently improves the slu performance on the massive dataset under both full-dataset and zero-shot settings. it is worth mentioning that our system ranked 1st in the mmnlu-22 competition under the full-dataset setting. we achieved an exact match accuracy of 49.65 points, outperforming the 2nd system by 1.02 points. ## method given the input utterance @xmath0, ..., x n ) with length n and the corresponding intent label o i and slot labels o @xmath1, ..., ...o s n ) from training corpus d, we define the loss for the two sub-tasks of slu in our fine-tuning process as: ## conclusion we propose to use consistency regularization based on a hybrid data augmentation strategy to improve the performance of multilingual slu. the proposed method is flexible and can be easily plugged into the fine-tuning process of both the encoderonly model and the text-to-text model. the experimental results demonstrate the importance of consistency regularization and the hybrid data augmentation strategy, respectively.