M o E - SLU : Towards ASR -Robust Spoken Language Understanding via Mixture-of-Experts (2024)

Abstract: As a crucial task in the task-oriented dialogue systems, spoken language understanding (SLU) has garnered increasing attention. However, errors from automatic speech recognition (ASR) often hinder the performance of understanding. To tackle this problem, we propose MoE-SLU, an ASR-Robust SLU framework based on the mixture-of-experts technique. Specifically, we first introduce three strategies to generate additional transcripts from clean transcripts. Then, we employ the mixture-of-experts technique to weigh the representations of the generated transcripts, ASR transcripts, and the corresponding clean manual transcripts. Additionally, we also regularize the weighted average of predictions and the predictions of ASR transcripts by minimizing the Jensen-Shannon Divergence (JSD) between these two output distributions. Experiment results on three benchmark SLU datasets demonstrate that our MoE-SLU achieves state-of-the-art performance. Further model analysis also verifies the superiority of our method.

## introduction spoken language understanding (slu) is a fundamental task in recent task-oriented dialogue systems, aimed at capturing the comprehensive semantics of human speech. it plays an important role in personal assistants like amazon's alexa, apple's siri, and microsoft's cortana @xcite @xcite . slu focuses on two typical subtasks: intent detection and slot filling @xcite . intent detection can be regarded as a semantic classification task @xcite , aiming to predict the user's intent @xcite . slot filling could be approached as a sequence labeling task, where the goal is to predict the slot for each token @xcite @xcite . ## method in this section, we first propose three strategies to generate additional transcripts from the clean transcripts ( §3.1). then, we begin to introduce our proposed asr-robust slu framework moe-slu, including a self-supervised contrastive learning module in pre-training ( §3.2) and a mixture-of-experts module in fine-tuning ( §3.3). finally, we introduce the training objective during the pre-training stage and the fine-tuning stage ( §3.4). ## conclusion in this paper, we propose a new asr-robust slu framework moe-slu. we design three strategies to simulate more kinds of errors in the real environment and apply mixture-of-experts to leverage the different transcripts more effectively. experiments and analysis on three datasets show that our model significantly outperforms previous models. future work will focus on exploring how to make better use of the unique strengths of different transcripts.