MRRL : Modifying the Reference via Reinforcement Learning for Non-Autoregressive Joint Multiple Intent Detection and Slot Filling (2023)

Abstract: With the rise of non-autoregressive approach, some non-autoregressive models for joint multiple intent detection and slot filling have obtained the promising inference speed. However, most existing SLU models (1) suffer from the multi-modality problem that leads to reference intents and slots may not be suitable for training; (2) lack of alignment between the correct predictions of the two tasks, which extremely limits the overall accuracy. Therefore, in this paper, we propose \textbf{M} odifying the \textbf{R} eference via \textbf{R} einforcement \textbf{L} earning (MRRL), a novel method for multiple intent detection and slot filling, which introduces a modifier and employs reinforcement learning. Specifically, we try to provide the better training target for the non-autoregressive SLU model via modifying the reference based on the output of the non-autoregressive SLU model, and propose a suitability reward to ensure that the output of the modifier module could fit well with the output of the non-autoregressive SLU model and does not deviate too far from the reference. In addition, we also propose a compromise reward to realize a flexible trade-off between the two subtasks. Experiments on two multi-intent datasets and non-autoregressive baselines demonstrate that our MRRL could consistently improve the performance of baselines. More encouragingly, our best variant achieves new state-of-the-art results, outperforming the previous best approach by 3.6 overall accuracy on MixATIS dataset.

## introduction as a crucial task in dialogue systems, spoken language understanding (slu) aims to understand the user's current goal through constructing semantic frames @xcite @xcite . intent detection and slot filling are two common subtasks of slu, where intent detection is an utterance-level † equal contribution. * corresponding author. however, an utterance often contains more than a just single intent in the real scenarios @xcite @xcite . with this in mind, multi-intent slu has received more and more attention @xcite @xcite . @xcite proposes a multi-task framework to jointly model intent detection and slot filling. @xcite introduces graph attention networks (gat) @xcite to develop a fine-grained multi-intent prediction framework called agif, which aims to incorporate intent information into the slot filling decoding process in an adaptive manner. @xcite proposes a novel global-locally graph interaction network gl-gin, which appplies the non-autoregressive modeling techniques to parallelize the decoding process, resulting in significant speedup compared to the traditional autoregressive slu models. @xcite further designs a two-stage slu framework and achieves the mutual guidance between intent and slot, which enhances the overall accuracy of slu. @xcite explores attention mechanisms to extract the relevant information from the independent utterance contexts and capture shared label-specific features across all utterances in the training set. @xcite proposes rela-net, which utilizes a heterogeneous label graph to represent the statistical dependencies and hierarchies. @xcite applies contrastive learning to explore and leverage the inherent relationships in multi-intent slu. @xcite proposes a scope-sensitive slu model ss-ran to reduce the distraction of the out-of-scope tokens and mitigate the error propagation problem caused by the bidirectional interaction. though existing non-autoregressive multi-intent slu models have made the promising progress, we find that most of them still face two issues: in this paper, we propose a novel model termed mrrl to tackle the above two issues, which introduces a modifier and applies reinforcement learning to provide a better training target for the nonautoregressive model. since the reference intents and slots may not be suitable for training, we modify the reference intents and slots according to the output of the non-autoregressive slu model. for the first problem, we propose a suitability reward to ensure that the output of the modifier fits well with the output of the non-autoregressive model and still maintain the original information. the first part of suitability reward is related to the training loss on the output of the modifier, and the second part is the similarity between the reference and the output of the modifier. for the second problem, we propose a compromise reward to improve the overall accuracy. we directly utilize the overall accuracy as the compromise reward to achieve a flexible trade-off between the accuracy of intent detection and the f1 score of slot filling. experiment results show that mrrl consistently improves the performance of baselines on two benchmark datasets mixatis and mixsnips @xcite @xcite . further analysis also verifies the advantages of our method. the contributions of our work are three-fold: ## background in this section, we first introduce the problem definition of the multi-intent slu task and then introduce several multi-intent slu models. given an input utterance @xmath0, x 2 , . . . , x n ), where n denotes the length of the utterance x, multiple intent detection is a multi-label classification task and the token-level predicted intent sequence is denoted as y i ′ = (y (1,i ′ ) , y (2,i ′ ) , . . . , y (n,i ′ ) ). the final utterance-level predicted intent sequence y i is obtained by the token-level intent voting strategy @xcite . the reference intent sequence is denoted as ŷi = (ŷ (1,i) , ŷ(2,i) , . . . , ŷ(m,i) ), where m denotes the number of intents in x. slot filling is a sequence labeling task @xcite @xcite . the predicted slot sequence is denoted as y @xmath1,s) , y (2,s) , . . . , y (n,s) ) and the reference slot sequence is denoted as ŷs = (ŷ (1,s) , ŷ(2,s) , . . . , ŷ(n,s) ). for simplicity, we use ŷ to denote the union of ŷi and ŷs . ## approach in this section, we first present the overview of our model ( §3.1), which introduces the modifier module including a intent modifier and a slot modifier into the original architecture. then, we introduce the pretrain-finetune paradigm ( §3.2), where pretraining could establish a better initial state for the fine-tuning of the modifier module. finally, we introduce the reinforcement learning module, which is applied to make the output of the modifier more suitable for model training ( §3.3). figure 2 illustrates the overview of our proposed method. ## conclusion in this paper, we propose mrrl, a simple yet effective method to alleviate the multi-modality problem and misalignment problem in non-autoregressive multi-intent slu. we introduce a modifier to provide a more suitable training target for the model, and apply reinforcement learning with the suitability reward and compromise reward. experiments and analysis demonstrate the effectiveness of our proposed method, which can consistently improve the performance of baselines and the best variant achieves new state-of-the-art performance. future work will focus on how to further alleviate the two problems for non-autoregressive multi-intent slu.