GL - GIN : Fast and Accurate Non-Autoregressive Model for Joint Multiple Intent Detection and Slot Filling (2021)

Abstract: Multi-intent SLU can handle multiple intents in an utterance, which has attracted increasing attention. However, the state-of-the-art joint models heavily rely on autoregressive approaches, resulting in two issues: slow inference speed and information leakage. In this paper, we explore a non-autoregressive model for joint multiple intent detection and slot filling, achieving more fast and accurate. Specifically, we propose a Global-Locally Graph Interaction Network (GL-GIN) where a local slot-aware graph interaction layer is proposed to model slot dependency for alleviating uncoordinated slots problem while a global intent-slot graph interaction layer is introduced to model the interaction between multiple intents and all slots in the utterance. Experimental results on two public datasets show that our framework achieves state-of-the-art performance while being 11.5 times faster.

## introduction spoken language understanding (slu) @xcite is a critical component in spoken dialog systems, which aims to understand user's queries. it typically includes two sub-tasks: intent detection and slot filling @xcite . since intents and slots are closely tied, dominant single-intent slu systems in the literature @xcite @xcite @xcite @xcite adopt joint models to consider the correlation between the two tasks, which have obtained remarkable success. multi-intent slu means that the system can handle an utterance containing multiple intents, which is shown to be more practical in the real-world scenario, attracting increasing attention. to this end, * corresponding author. xu and sarikaya (2013) and @xcite begin to explore the multi-intent slu. however, their models only consider the multiple intent detection while ignoring slot filling task. recently, gangadharaiah and narayanaswamy (2019) make the first attempt to propose a multi-task framework to joint model the multiple intent detection and slot filling. @xcite further propose an adaptive interaction framework (agif) to achieve fine-grained multi-intent information integration for slot filling, obtaining state-of-the-art performance. though achieving the promising performance, the existing multi-intent slu joint models heavily rely on an autoregressive fashion, as shown in figure 1 (a), leading to two issues: in this paper, we explore a non-autoregressive framework for joint multiple intent detection and slot filling, with the goal of accelerating inference speed while achieving high accuracy, which is shown in figure 1 @xcite . to this end, we propose a global-locally graph-interaction network (gl-gin) where the core module is a proposed local slot-aware graph layer and global intent-slot interaction layer, which achieves to generate intents and slots sequence simultaneously and nonautoregressively. in gl-gin, a local slot-aware graph interaction layer where each slot hidden states connect with each other is proposed to explicitly model slot dependency, in order to alleviate uncoordinated slot problem (e.g., b-singer followed by i-song) @xcite due to the non-autoregressive fashion. a global intent-slot graph interaction layer is further introduced to perform sentence-level intent-slot interaction. unlike the prior works that only consider the tokenlevel intent-slot interaction, the global graph is constructed of all tokens with multiple intents, achieving to generate slots sequence in parallel and speed up the decoding process. experimental results on two public datasets mixsnips @xcite and mix-atis @xcite show that our framework not only obtains state-of-the-art performance but also enables decoding in parallel. in addition, we explore the pre-trained model (i.e., roberta @xcite ) in our framework. in summary, the contributions of this work can be concluded as follows: (1) to the best of our knowledge, we make the first attempt to explore a non-autoregressive approach for joint multiple intent detection and slot filling; (2) we propose a global-locally graph-interaction network, where the local graph is used to handle uncoordinated slots problem while a global graph is introduced to model sequence-level intent-slot interaction; (3) experiment results on two benchmarks show that our framework not only achieves the state-of-theart performance but also considerably speeds up the slot decoding (up to ×11.5); (4) finally, we explore the pre-trained model in our framework. with the pre-trained model, our model reaches a new state-of-the-art level. for reproducibility, our code for this paper is publicly available at https://github.com/ yizhen20133868/gl-gin . ## problem definition multiple intent detection given input sequence @xmath0, . . . , x n ), multiple intent detection can be defined as a multi-label classification task that outputs a sequence intent label o @xmath1, . . . , o i m ), where m is the number of intents in given utterance and n is the length of utterance. slot filling slot filling can be seen as a sequence labeling task that maps the input utterance x into a slot output sequence o @xmath2, . . . , o s n ). ## approach as shown in figure 2 (a), we describe the proposed framework, which consists of a shared selfattentive encoder ( §3.1), a token-level intent detection decoder ( §3.2) and a global-local graphinteraction graph decoder for slot filling ( §3.3). both intent detection and slot filling are optimized simultaneously via a joint learning scheme. ## conclusion in this paper, we investigated a non-autoregressive model for joint multiple intent detection and slot filling. to this end, we proposed a global-locally graph interaction network where the uncoordinatedslots problem can be addressed with the proposed local slot-aware graph while the interaction between intents and slots can be modeled by the proposed global intent-slot graph. experimental results on two datasets show that our framework achieves state-of-the-art performance with ×11.5 times faster than the prior work.