A Transformer-based Threshold-Free Framework for Multi-Intent NLU (2022)

Abstract: Multi-intent natural language understanding (NLU) has recently gained attention. It detects multiple intents in an utterance, which is better suited to real-world scenarios. However, the state-of-the-art joint NLU models mainly detect multiple intents on threshold-based strategy, resulting in one main issue: the model is extremely sensitive to the threshold settings. In this paper, we propose a transformer-based Threshold-Free Multi-intent NLU model (TFMN) with multi-task learning (MTL). Specifically, we first leverage multiple layers of a transformer-based encoder to generate multi-grain representations. Then we exploit the information of the number of multiple intents in each utterance without additional manual annotations and propose an auxiliary detection task: Intent Number detection (IND). Furthermore, we propose a threshold-free intent multi-intent classifier that utilizes the output of IND task and detects the multiple intents without depending on the threshold. Extensive experiments demonstrate that our proposed model achieves superior results on two public multi-intent datasets.

## introduction natural language understanding (nlu) consists of two sub-tasks, including intent detection (id) and slot filling (sf) which allow the dialogue system to create a semantic frame that summarizes the user's requests. early works often approach these two tasks separately @xcite @xcite . considering intent detection and slot filling are highly related, recent works tend to model these two tasks jointly, where the correlation between the intent and slots are utilized @xcite @xcite . the works above only consider the scenario where each utterance has one intent. however, in real-life situations, users may express multiple intents in an utterance, thus making it difficult to apply single intent nlu models. recently, several works have studied multi-intent nlu problem. gangadharaiah et al.(2019) investigated an attention-based neural network. qin et al.(2020) proposed an adaptive graph interactive framework (agif). qin et al.(2021) explored a non-autoregressive approach to speed up the inference time. chen et al.(2022a) proposed a selfdistillation joint nlu model. however, these works all predict multiple intents with threshold, in this paper, we propose a transformer-based threshold-free multi-nlu model (tfmn) and detect multiple intents without relying on the threshold. specifically, we leverage the upper layers of a transformer-based encoder to generate multi-grain representations. next, we fully exploit the annotations from original multiple intents data and propose an intent number detection (ind) task. the motivation is to allow the model to detect the intent numbers in a given utterance. then we propose a threshold-free intent classifier that utilizes the output of ind task to detect the multiple intents. we validate tfmn on two public datasets @xcite : mixatis and mixsnips, and show that our method outperforms competitive baselines. the contributions of our work are summarized as follows: (1)we propose a novel threshold-free multi-nlu model based on transformers.(2) we propose ind task, a feasible task to improve the multi-intent nlu without additional manual annotation, and a threshold-free multi-intent classifier that detects multiple intents without relying on threshold. (3) we present extensive experiments demonstrating the effectiveness of our approach. ## problem formulation given an input sequence @xmath0, ..., x n ), multiintent detection is defined as a multi-label classification task that outputs o @xmath1, ..., o i m ), where m is the number of predicted intent labels. slot filling task can be regarded as a sequence labeling task that outputs a slot sequence ## approach in this section, we first introduce the architecture of tfmn model, then detail the proposed ind task and threshold-free intent classifier. ## conclusion in this paper, we propose tfmn model which detects intent numbers in an utterance by a novel ind task that does not require additional manual annotations. then we propose a threshold-free intent classifier to detect multiple intents without relying on the threshold. extensive experiments show that tfmn achieves performance gains over strong baselines.