Incorporating Instructional Prompts into a Unified Generative Framework for Joint Multiple Intent Detection and Slot Filling (2022)

Abstract: The joint multiple Intent Detection (ID) and Slot Filling (SF) is a significant challenge in spoken language understanding. Because the slots in an utterance may relate to multi-intents, most existing approaches focus on utilizing task-specific components to capture the relations between intents and slots. The customized networks restrict models from modeling commonalities between tasks and generalization for broader applications. To address the above issue, we propose a Unified Generative framework (UGEN) based on a prompt-based paradigm, and formulate the task as a question-answering problem. Specifically, we design 5-type templates as instructional prompts, and each template includes a question that acts as the driver to teach UGEN to grasp the paradigm, options that list the candidate intents or slots to reduce the answer search space, and the context denotes original utterance. Through the instructional prompts, UGEN is guided to understand intents, slots, and their implicit correlations. On two popular multi-intent benchmark datasets, experimental results demonstrate that UGEN achieves new SOTA performances on full-data and surpasses the baselines by a large margin on 5-shot (28.1%) and 10-shot (23%) scenarios, which verify that UGEN is robust and effective.

## introduction in task-oriented dialogue systems, spoken language understanding (slu) is a crucial component that aims to understand users' queries and use a semantic frame to represent users' requirements. the semantic frame usually contains intents and slot names @xcite . recently, multiple intent slu has attracted lots of attention @xcite @xcite * * corresponding author figure 1 : the semantic frame. an example from mixs-nips dataset @xcite . gangadharaiah and narayanaswamy, 2019) due to the wide variety of practical application scenarios. considering the example shown in figure 1 , the models are expected to identify the intents (addto-playlist and ratebook) and the slot values with tags for the utterance. current works @xcite @xcite @xcite usually treat intent detection (id) as a classification task and slot filling (sf) as a sequence labeling task. the task-specific components are employed by current works to capture the connection or interaction between id and sf, which achieve fine-grained multi-intent information integration for slot filling and obtain remarkable success. in this paper, we're interested in exploiting a united paradigm to handle the task instead of customized networks. prompt-learning @xcite is a novel paradigm, which replaces the "pre-train, fine-tune" procedure with "pre-train, prompt, and predict" analogous to original pre-training language models (plms). with the help of a prompt template, prompt-learning benefits from fully exploiting the latent knowledge in plms while relieving the dependency on annotated data. thus, prompt-based plms perform excellently in different tasks (classification, ner, summarization, etc.) and the few-shot setting. to this end, we treat the joint multiple id_sf as a question-answering problem and present a simple unified generative framework (ugen) based on instructional prompts. briefly, we first define 5-type descriptive templates (shown in figure 2 ) as inputs. per template contains one context that refers to the original utterance, one question (e.g., "what are the intents of the sentence according to options?") as the driver to direct ugen to realize the paradigm, and the corresponding options (e.g., play music, rate book) to restraint the answer search space. through these instructional prompts, ugen is directed to acquire the ability to capture the relationship between intents and slots. then the correct intents and slots are predicted as the final answer (e.g., "add to playlist, rate book"). experiments on two multi-intent benchmarks show that ugen outperforms the baselines and achieves new sota performances. remarkably, ugen exceeds the comparison models by a large margin (28.1%, 23%, and 5.1%) in the 5/10-shot settings and 10% training data. the further analyses demonstrate that our approach has a strong ability of robustness and generalization. meanwhile, it has the advantage of fast adaptation to practical scenario with limited annotation data and easy reproduction without task-specific components. few-shot learning (fsl) with plms. fsl aims to absorb experience from only a few samples and make a great adaptation to the new problem @xcite . usually, the models for fsl are trained on one accessible set of source domains and then evaluated on another set of unseen target domains. as the pre-trained models become more and more powerful, prompt-based methods with plms have achieved substantial improvements compared to those fine-tuned in low-resource settings, which displays promising prospects for 3 methodology in this section, we briefly illustrate the problem definition of multiple id_sf and main architecture. then, we discuss the design of instruction-based templates and how to convert the id_sf to the generation task. ## conclusion in this work, we present a novel unified generative framework (ugen) to treat the joint multiple intent detection and slot filling as a question-answering problem. to leverage the knowledge learned in the plms, we define 5-type prompt templates as the drivers to lead ugen to grasp the prompt paradigm and capture the implicit correlations between intents and slots. on two multi-intent benchmark datasets, our approach accomplishes the new stateof-the-art performances in all the metrics, which validates that our design is effective. meanwhile, ugen leads to 28.1%, 23.0%, and 5.1% improvements in the 5/10-shot and 10% training data settings, which verify that ugen is robust with limited annotation data.