DC -Instruct: An Effective Framework for Generative Multi-intent Spoken Language Understanding (2024)

Abstract: In the realm of multi-intent spoken language understanding, recent advancements have leveraged the potential of prompt learning frameworks. However, critical gaps exist in these frameworks: the lack of explicit modeling of dual-task dependencies and the oversight of task-specific semantic differences among utterances. To address these shortcomings, we propose DC-Instruct, a novel generative framework based on Dual-task Inter-dependent Instructions (DII) and Supervised Contrastive Instructions (SCI). Specifically, DII guides large language models (LLMs) to generate labels for one task based on the other task’s labels, thereby explicitly capturing dual-task inter-dependencies. Moreover, SCI leverages utterance semantics differences by guiding LLMs to determine whether a pair of utterances share the same or similar labels. This can improve LLMs on extracting and discriminating task-specific semantics, thus enhancing their SLU reasoning abilities. Extensive experiments on public benchmark datasets show that DC-Instruct markedly outperforms current generative models and state-of-the-art methods, demonstrating its effectiveness in enhancing dialogue language understanding and reasoning.

## introduction in dialogue systems, spoken language understanding (slu) @xcite is a fundamental component for comprehensively understanding users' queries. in recent developments, multi-intent slu @xcite has garnered significant attention for its various and practical application scenarios. it typically includes two subtasks: multiple intent detection and slot filling. multiple intent detection aims to identify the intents expressed in the utterance, while slot filling extracts specific pieces of semantics information from the utterance. some examples are shown in fig. 1 . since there exist inherent inter-dependencies between intents and slots, recent models widely adopt the multi-task framework based on a shared semantics encoder and model the dual-task interactions through some specialized components @xcite @xcite @xcite . among them, co-guiding net @xcite achieves the mutual guidance between the two tasks via heterogeneous graph attention networks. these models show potential, but their specialized components limit their generalization ability. to this end, the prompt learning paradigm is integrated, and @xcite propose a unified generative framework (ugen), which includes five kinds of templates in the question-answer formulation. nonetheless, we discover that up-to-date multiintent slu methods still suffer from two issues. firstly, current prompt instructions fail to effectively model the inter-dependencies between multiple intent detection and slot filling. the five instructions (i 1 -i 5 ) in ugen tackle the two tasks separately: i 1 targets multiple intent detection, while i 2 -i 5 focus on slot filling. we propose that explicitly modeling the dual-task inter-dependencies within the prompt learning framework could significantly enhance the reasoning ability of large language models (llms). secondly, there is an oversight of the semantic variations among utterances. in fig. 1 , there exist specific semantics differences among the three utterances, which can be reflected by their labels. utterance a and b have the same intent labels, while utterance c has different intent labels with them. utterance b and c have similar slot labels, while utterance a has quite different slot labels with them. we argue that these task-specific contrastive relations can benefit llms on understanding task-specific semantics while existing methods ignore them. to resolve the above challenges, in this work, we introduce a novel generative model termed dc-instruct. we propose dual-task inter-dependent instructions (dii) and supervised contrastive instructions (sci) to adeptly model dual-task interdependencies and exploit task-specific semantic differences within the prompt learning framework. dii introduces two auxiliary tasks: slot-guided multiple intent detection and intent-guided slot filling, integrating inter-dependent instructions by embedding one task's golden labels into the instructional context of the other. this enables llms to conditionally generate task-specific labels, effectively capturing dual-task dependencies and alignments. to address the challenge of exploiting utterance contrastive relations, sci introduces an auxiliary task for determining whether a pair of utterances share the same intents or similar slot types. sci guides llms to discern true/false outcomes regarding the task-specific semantics of both the an-chor utterance and its corresponding positive/negative examples. in this way, sci can enhance llms' ability to distinguish and align task-specific semantics for improving slu reasoning. taking the public benchmark datasets as test beds, we conduct extensive experiments based on various llms scaling from 220m to 13b. the experimental results show that our models can achieve consistent and significant improvements over state-of-the-art models. the ablation study and experiments in different low-resource settings further verify our method's advantages. our major contributions are three-fold: (1) we propose dc-instruct, a novel generative model based on dual-task inter-dependent instructions and supervised contrastive instructions. prompt learning recently, prompt learning has been attracting increasing attention since it achieves promising performance in various nlp tasks @xcite @xcite . this paradigm can unify the pre-training and fine-tuning stages into the text-to-text generation tasks. for example, @xcite propose a a dual-slot multiprompt template to unify entity locating and entity typing. @xcite propose a path prediction method based on prompt learning to incorporate the hierarchy structure. in this work, we investigate prompt learning for multi-intent slu and propose a novel model distinguished by dual-task inter-dependent instructions and supervised contrastive instructions. ## preliminary multi-intent slu aims to predict all the intents expressed in the utterance and the slot label corresponding to each word. traditional framework regards multiple intent detection as a multi-label sentence classification task and regards slot filling as a sequence labeling task. ugen @xcite makes the first attempt to explore generative multi-intent slu based on the prompt learning paradigm. in generative multi-intent slu, the output of the multiple intent detection task is a sequence of intents expressed in the utterance. the output of the slot filling task is a sequence of slot value-name pairs. a slot value is a word or phrase expressing specific semantics and the slot name is the slot type or label corresponding to the slot value. considering the first example in fig. 1 , there are two slot value-name pairs: [cheapest, cost relative] and [general mitchell international, airport name]. ## method in this section, we introduce our proposed dc-instruct framework, as shown in fig. 3 . following @xcite , we formulate our instructions in the question-answer (qa) form. there are total five types of instructions in ugen to tackle the two tasks separately, while they cannot cap-ture the dual-task dependencies nor contrastive relations. our framework also includes these five instructions, and they are referred to as basic instructions. we first briefly introduce the basic instructions (i 1 , ..., i 5 ) and then depict our proposed dual-task inter-dependent instructions (i 6 , i 7 ) and supervised contrastive instructions (i 8 , i 9 ). ## conclusion in this paper, we propose dc-instruct, addressing the challenges in generative multi-intent slu from two perspectives. firstly, we propose dual-task inter-dependent instructions to explicitly model the dual-task dependencies. secondly, we propose supervised contrastive instructions, which exploit the utterance contrastive relations in the prompt learning paradigm. extensive evaluations on benchmarks demonstrate the superiority of our method, which can achieve promising improvements over various llms scaling from 220m to 13b.