PICD -Instruct: A Generative Instruction Learning Framework for Few-Shot Multi-Intent Spoken Language Understanding (2025)

Abstract: Few-shot multi-intent spoken language understanding (SLU) aims to identify users’ multiple intents and key slots using a tiny amount of annotated data. Recent advances in large language models (LLMs) have utilized instruction learning frameworks to model intent-slot interdependencies, typically requiring abundant data for effective training. However, in few-shot scenarios, these frameworks face challenges such as mismatches between the number of generated slots and input lengths, relational confusion in multi-intent scenarios and neglect of task-specific variations in intent counts across utterances. To overcome the challenges, we propose PICD-Instruct, a novel generative framework based on Basic Instructions (BI), Pairwise Interaction Instructions (PII) and Contrastive Distinct Instructions (CDI). Specifically, BI directs LLMs to generate entities along with associated words, thereby mitigating mismatches in quantitative correspondences. PII explicitly captures dual-task interdependencies by guiding LLMs to pair each intent with its related entities. CDI enhances understanding of utterances by guiding LLMs to determine whether two utterances share the same intent count. Experimental results on public datasets indicate that PICD-Instruct achieves state-of-the-art performance.

## introduction spoken language understanding (slu) @xcite is a fundamental component of taskoriented dialogue systems. among the various aspects of slu, multi-intent slu has gained significant attention due to its practical necessity in complex interactive scenarios. this task involves two closely linked subtasks: multi-intent detection and slot filling. multi-intent detection focuses on *corresponding author 1 our source code is available at https://github.com/ heavencsh/llm_slu identifying the intents embedded within a user utterance, whereas slot filling extracts key semantic information from the utterance. in practical applications, however, obtaining sufficient labeled data for domain-specific slu models is often timeintensive and costly. these challenges highlight the critical importance of exploring multi-intent slu in few-shot settings. given the bidirectional relationship between intents and slots, recent models leverage multi-task joint frameworks to capture these interdependencies, achieving strong performance with sufficient training data @xcite @xcite @xcite @xcite @xcite @xcite . meanwhile, large language models (llms) show promise in the zero-shot slu task @xcite but remain largely designed for single-intent scenarios. for instance, @xcite explored prompt-based zero-shot slu with chatgpt, but its slot filling lagged far behind fine-tuned models. similarly, @xcite proposed a pseudo-labeling framework to enhance task collaboration but faced error propagation issues. to address these limitations, @xcite first introduced instruction learning into generative multi-intent slu. their framework leverages instruction learning and contrastive learning to model intent-slot relationships through mutual prediction of ground-truth labels. by distinguishing task-specific semantics across utterances, this approach enhances slu reasoning. this raises a key question: can instruction-guided llms achieve superior performance in few-shot multi-intent slu? beyond addressing traditional slu challenges, llms introduce new opportunities by enhancing structured and reliable information extraction @xcite . slu plays a crucial role in intelligent agent-driven task completion, where accurate intent detection ensures effective execution of user q6r# im 37 96"s8 $im ! 37 44ttt uuvw x4yz[ commands @xcite . unlike openended generation, slu requires structured output to maintain schema consistency, which is critical for applications in domains such as voice assistants, customer service automation, and smart device control @xcite . however, we discover three core challenges in leveraging llms for few-shot multi-intent slu. firstly, the uncontrollable nature of llmgenerated outputs poses significant challenges for slot filling, as the number of generated slots often fails to correspond with the input length. this issue is exacerbated in few-shot settings, where limited training data restricts the model's ability to accurately map slots to tokens. as shown in fig. 1 , the example demonstrates the over-generation and mismatch of slot labels. secondly, existing generative frameworks exhibit a strong dependence on extensive annotated data and fail to effectively capture the semantic dependencies between intents and slots. dc-instruct @xcite predicts slot labels based on the provided utterance and intent labels, but it falls short in establishing a one-to-one correspondence between each intent and its associated slots. this leads to confusion in multi-intent scenarios, making it harder for models to learn dual-task interdependencies with limited training data. thirdly, unlike single-intent scenarios, the number of intents contained in a user's utterance in multi-intent scenarios is often uncertain, making it more challenging for models to accurately identify all intents. therefore, improving the sensitivity of llms to the variations in intent counts across utterances can enhance their understanding of such cases. however, current approaches often overlook this task-specific feature, potentially hindering the models' ability to effectively comprehend utterances with multiple intents. to overcome these challenges, we propose picd-instruct, a novel generative model based on instruction learning. picd-instruct employs three types of instructions: basic instructions (bi), pairwise interaction instructions (pii) and contrastive distinct instructions (cdi). bi shifts from the tra-ditional approach of assigning a slot label to each word to a formulation based on entity-word pairings, effectively mitigating mismatches between generated slots and input lengths commonly encountered when using llms for direct slot generation. considering that each green entity label in fig. 1 aligns exactly with its associated words, pii incorporates an auxiliary intent-slot pairing task that explicitly models the bidirectional dependencies between intents and slots. by aligning golden intent labels with corresponding entity labels, pii mitigates relational confusions in multi-intent scenarios. cdi enhances the ability to perceive variations in the number of intents within an utterance by introducing a task that determines whether two utterances contain the same number of intents. by leveraging positive and negative samples alongside the current utterance, cdi trains the model to distinguish between utterances based on intent counts, thereby improving its comprehension capabilities. we conduct experiments on two few-shot datasets, fewshotmixatis and fewshotmixs-nips @xcite . experimental results show that picd-instruct significantly outperforms existing baselines, achieving state-of-the-art (sota) performance in the few-shot multi-intent slu task. moreover, it demonstrates strong generalization capability, transferring from a singledomain dataset (fewshotmixatis) to a multidomain dataset (fewshotmixsnips). in summary, our contributions are three-fold: the above approaches primarily focus on scenarios with abundant training data. however, in fewshot settings, capturing the correlations between the two tasks in slu becomes more challenging, leading to degraded performance for most models @xcite . while ugen and dc-instruct have demonstrated performance in low-resource settings, the few-shot training data they utilize does not align well with real-world application scenarios in terms of sample quantity and distribution. to better simulate practical application scenarios, we employ fewshotmixatis and fewshotmixsnips, two datasets specifically tailored for few-shot scenarios, as the data for model training. different from recent works, we propose a novel generative framework incorporating various instructions to ensure the accuracy of llm outputs. our approach explicitly captures dual-task interdependencies by reducing relational confusions and effectively harnesses the variations of intent counts across different utterances, enabling improved performance in the few-shot multi-intent slu task. instruction learning recently, the rise of llms in the natural language processing (nlp) field has positioned instruction learning as a competitive approach across various nlp tasks @xcite . this paradigm effectively leverages the advanced conversational abilities of llms to perform generative tasks, bridging the gap between pre-training and fine-tuning stages. in this work, we investigate instruction learning for few-shot multi-intent slu and propose a novel model characterized by pairwise interactive instructions and contrastive distinct instructions. ## conclusion in this paper, we conduct an in-depth investigation of few-shot multi-intent slu. we propose picd-instruct, a framework designed to address the challenges of generative few-shot multi-intent slu from three key perspectives. firstly, we propose basic instructions to tackle mismatches between the number of generated slots and input lengths. secondly, we introduce pairwise interaction instructions to explicitly model dual-task dependencies while minimizing relational confusions in multiintent scenarios. thirdly, we present contrastive distinct instructions that leverage contrastive relations in intent counts to enhance understanding. experimental results demonstrate that our proposed model achieves sota performance on fewshot-mixatis and fewshotmixsnips, thereby highlighting our model's robust generalization capabilities in a simulated real-world application scenario.