On Collaborating Small and Large Models For Few-shot Intent Detection (2025)

Abstract: Few-shot intent detection (FSID) targets the classification of user queries into in-scope intent categories or detecting them as out-of-scope, with only a few or even zero labeled examples per class. Existing PLM-based methods struggle in low-resource situations; while LLM-based methods face high inference cost and label interference. To harness their complementary strengths, we propose the \textbf{FCSLM} , a framework that collaborates a small prediction model with a large language model for the FSID task. During training, we leverage LLMs for data augmentation in self-supervised pretraining and supervised fine-tuning a task-specific prediction model. During inference, a multi-round reasoning process first applies the small prediction model to output candidate intents with uncertainty estimations, then invokes an LLM with enriched intent descriptions for refined prediction and OOS detection. Extensive experiments on three benchmark datasets demonstrate that our FCSLM outperforms strong competitors, achieving the new state-of-the-art performance in both intent classification and OOS detection. Our code is available at: https://github.com/hustchenpeng/FCSLM

## introduction intent detection is a core task in dialogue systems, aiming to interpret a user query and classify it into one of the predefined intent categories, or out-ofscope (oos). in real-world scenarios, as manual annotation is costly, training data is scarce and fewshot intent detection (fsid) has recently become a central focus. the fsid task aims to classify a user query x into a predefined intent category @xmath0, or detect it as out-of-scope (oos), given only a limited number of labeled training examples per in-scope intent. for the fsid task, fine-tuning pre-trained language models (plms) have long been a dominant approach @xcite @xcite . their semantic modeling capabilities have been further enhanced by recent advances in self-supervised pretraining and contrastive learning @xcite @xcite . however, the performance of plm-based models is not satisfactory due to the data scarcity when training a model. this issue becomes particularly pronounced in oos detection @xcite , where a plm-based model needs to first accurately classify in-scope queries, and then reject out-ofscope inputs. with the rapid development of large language models (llms), recent studies have explored instruction-tuned llms for direct intent classification, typically adopting a full-label enumeration strategy in which all intent labels and examples are included into an input prompt @xcite . while llms demonstrate strong detection capabilities under low-resource conditions, they also encounter several practical challenges. first, real-world applications often involve a large number of intent categories, leading to long prompt sequences and high inference costs @xcite . second, when semantic boundaries between intent categories are ambiguous, llms are prone to confusing similar intents, resulting in degraded discriminative capabilities even worse than plm-based models. this issue is especially evident in oos detection, where llms tend to misclassify oos queries as in-scope (is) when their semantics are close to a known intent category. moreover, the full-label enumeration strategy inherently increases label-space interference, further raising the likelihood of an oos query being incorrectly assigned to an is intent category. the existing studies indicate that plms and llms exhibit complementary strengths yet also respective limitations in the fsid task. those su-pervised plm-based models retain high inference efficiency and stable performance even under large label spaces. in contrast, those llm-based approaches excel in scenarios with fewer candidate intents, where their semantic reasoning capabilities can be well leveraged with lower inference cost and reduced label interference. this suggests that using a single technique, i.e., plm or llm, is often insufficient to balance the detection efficiency of a small model and the semantic inference of a large model. furthermore, achieving reliable oos detection remains a significant challenge for either plm-based or llm-based approaches. these considerations motivate us to reexamine the capabilities and roles of a small plm-based model and a large language model for the fsid task, calling for an effective collaboration between a small and a large model in both offline training and online reasoning phase. motivated from the aforementioned considerations, we propose a framework for collaborating a small prediction model with a large language model for the fsid task, called fcslm. the basic idea is to exploit the complementary capabilities of plms and llms in different task phases. the execution logic is to first train and use a task-specific prediction model (pm) to output a set of candidate intents for a query, and then employ an llm to further discriminate those unconfident candidate intents for a final decision. the workflow of our fcslm is divided into the offline training and online reasoning phase. we propose an offline augmentative training process to train a task-specific pm with the help of an llm for data augmentation. the training process includes three steps: self-supervised pre-training, supervised fine-tuning, and multi-predictor sampling. we propose an online collaborative reasoning process to output the predicted intent for an input query with the collaboration of the task-specific pm and an llm. the reasoning process includes three rounds: the first round applies the pm to output a set of candidate intents. if they fail the confidence test in the first round, the second round employs an llm to distinguish the most suitable candidate intent. if the prediction of the llm differs from that of the pm, the third round adopts the llm to perform a second-thought comparative reasoning for the final decision. figure 1 presents the fcslm workflow. we conduct experiments on three benchmark intent detection datasets under 0/5/10-shot settings. results show that our fcslm achieves the new state-of-the-art performance in both intent classification and oos detection for the fsid task. more importantly, our collaborative framework addresses the high costs of existing llm-based methods by significantly reducing the number of llm calls and token overhead. a comprehensive discussion of related work is deferred to appendix a. ## offline training phase we propose an offline augmentative training process (oatp) to train a task-specific prediction model (pm) via llm-assisted data augmentation. we note that although the fsid task is only with a few labeled samples, it is still possible to train a task-specific pm through data augmentation. such a task-specific pm can itself execute intent detection and its detection outputs can also complement the reasoning process of an llm for more confident results. let b raw denote an off-the-shelf plm, such as the roberta @xcite . first, we propose to use self-supervised pretraining to train b raw into b pre , and then to use supervised fine-tuning to train b pre into b ft . based on b ft , we design a multi-predictor sampling strategy to obtain a set of prediction heads {p c }. ## online reasoning phase we propose an online collaborative reasoning process (ocrp), which utilizes both the task-specific pm and a llm to output the target intent y * for an input query q. in the ocrp, we first use the trained pm to make the first round prediction based on a multi-head voting mechanism (mvm). if the voting is not confident, we next employ an llm to make the second round prediction via an uncertainty resolution mechanism (urm). if the uncertainty still exists, we design a comparative prediction mechanism (cpm) to make the third round and final prediction. furthermore, we also design a plug-in null intent detection (nid) module to deal with out-of-scope scenarios. ## conclusion this paper has proposed a collaborative framework, called fcslm, for the fsid task, which effectively enhances intent classification and out-of-scope detection by collaborating the strengths of a small prediction model and a large language model. in the training phase, the fcslm leverages llms to generate for high-quality augmented data for selfsupervised pretraining and supervised fine-tuning a task-specific prediction. in the inference phase, a multi-round collaborative reasoning mechanism is introduced to utilize the complementary inference capabilities of the small prediction model and a large language model. extensive experiments on three benchmark datasets have shown that our fc-slm outperforms the state-of-the-art competitors in both intent classification and oos detection performance. further evaluations (see appendix 5.4) reveal that our fcslm maintains strong compatibility with llms of varying parameter sizes, highlighting both the effectiveness of its collaborative design and its potential for practical deployment. future work will focus on further improving oos detection by deeply mining the capabilities of both small and large models, while preserving in-scope intent classification performance.