Small Models are Valuable Plug-ins for Large Language Models (2024)

Abstract: Large language models (LLMs) such as GPT-3 and GPT-4 are powerful but their weights are often publicly unavailable and their immense sizes make the models difficult to be tuned with common hardware. As a result, effectively tuning these models with large-scale supervised data can be challenging. As an alternative, In-Context Learning (ICL) can only use a small number of supervised examples due to context length limits. In this paper, we propose Super In-Context Learning (SuperICL) which allows black-box LLMs to work with locally fine-tuned smaller models, resulting in superior performance on supervised tasks. Our experiments demonstrate that SuperICL can improve performance beyond state-of-the-art fine-tuned models while addressing the instability problem of in-context learning.

## introduction large-scale pre-trained language models, such as @xcite and @xcite , have demonstrated remarkable capabilities in a wide range of nlp tasks. despite the impressive performance of these recently released models, their size can lead to difficulties in fine-tuning these models with supervised data, which is an effective way to adapt the models to specific tasks @xcite . an alternative approach, in-context learning @xcite , involves concatenating a few labeled examples with the test input, enabling the model to learn from the context. however, icl is limited by the maximum context length of the llm, restricting the number of examples it can utilize. consequently, while icl can usually perform few-shot learning with 16 or 32 examples, it cannot fully exploit supervised data when there are hundreds or thousands of examples. to address these limitations, we propose super in-context learning (supericl), a novel approach that enables black-box language models (e.g., in-structgpt, gpt-4) to work with locally fine-tuned smaller models (e.g., roberta, @xcite , resulting in improved performance on supervised tasks. supericl is designed to overcome the challenges of poor performance and instability of icl. supericl builds on the strengths of icl while mitigating its limitations. as shown in figure 1 , supericl leverages a combination of an llm with smaller models, which act as plug-ins, to perform supervised tasks efficiently. specifically, we use the plug-in model to predict labels with confidence for in-context examples and concatenate them with the input text and ground-truth labels as context. for test examples, we also add the plug-in model's prediction and confidence to the test input and let the llm predict the final label and an explanation. as these plug-in models have been fine-tuned on the task-specific data, they serve as a bridge between the large pre-trained model and the task-specific data, allowing for effective knowledge transfer and improved performance. we conduct extensive experiments to evaluate the effectiveness of supericl on glue @xcite , a standard benchmark for natural language understanding. our results show that super-icl: (1) achieves superior performance compared to state-of-the-art fine-tuned models and llms; ## related work in-context learning originally proposed in the gpt-3 paper @xcite , in-context learning (icl) is considered as a new paradigm that exploits llms on new tasks without updating the parameters of the model. it prepends few-shot training examples before the test input as a prompt, to enable large language models to find patterns and "learn" to predict. there have been successful applications of icl in downstream tasks, such as machine translation @xcite and data generation @xcite . despite its success in few-shot learning, a major drawback of icl is instability. the performance of icl is sensitive to the selected in-context examples @xcite and even their order @xcite . based on these discoveries, there is a line of studies focused on constructing the context. lm-bff @xcite and kate @xcite select training examples that are semantically similar to the test example. another line of works @xcite @xcite focus on mining diverse and representative examples from a training set. @xcite utilizes active learning and reinforcement learning to select examples for icl. self-adaptive icl @xcite proposes a two-stage search framework to obtain the optimal in-context examples for each test input without using a separate validation set. different from these works, supericl demonstrates that smaller models can be integrated into large language models for supervised tasks. although it is orthogonal to these prior works, by fine-tuning the plug-in model with the entire training set, supericl reduces the necessity for selecting the optimal examples from the training set. moreover, prior studies also investigate how to prepare language models for icl. @xcite propose calibration with an empty test input to reduce the influence of the label distribution and ordering. metaicl @xcite ) meta-trains the language model to generalize to unseen tasks for better icl performance. @xcite propose four self-supervised objectives as intermediate tasks, to improve performance of language models on icl. notably, both @xcite and @xcite require updating the weights, thus are not applicable to larger black-box models algorithm 1 super in-context learning (supericl) require: training set @xmath0, y 1 ), ..., (x n , y n )}, llm m , a small pre-trained language model p ensure: predicted label y t and optional explanation e t 1: fine-tune p on d to obtain the fine-tuned plug-in model p ′ 2: randomly sample a set of examples (x i , y i ) from d to be the set of in-context examples d ′ 3: for each example (x i , y i ) in d ′ do 4: predict y ′ i and c i with p ′ where y ′ i is the predicted label and c i is the confidence score 5: end for 6: construct the context c by concatenating all (x i , y ′ i , c i , y i ) 7: for each test example x t do 8: predict y ′ t and c t with p ′ for the test example 9: formulate complete input @xmath1, y ′ t , c t ) where ⊕ denotes concatenation 10: use m to predict y t from i 11: besides studies aiming to improve icl's performance, some studies have analyzed the underlying mechanism of icl. @xcite find that the label space, the distribution of the input text, and the overall format of the sequence are the key factors to icl's performance. they also claim that the ground labels are not significant to the performance of icl, but this conclusion is contradicted by a later study @xcite . additionally, prior studies suggest icl could be implicitly performing bayesian inference @xcite or gradient descent @xcite @xcite . language model plug-ins large language models can exploit external tools to improve their capabilities. toolformer @xcite introduces special symbols that allow the large language models to call external apis to complete tasks. visual chatgpt @xcite plugs vision models into chatgpt, allowing for multimodal generation. hugginggpt @xcite uses chatgpt to conduct task planning and select models according to their function descriptions available in hugging face, execute each subtask with the selected ai model, and summarize the response according to the execution results. different from these works, our work is under a classic supervised learning and demonstrates that even tasks like text classification, which is sometimes considered "solved" by smaller language models, can still benefit from combination with a large language model. ## super in-context learning super in-context learning (supericl) combines llms with locally fine-tuned smaller models, allowing them to work together to improve performance on supervised tasks. the smaller models act as plug-ins, providing task-specific knowledge and predictions, while the large pre-trained models focus on general language understanding. the overall workflow of supericl is shown in figure 1 and the complete algorithm is depicted in algorithm 1. ## case study we conduct a case study to better understand the behavior of supericl, with three examples presented in ## conclusion and future work in this paper, we propose supericl, a simple yet effective method for combining a large language model api with a locally fine-tuned plug-in model. for future work, a theoretical analysis may be important to further reveal the internal mechanism of supericl.