Retrieval-based Knowledge Transfer: An Effective Approach for Extreme Large Language Model Compression (2023)

Abstract: Large-scale pre-trained language models (LLMs) have demonstrated exceptional performance in various natural language processing (NLP) tasks. However, the massive size of these models poses huge challenges for their deployment in real-world applications. While numerous model compression techniques have been proposed, most of them are not well-suited for achieving extreme model compression when there is a significant gap in model scale. In this paper, we introduce a novel compression paradigm called Retrieval-based Knowledge Transfer (RetriKT), which effectively transfers the knowledge of LLMs to extremely small-scale models (e.g., 1%). In particular, our approach extracts knowledge from LLMs to construct a knowledge store, from which the small-scale model can retrieve relevant information and leverage it for effective inference. To improve the quality of the model, soft prompt tuning and Proximal Policy Optimization (PPO) reinforcement learning techniques are employed. Extensive experiments are conducted on low-resource tasks from SuperGLUE and GLUE benchmarks. The results demonstrate that the proposed approach significantly enhances the performance of small-scale models by leveraging the knowledge from LLMs.

## introduction pre-trained language models (plms), such as bert/roberta @xcite , have demonstrated exceptional performance across various natural language processing (nlp) applications. however, these models typically comprise hundreds of millions of parameters, presenting a substantial challenge for researchers due to their massive scale. as a result, the full potential of large-scale pre-trained language models (plms) remains untapped. to tackle this challenge, a multitude of model compression techniques have been proposed, encompassing knowledge distillation @xcite @xcite , network pruning @xcite , quantization @xcite and weight sharing @xcite . however, these model compression methods are not directly applicable to scenarios requiring high compression ratios, such as knowledge distillation. in such cases, the introduction of assistant models @xcite often leads to decreased and unstable performance. recently, there has been a growing interest in leveraging large language models (llms) @xcite @xcite that possess extensive language knowledge and can be effectively employed in various downstream tasks. consequently, it is essential to explore methods for transferring this knowledge to small-scale models. nonetheless, existing approaches are inadequate for compressing llms due to their exceptionally high compression ratios. some prior research @xcite @xcite has suggested utilizing llms for data augmentation and knowledge transfer to small-scale models, which allows the latter to demonstrate improved performance on lowresource datasets. however, when tackling more challenging tasks like the superglue benchmark @xcite , the limited parameter size of small-scale models becomes a hindrance, preventing them from effectively retaining the knowledge transferred by llms. consequently, the performance enhancement achieved for small-scale models remains constrained. to effectively transfer the knowledge of large language models (llms) to small-scale models, enabling them to efficiently and accurately complete tasks, we propose a novel compression paradigm called retrieval-based knowledge transfer (retrikt). our approach involves two main steps: knowledge extraction from the llm to construct a knowledge store, and subsequent retrieval of relevant information from the knowledge store by the small-scale model to accomplish the task. more specifically, we employ the technique of soft prompt tuning to fine-tune an llm, ensuring that it generates in-domain samples. additionally, we introduce the proximal policy optimization (ppo) @xcite reinforcement learning algorithm to enhance the generation quality. as a final step, the small-scale model learns to retrieve pertinent information from the knowledge store. we perform an extensive set of experiments on truly low-resource and challenging tasks sourced from superglue @xcite and glue @xcite benchmarks. the experimental results demonstrate that retrikt significantly enhances the performance of small-scale models and outperforms previous sota knowledge distillation methods by leveraging the knowledge of llms. this indicates the effectiveness and practicality of the retrieval-based knowledge transfer paradigm for extreme model compression. our contributions can be summarized as follows: ## related work this paper involves three areas of work: knowledge distillation, reinforcement learning, and data augmentation. ## methodology our approach revolves around extracting knowledge from the parameters of a large language model (llm) and leveraging it for the benefit of an extreme small-scale model through retrieval. we introduce the technique to extract in-domain knowledge from the llm for each task by freezing its parameters and fine-tuning additional soft prompts using the corresponding dataset (section 3.1). to further enhance the generation quality, we introduce reinforcement learning and carefully design the reward function to fine-tune the soft prompts (section 3.2). subsequently, we extract the in-domain knowledge from the llm and create a knowledge store (section 3.3). finally, we enable the small-scale model to retrieve and effectively utilize the knowledge generated by the llm to successfully perform specific tasks (section 3.4). ## conclusion our study tackles the task of compressing llms and introduces a pioneering compression paradigm called retrieval-based knowledge transfer. this approach efficiently transfers the knowledge of llms to small-scale models by creating a knowledge store, enabling the small model to retrieve pertinent information during inference. through extensive experiments conducted on commonly used benchmarks, we demonstrate that our framework substantially improves the performance of smallscale models by leveraging the knowledge contained within llms. in future research, we plan to investigate the application and performance of our proposed method on even larger language models, such as t5-11b.