L a M ini- LM : A Diverse Herd of Distilled Models from Large-Scale Instructions (2024)

Abstract: Large language models (LLMs) with instruction fine-tuning demonstrate superior generative capabilities. However, these models are resource-intensive. To alleviate this issue, we explore distilling knowledge from instruction-tuned LLMs into much smaller ones. While other similar works have been done, they are often conducted on a limited set of (usually still large) models and are not accompanied by proper evaluations. To this end, we carefully develop a large set of 2.58M instructions based on both existing and newly-generated instructions. In addition to being sizable, we design our instructions to cover a broad set of topics to ensure diversity. Extensive analysis of our instruction dataset confirms its diversity, and we generate responses for these instructions using gpt-3.5-turbo. Leveraging these instructions, we fine-tune a diverse herd of models, collectively referred to as LaMini-LM, which includes models from both the encoder-decoder and decoder-only families, with varying sizes. We evaluate the performance of our models using automatic metrics on 15 different natural language processing (NLP) benchmarks, as well as through human assessment. We also assess the model for hallucination and toxicity, and for the former, we introduce a new benchmark dataset for hallucination-inducing QA. The results demonstrate that our proposed LaMini-LM models are comparable to strong baselines while being much smaller in size.

## introduction large language models (llms) with instruction tuning have demonstrated remarkable capabilities in generating high-quality outputs for a diverse set of applications @xcite @xcite @xcite . these models typically consist of billions of parameters, demanding substantial computational resources for both training and inference in this work, we introduce lamini-lm, a collection of language models that stand out due to their smaller size compared to the majority of existing instruction-tuned models. we develop lamini-lm models by employing sequence distillation (also known as offline distillation) @xcite from llms. while previous studies @xcite @xcite have attempted similar approaches, there are several gaps in the current literature that we aim to address. these gaps include: (i) the provision of a small-scale distilled dataset, (ii) limited diversity in the dataset, (iii) a restricted number of models (typically only one), and (iv) a lack of comprehensive evaluation and analysis regarding the performance of the models. additionally, it is important to note that many distilled models resulting from previous work remain computationally demanding. these recent models typically range from 7b to 13b parameters, which presents challenges for deployment in resource-constrained settings. therefore, our objective is to develop a solution that overcomes these limitations and facilitates easier deployment in such settings. to address these challenges, we undertake several steps as shown in figure 1 . firstly, we create a large-scale offline-distillation instruction dataset, consisting of 2.58m examples. we curate these instructions from diverse existing datasets, including self-instruct @xcite , p3 @xcite , flan @xcite , and alpaca @xcite . to augment the dataset, we use the example-guided instruction generation technique with gpt-3.5-turbo to generate additional diverse instructions that match humanwritten prompts in style and quality. 2 we also employ the topic-guided instruction generation technique to enhance instruction diversity by incorporating specific topics of interest from wikipedia. finally, we utilize gpt-3.5-turbo to generate responses for each instruction. the resulting dataset is called the lamini instruction dataset. after creating the dataset, we fine-tune multiple smaller language models with different sizes (ranging from 61m to 7b) and architectures (encoderdecoder and decoder-only). we also conduct extensive experiments and analyses, setting our work apart from previous research. we evaluate their performance on diverse nlp downstream tasks and incorporate human evaluation to assess the quality of model outputs. given the growing power of language models, we recognize the potential risks they pose. hence, we evaluate our lamini language models for hallucination and toxicity. the toxicity assessment utilizes an existing test suite, while we curate a separate test suite with 40 carefully crafted questions to specifically probe hallucination risks. through these comprehensive analyses, we gain deep insights into the models' strengths and weaknesses, enabling us to better understand their potential applications and risks. our contributions can be summarized as follows: large language models supervised fine-tuning with natural language instructions empowers the large language models (llms) to achieve remarkable zero-shot performance on a diverse set of applications @xcite @xcite @xcite . prior studies demonstrate that finetuning vanilla language models with human-written instructions can effectively enable them to follow general language instructions @xcite @xcite @xcite @xcite @xcite . moreover, a recent study by @xcite demonstrates that model-generated instructions can be used for instruction tuning, resulting in significant improvements in vanilla language models' responsiveness to instructions. inspired by these findings, other works have focused on instruction tuning vanilla language models using model-generated instructions @xcite @xcite @xcite . in this study, we present the largest instruction dataset generated by gpt-3.5-turbo to date. we then fine-tune a collection of language models to create our lamini-lm models. knowledge distillation knowledge distillation is a technique that trains a smaller model, called the student, by leveraging knowledge from a larger model, the teacher @xcite . one common method is to train the student to match the teacher's representation, such as logits, output probability, or intermediate activation @xcite @xcite @xcite . for sequence-to-sequence models, sequence-level distillation was introduced by @xcite , where a synthetic output generated by the teacher model is used to train the student. this approach is efficient as it only requires running the teacher model once. previous research has shown the effectiveness of sequencelevel distillation @xcite @xcite . in our work, we adopt sequence-level distillation using the output of gpt-3.5-turbo to train our model. our approach stands out by training on a significantly larger dataset and distilling it into much smaller models. additionally, we provide various student models as part of our contributions. ## dataset generation our approach involves the distillation of knowledge from large language models through sequence/offline distillation @xcite . in this process, the student model learns from the outputs of a teacher model. to create our dataset, we make use of various existing resources of prompts, including self-instruct @xcite and alpaca @xcite as well as random subsets of p3 @xcite and flan @xcite . leveraging these resources, we generate a dataset consisting of 2.58m pairs of instructions and responses using chatgpt. furthermore, we perform an exploratory analysis of the resulting text to gain additional insights. ## results and discussions in this section, we provide evaluation results and a discussion of lamini-lm for both automatic evaluation on the downstream nlp tasks and human evaluation on user-oriented instructions. ## hallucination and toxicity hallucination llms often generate hallucinations, producing text that is either factually incorrect or incoherent. to investigate this problem, we simplify it as a "question rejection" challenge, treating it as a binary classification task. the goal is to determine whether an llm can accurately identify and reject unanswerable or inappropriate questions. an ideal model should reject a question with a justified explanation (if provided). to achieve this, we created the lamini-hallucination test set, 8 which consists of four categories: "did toxicity llms have been observed to demonstrate a tendency to generate toxic language, making their safe deployment challenging. to assess this issue with our lamini-lm models, we utilize the realtoxicityprompts dataset @xcite . we randomly select 1k non-toxic prompts (toxicity @xmath0.1) and 1k toxic prompts (toxicity @xmath1.9) from this dataset. using the instruction prefix "complete the sentence:", we generate outputs using recommended lamini models and their baselines. we then employ the openai moderation api detect the toxicity of the generated outputs, as shown in ## conclusion in this study, we present a large-scale instruction dataset derived from gpt-3.5-turbo, containing over 2.58m examples. we refer to this dataset as the lamini instruction dataset, which currently holds the distinction of being the largest dataset of its kind. our research focuses on distilling knowledge from llms into smaller, more efficient model architectures. we introduce a family of language models called lamini-lm, consisting of 6 encoderdecoder models and 11 decoder-only models with different sizes (ranging from 61m to 7b). through a comprehensive evaluation, including automatic evaluation of downstream nlp tasks and human evaluation of general usage, hallucination, and toxicity, we demonstrate that our proposed models achieve comparable performance to alpaca @xcite while being significantly smaller in size. for the hallucination problem, we carefully curate 40 questions and find out that current llms still face significant challenge in this area. our work sheds light on the process of distilling knowledge from llms to significantly smaller models and the potential of training efficient yet effective language models.