Auto-Instruct: Automatic Instruction Generation and Ranking for Black-Box Language Models (2023)

Abstract: Large language models (LLMs) can perform a wide range of tasks by following natural language instructions, without the necessity of task-specific fine-tuning. Unfortunately, the performance of LLMs is greatly influenced by the quality of these instructions, and manually writing effective instructions for each task is a laborious and subjective process. In this paper, we introduce Auto-Instruct, a novel method to automatically improve the quality of instructions provided to LLMs. Our method leverages the inherent generative ability of LLMs to produce diverse candidate instructions for a given task, and then ranks them using a scoring model trained on a variety of 575 existing NLP tasks. In experiments on 118 out-of-domain tasks, Auto-Instruct surpasses both human-written instructions and existing baselines of LLM-generated instructions. Furthermore, our method exhibits notable generalizability even with other LLMs that are not incorporated into its training process.

## introduction instruction-tuned large language models (llms) have gained considerable popularity as solutions to a myriad of nlp tasks, owing to their proficiency in interpreting natural language instructions @xcite @xcite . as fine-tuning llms often becomes unfeasible, instructions play an increasingly crucial role in prompting such black-box llms. especially in the true few-shot 2 setting @xcite where the user aims to tackle a new task with only a basic task description and a few data order adjectives correctly in english sentences. input: which sentence has the correct adjective order: (a) rubber terrible ship (b) terrible rubber ship output: (b) flan-t5 demonstrations seed instruction in this task, you will be given two sentences with adjectives… generate rank & select + concatenate (b) inference selected instruction 575 training tasks …… candidate instructions which sentence has the correct adjective order: (a) green new chair (b) new green chair ## related work the choice of instructions plays a pivotal role in effectively utilizing llms. to this end, a range of approaches has been implemented, with parametric optimization and llm-based generation standing out as prominent methods. parametric optimization primarily involves utilizing parameters to tune instructions @xcite @xcite . for instance, @xcite employed a gradient-based search over a predetermined length of discrete tokens as the instruction. @xcite further improved this approach by preserving the readability of the sampled tokens through a perplexity constraint. as a more flexible approach, @xcite optimized instruction generation through reinforcement learning, with rewards computed based on the llm output. however, these strategies require access to either llm parameters or a training set for optimization, making them less applicable to black-box llms with only a limited number of available examples. moreover, instructions generated by these methods often lack fluency or even become gibberish, thereby compromising their interpretability. in contrast, the llm-based generation thread selects instructions by directly prompting the llm @xcite @xcite . for example, @xcite were among the first to reveal that llms could write an instruction for a given task after observing just a few demonstrations, and zhou et al. (2022) improved the quality of the generated instructions by selecting the best performed one on the validation data. iprompt @xcite is the most capable method so far with its iterative generation and validation process for selecting instructions. nevertheless, these approaches still necessitate a validation set for instruction ranking, and the instructions they generate typically underperform compared to those written by humans. besides the choice of instructions, researchers have also explored other orthogonal directions of improving llm prompts, such as the selection of in-context demonstrations. some works focused on identifying the most suitable demonstrations from training examples @xcite @xcite and their optimal ordering @xcite in the few-shot prompt. other studies examined the engineering and selection of reasoning chains that are paired with the few-shot demonstrations on multi-step reasoning tasks @xcite @xcite . we reserve the exploration of integrating these orthogonal techniques with our approach to holistically optimize the entire llm prompt for future work. ## problem formulation in this work, we focus on the true few-shot setting where a user aims to tackle a new task with a black-box llm. while it is easy to come up with a handful of examples and a basic description, the user may not have insights into what kind of instructions would be effective for unseen examples. hence, given the few-shot examples as demonstrations and the basic description as a seed instruction, our goal is to automate the process of creating a more effective instruction for the given task. we formulate our problem following the conventional practices of in-context learning @xcite . in the aforementioned few-shot setting, the prompt to query a black-box llm comprises an instruction i, the test input x, and a few input-output pairs as demonstrations @xcite . this work aims to automatically find a superior instruction i ′ based on the human-written seed instruction i s , thereby circumventing the need for substantial manual engineering. besides, we also explore the zero-shot setting where no demonstrations are given to the llm. despite the instruction potentially having multiple ways of integrating with the demonstrations and the test input, to reduce the complexity of the problem, we format the whole prompt in the order of (i, ## auto-instruct auto-instruct is composed of two steps: instruction generation and instruction ranking. we first prompt the black-box llm to generate a diverse set of candidate instructions ( §4.1) for each downstream task. next, we train a scoring model to rank all candidate instructions for each given test example, as different examples can benefit from different instructions ( §4.2). then, the top-ranked instruction is selected to prompt the black-box llm on that specific test example for downstream inference. write a step-by-step instruction on how to solve the following task. task: [seed instruction] examples: input: [input of demonstration #1] output: [output of demonstration #1] …… input: [input of demonstration #n] output: [output of demonstration #n] instruction: figure 2: the meta-prompt that guides the llm to generate a step-by-step instruction for the given task. other meta-prompts are shown in appendix e. ## conclusion in this work, we introduce auto-instruct, an automatic approach of generating, ranking and selecting instructions, which offers a solution to the high cost and subjectivity associated with humanengineered instructions. our approach begins by prompting the llm to generate a diverse set of candidate instructions. next, an instruction ranking model trained on hundreds of tasks is used to rank the candidate instructions and select the most effective one to solve a specific example. experimental results demonstrate that our approach provides better instructions than both human-written ones and those produced by previous instruction generation approaches, as tested on 118 out-of-domain tasks.