G r IPS : Gradient-free, Edit-based Instruction Search for Prompting Large Language Models (2023)

Abstract: Providing natural language instructions in prompts is a useful new paradigm for improving task performance of large language models in a zero-shot setting. Recent work has aimed to improve such prompts via manual rewriting or gradient-based tuning. However, manual rewriting is time-consuming and requires subjective interpretation, while gradient-based tuning can be extremely computationally demanding for large models and may not be feasible for API-based models. In this work, we introduce Gradient-free Instructional Prompt Search (GrIPS), a gradient-free, edit-based search approach for improving task instructions for large language models. GrIPS takes in instructions designed for humans and automatically returns an improved, edited prompt, while allowing for API-based tuning. With InstructGPT models, GrIPS improves the average task performance by up to 4.30 percentage points on eight classification tasks from the Natural Instructions dataset (with similar improvements for OPT, BLOOM, and FLAN-T5). We see improvements for both instruction-only prompts and instruction + k-shot examples prompts. Notably, GrIPS outperforms manual rewriting and purely example-based prompts while controlling for the available compute and data budget. Further, performance of GrIPS is comparable to select gradient-based tuning approaches. Qualitatively, we show our edits can simplify instructions and at times make them incoherent but nonetheless improve accuracy.

## introduction recent advancements in prompting large language models (lms) such as gpt-3 show that models can perform nlp tasks without any task-specific tuning @xcite . most of the work in this area focuses on few-shot learning, where models rely on textual prompts containing input-output example pairs (exemplar prompts). however, humans are often able to perform a new task when provided with a relevant set of instructions or a task description, not necessarily including any examples. in this direction, past works explore a new paradigm of instructional prompts where a prompt is tailored for a particular task by including natural language instructions @xcite . following @xcite , we characterize instructions as a natural language description of the task that includes what is required for a person to complete the task correctly. 2 demonstrative examples of the task are not considered a part of the instructions. for purposes of improving task performance via instructional prompts, @xcite provide a set of guidelines to manually rewrite raw instructions. yet this kind of rewriting process requires substantial manual effort and subjective interpretation of the guidelines. in addition, an underlying assumption in @xcite is that instructions should be semantically coherent to humans. however, it is possible that the prompts that most improve model performance are semantically confusing to humans in some ways. past works attempt to automatically improve prompt quality for large lms by means of prompt tuning @xcite . existing prompt tuning methods use gradient-based approaches which have a few notable shortcomings. first, computing gradients with large lms can be prohibitively computationally demanding. second, this is entirely infeasible for models available only via apis, because model gradients and weights are not standardly accessible. 3 third, output continuous representations may not directly map back onto tokens in the original vocabulary. thus, we cannot verify @xcite [is to classify the tweet ] [as "positive" or "negative"] [based on its content]. ## methodology in this section, we first describe and illustrate different prompt modes ( §3.1). then, in §3.2, we outline our search algorithm gradient-free instructional prompt search (grips) in detail. ## results and discussion in this section, we present the results of our experiments. first, we establish the effectiveness of grips across models in §5.1. then, we compare our search to other methods in §5.2 and §5.3 and provide additional analysis in subsequent sections. ## conclusion we introduce grips, an automatic search algorithm that edits task instructions to improve downstream task performance. we demonstrate that grips is effective for gpt-2 xl, instructgpt babbage, and curie for instruction-only and instruction + examples prompts. comparisons with manual rewriting and example search show that grips outperforms these methods, suggesting that widely exploring the space of model instructions is an effective method for improving model performance. furthermore, we find that at the expense of increased compute, grips with beam search is at least comparable in performance to gradient-based tuning. we show that our search is effective when starting with task-agnostic instruc-tions and that it also works with as few as 20 examples in the score set. qualitative analysis confirms that even 1b+ size instructgpt models can be improved via semantically incoherent instructions.