I nst O ptima: Evolutionary Multi-objective Instruction Optimization via Large Language Model-based Instruction Operators (2023)

Abstract: Instruction-based language modeling has received significant attention in pretrained language models. However, the efficiency of instruction engineering remains low and hinders the development of instruction studies. Recent studies have focused on automating instruction generation, but they primarily aim to improve performance without considering other crucial objectives that impact instruction quality, such as instruction length and perplexity. Therefore, we propose a novel approach (i.e., InstOptima) that treats instruction generation as an evolutionary multi-objective optimization problem. In contrast to text edition-based methods, our approach utilizes a large language model (LLM) to simulate instruction operators, including mutation and crossover. Furthermore, we introduce an objective-guided mechanism for these operators, allowing the LLM to comprehend the objectives and enhance the quality of the generated instructions. Experimental results demonstrate improved fine-tuning performance and the generation of a diverse set of high-quality instructions.

## introduction with the rapid development of language models @xcite @xcite , instructions (also known as prompts) play a crucial role in instruction-based language modeling, and different instructions may lead to significant differences in model outputs @xcite @xcite . for instance, even slightly perturbed instructions (e.g., synonym substitutions @xcite or adversarial attacks @xcite ) can result in unexpectedly low performance. however, there are three problems regarding instruction-based learning that still need to be addressed in existing works. firstly, existing works @xcite @xcite @xcite aim to obtain a large number of instructions through automated instruction generation to filter high-performance instructions. however, due to the large and non-differentiable textual search space @xcite , the automated instruction generation and instruction engineering methods @xcite are inefficient and struggle to search for various high-quality instructions. secondly, the objectives of instruction generation are not clear. current research @xcite @xcite regards performance (i.e., metrics) as the sole criterion for instruction quality. however, model performance alone cannot precisely explain instruction quality. we propose to refine instruction quality by considering fine-grained objectives, such as length and perplexity. shorter instructions can lower computational costs, especially for large-scale models and datasets. lower perplexity indicates that instructions are more easily understood by language models. lastly, the diversity of instructions has been neglected in existing studies, while increasing the diversity of instructions can mitigate adversarial attacks @xcite and improve instruction robustness @xcite . we aim to obtain multiple alternative instructions based on multi-objective optimization, which can facilitate comprehensive evaluation of instructions. to address these three problems, we formulate the task as an evolutionary multi-objective optimization problem and propose our framework called instoptima. we leverage a large language model, specifically chatgpt (openai, 2023), to facilitate instruction operations such as mutation and crossover. furthermore, we introduce an objective-guided mechanism to assist the language model in generating high-quality instructions. in terms of optimization objectives for instruction generation, instoptima incorporates three objectives: performance (metrics), length, and perplexity, enabling the exploration of a diverse and high-quality set of instructions. we adopt nsga-ii @xcite in instoptima to obtain a pareto front of instruction sets. to validate the efficacy of instoptima, we conducted experiments on three generation-based classification tasks. the experimental results indicate that instoptima can concurrently obtain a diverse set of instructions that outperform the counterparts regarding performance. in summary, our contributions are as follows: • we simulate instruction operators based on an llm. we also show that the objective-guided operators help the llm understand optimization objective values and improve instruction quality. ## proposed method in this section, we first introduce the instructionbased text generation, followed by the details of instoptima. ## experimental setup we conducted a comprehensive set of experiments foot_2 to validate the performance of instoptima. the detailed experiments setups and implementations are described in appendix a.1. ## conclusion we propose a multi-objective instruction optimization framework to obtain a diversified set of instructions. to address the challenges posed by the large and non-differentiable text search space, instoptima utilizes objective-guided instruction operators based on llm, which shows impressive performance in instruction generation. however, it is important to note that multi-objective instruction optimization is still in the early stages and requires further research in the future.