Plum: Prompt Learning using Metaheuristics (2024)

Abstract: Since the emergence of large language models, prompt learning has become a popular method for optimizing and customizing these models. Special prompts, such as Chain-of-Thought, have even revealed previously unknown reasoning capabilities within these models. However, the progress of discovering effective prompts has been slow, driving a desire for general prompt optimization methods. Unfortunately, few existing prompt learning methods satisfy the criteria of being truly “general”, i.e., automatic, discrete, black-box, gradient-free, and interpretable all at once. In this paper, we introduce metaheuristics, a branch of discrete non-convex optimization methods with over 100 options, as a promising approach to prompt learning. Within our paradigm, we test six typical methods: hill climbing, simulated annealing, genetic algorithms with/without crossover, tabu search, and harmony search, demonstrating their effectiveness in white-box and black-box prompt learning. Furthermore, we show that these methods can be used to discover more human-understandable prompts that were previously unknown in both reasoning and image generation tasks, opening the door to a cornucopia of possibilities in prompt optimization.

## introduction the advent of powerful large language models (llms) @xcite @xcite @xcite has paved the way for numerous real-world applications, including multi-round chat @xcite , instruction following @xcite and reasoning @xcite . to further enhance the performance of llms in specific domains, additional optimization techniques are commonly employed, such as finetuning @xcite , adapter methods @xcite , alignment approaches @xcite , and prompt learning @xcite . notably, prompt learning distinguishes itself from other methods by eliminating the need for gradient information from models, resulting in substantially reduced memory consumption and computational resource requirements. furthermore, prompt learning often yields interpretable outcomes, which help researchers and engineers intuitively understand its effectiveness, thereby being beneficial in inspiring more generalizable prompts for various tasks @xcite @xcite . since the introduction of prompt engineering and prompt learning, significant advancements have been made in the discovery of effective prompts. a noteworthy illustration is chain-of-thought (cot), whereby the simple inclusion of accurate deduction steps for few-shot examples within the original prompt empowers llms to achieve substantial performance improvements in reasoning tasks @xcite ). an even more impressive result is zero-shot-cot @xcite , where adding the magic phrase "let's think step by step" produces a remarkable accuracy gain of over 10% across multiple models engaged in a diverse spectrum of reasoning tasks. however, the quest for such highly effective prompts remains unfulfilled, underscoring the need for tools that accelerate the discovery process. ideally, these prompt learning tools should possess a notable level of generality, while simultaneously meeting the following criteria: in this paper, we present a novel approach to simultaneously achieve these desirable properties by integrating prompt learning with metaheuristics. by treating prompt learning as a non-convex discrete optimization problem within a black-box framework, we harness the potential of metaheuristics, which offer interpretable and automated optimization processes. moreover, this domain of optimization research encompasses a vast array of over 100 distinct algorithms @xcite , ushering in a new paradigm for prompt learning. many of these algorithms even boast robust theoretical guarantees, ensuring the discovery of both local and even global optimal solutions within limited time frames @xcite @xcite @xcite . ## methods in this paper, we propose prompt learning using metaheuristic (plum), a general paradigm that enables the application of numerous metaheuristics in the setting of discrete black-box prompt learning. based on this framework, the flexible composition of different techniques becomes possible, leading to a "template" for designing general prompt learning algorithms, guaranteed to simultaneously satisfy the properties of automatic, discrete, blackbox, gradient-free, and interpretable. ## experiments we conduct three types of experiments to demonstrate the empirical superiority of proposed algorithms. as a starting point, we first show plum's computational efficiency by applying them to general prompt learning tasks with white-box models, where plum produces nontrivial improvements over baselines within a fixed time limit. we further investigate plum on prompt learning tasks with blackbox models, where only discrete-input apis are available. on these tasks, plum also obtains significant performance gains with much fewer api calls. eventually, we show that plum is capable of discovering effective prompt patterns in both chain-of-thought and text-to-image generation tasks, making it a promising paradigm for boosting research methods accuracy(%) bbt @xcite 53.67±1.71 ‡ bdpl @xcite 53.13±0.61 grips @xcite 53.67±0.87 apo @xcite 54.63±0. ## discussion plum with llm-driven mutators llm-based prompt tuning methods are emerging as one of the most competitive approaches in prompt learning. recent works like @xcite ; @xcite have shown significant performance improvements through utilizing the llm-driven prompt mutators, highlighting the success of such approaches. compared with them, plum offers a complementary and orthogonal paradigm. for ex-ample, by incorporating the llm-based operators in plum algorithms, our chain-of-thought task experiments in appendix b.2 demonstrate effectiveness in generating high-quality, interpretable fewshot-cot prompts for reasoning tasks (as shown in ## conclusion in this paper, a novel paradigm of prompt learning is proposed. by formulating prompt learning as a black-box discrete optimization problem, we are able to apply various metaheuristics algorithms to help discover effective prompts, while guaranteeing the whole process to be automatic, discrete, gradient-free, interpretable, and applicable to black-box models. six typical plum algorithms, hill climbing, simulated annealing, genetic algorithm (mutation only), genetic algorithm (with crossover), tabu search, and harmony search all obtain non-trivial improvements over grips on instruction following tasks. furthermore, with the assistance of metaheuristics-based prompt learning, we are capable of discovering new patterns of effective prompts unexplored in the past.