Plum: Prompt Learning using Metaheuristics (2024)

Abstract: Since the emergence of large language models, prompt learning has become a popular method for optimizing and customizing these models. Special prompts, such as Chain-of-Thought, have even revealed previously unknown reasoning capabilities within these models. However, the progress of discovering effective prompts has been slow, driving a desire for general prompt optimization methods. Unfortunately, few existing prompt learning methods satisfy the criteria of being truly "general", i.e., automatic, discrete, black-box, gradient-free, and interpretable all at once. In this paper, we introduce metaheuristics, a branch of discrete nonconvex optimization methods with over 100 options, as a promising approach to prompt learning. Within our paradigm, we test six typical methods: hill climbing, simulated annealing, genetic algorithms with/without crossover, tabu search, and harmony search, demonstrating their effectiveness in white-box and blackbox prompt learning. Furthermore, we show that these methods can be used to discover more human-understandable prompts that were previously unknown in both reasoning and image generation tasks, opening the door to a cornucopia of possibilities in prompt optimization.

heading: 1 Introduction
text: The advent of powerful large language models (LLMs) (Devlin et al., 2018; Radford et al., 2019; Brown et al., 2020; Team et al., 2023; Roziere et al., 2023; Touvron et al., 2023a,b) has paved the way for numerous real-world applications, including multi-round chat (OpenAI, 2023; Touvron et al., 2023b), instruction following (Ouyang et al., 2022) and reasoning (Roziere et al., 2023; Azerbayev et al., 2023). To further enhance the performance of LLMs in specific domains, additional optimization
* Equal contribution. † Work done during internship at the Hong Kong Univer-
sity of Science and Technology.
techniques are commonly employed, such as finetuning (Chiang et al., 2023), adapter methods (Hu et al., 2021; Dettmers et al., 2023), alignment approaches (Ouyang et al., 2022; Dong et al., 2023), and prompt learning (Sun et al., 2022b; Diao et al., 2022). Notably, prompt learning distinguishes itself from other methods by eliminating the need for gradient information from models, resulting in substantially reduced memory consumption and computational resource requirements. Furthermore, prompt learning often yields interpretable outcomes, which help researchers and engineers intuitively understand its effectiveness, thereby being beneficial in inspiring more generalizable prompts for various tasks (Prasad et al., 2022; Guo et al., 2023; Yu et al., 2023).
Since the introduction of prompt engineering and prompt learning, significant advancements have been made in the discovery of effective prompts. A noteworthy illustration is Chain-of-Thought (COT), whereby the simple inclusion of accurate deduction steps for few-shot examples within the original prompt empowers LLMs to achieve substantial performance improvements in reasoning tasks (Wei et al., 2022b). An even more impressive result is Zero-shot-COT (Kojima et al., 2022), where adding the magic phrase “Let’s think step by step” produces a remarkable accuracy gain of over 10% across multiple models engaged in a diverse spectrum of reasoning tasks.
However, the quest for such highly effective prompts remains unfulfilled, underscoring the need for tools that accelerate the discovery process. Ideally, these prompt learning tools should possess a notable level of generality, while simultaneously meeting the following criteria:
• Automatic: since human involvements are normally expensive and time-consuming.
• Discrete: as commercial LLMs usually provide APIs that only accept discrete in-
2177
puts (OpenAI, 2023; Team et al., 2023). • Black-box: applicable to black-box LLMs. • Gradient-free: hence still possess the good
property of low memory cost and time cost. • Interpretable: thus researchers can understand
the logics behind its effectiveness and extend its applications with ease. Regretfully, to the best of our knowledge, only a limited number of prompt learning methods meet the criteria for being considered “general” as per the aforementioned definition. Manual prompt engineering techniques employing handcrafted prompts (Mishra et al., 2021; Kojima et al., 2022; Bsharat et al., 2023) lack automation, while continuous prompt learning approaches (Liu et al., 2021a; Li and Liang, 2021) fail to satisfy the requirements of discreteness and interpretability. Prompt learning methods employing reinforcement learning (Deng et al., 2022) often necessitate an additional neural network, the learning process of which is typically non-interpretable, and their theoretical guarantees are only attainable under stringent assumptions.
In this paper, we present a novel approach to simultaneously achieve these desirable properties by integrating prompt learning with metaheuristics. By treating prompt learning as a non-convex discrete optimization problem within a black-box framework, we harness the potential of metaheuristics, which offer interpretable and automated optimization processes. Moreover, this domain of optimization research encompasses a vast array of over 100 distinct algorithms (Hussain et al., 2019), ushering in a new paradigm for prompt learning. Many of these algorithms even boast robust theoretical guarantees, ensuring the discovery of both local and even global optimal solutions within limited time frames (Locatelli, 2000; Glover and Hanafi, 2002; Schmitt, 2004; Dorigo et al., 2006; He et al., 2018).

heading: 2 Related Work

heading: 2.1 Prompt Learning
text: Prompt-based learning is an effective approach that harnesses the power of large language models (LLMs) to facilitate downstream tasks by extracting relevant knowledge. This approach offers several advantages, notably in terms of computational efficiency, as only a small set of parameters needs to be optimized, in contrast to traditional methods that require fine-tuning the entire LLM for each
task. Prompting methods can be categorized into discrete prompts (Davison et al., 2019; Wallace et al., 2019; Jiang et al., 2020; Shin et al., 2020; Haviv et al., 2021; Yuan et al., 2021; Gao et al., 2021; Ben-David et al., 2022; Su et al., 2022a; Diao et al., 2022; Datta et al., 2023; Pan et al., 2024) and continuous prompts (Hambardzumyan et al., 2021; Zhong et al., 2021; Han et al., 2021; Li and Liang, 2021; Qin and Eisner, 2021; Liu et al., 2021a) based on the format of prompts. Discrete prompts are typically represented as token sequences or natural language phrases, while continuous prompts are designed as sequences of vectors. Significant advancements have been made in leveraging LLMs for reasoning tasks using chainof-thought prompting techniques. Notable progress is evidenced by the research of Wei et al. (2022a); Wang et al. (2022); Zhou et al. (2023); Zhang et al. (2022); Shum et al. (2023); Diao et al. (2023), in which well-structured prompts guide the models to showcase superior reasoning capabilities, surpassing previous benchmarks. Further studies in prompt attacks (Yu et al., 2023; Zhu et al., 2023) have also utilized several of these advancements to improve the robustness of modern LLMs. However, most of the prior studies on prompting have been limited to a white-box setting, requiring access to all parameters of a large language model. The reliance on white-box optimization methods has become less practical in light of the widespread use of closed-source black-box models.

heading: 2.2 Black-Box Prompt Learning
text: Black-box prompt learning (BPL) (Diao et al., 2022; Sun et al., 2022b,a) represents a promising research direction that tunes prompts without requiring access to the parameters and gradients of the underlying LLMs. BPL proves particularly valuable for closed-source models, which have demonstrated superior performance compared to opensource models (Liang et al., 2022). This advancement allows for the effective optimization of closedsource models, overcoming the previous limitations posed by the unavailability of model parameters. Similar to white-box prompts, black-box prompts can be categorized into discrete prompts (Diao et al., 2022; Prasad et al., 2022; Deng et al., 2022; Hou et al., 2022; Cheng et al., 2023) and continuous prompts (Sun et al., 2022b,a; Su et al., 2022b). However, in the black-box setting, discrete prompts prove more practical as the interface of LLMs only accepts discrete inputs and cannot process continu-
ous vectors.

heading: 2.3 Metaheuristics
text: There have been efforts combining the power of metaheuristics with black-box prompt learning in the past literature. For instance, GrIPS (Prasad et al., 2022) applied greedy search methods, as well as simulated annealing (Kirkpatrick et al., 1983), to search better prompts with simple edit operations such as addition, swap, paraphrase, and deletion. GPS (Xu et al., 2022) utilized genetic algorithms for few-shot prompt learning, improving handcrafted prompts on multiple tasks. Additionally, Kumar and Talukdar (2021) and Lu et al. (2021) investigated the impact of example orders in few-shot learning settings, respectively improving performance through the use of genetic algorithms and alternate search heuristics. However, these methods are limited in their applicability, focusing on specific prompt learning settings and failing to fully explore the inherent potential of discrete optimization, which is a bridge between black-box prompt learning and general metaheuristics. As an attempt to overcome this limitation, (Guo et al., 2023) investigated the performance of combining the rephrasing power of LLMs with Genetic Algorithm and Differential Evolution, showing superior performance over hand-crafted prompts. However, compared with our work, Guo et al. (2023) focused more on leveraging the paraphrasing ability of LLMs and restricted search strategy to only evolutionary algorithms, which is a subset of metaheuristics.
Metaheuristics is a well-established and versatile branch of discrete optimization, known for its effectiveness in solving various non-convex optimization problems. This field has witnessed the development of over 100 different methods (Hussain et al., 2019), which have been successfully applied to a wide range of problem domains. For instance, they have been utilized in Neural Architecture Search (Elsken et al., 2019; Liu et al., 2021b), Travelling Salesman Problem (Ouaarab et al., 2014; Hussain et al., 2017; Grabusts et al., 2019), and Program Search (Chen et al., 2023). Prominent algorithms in this field include simulated annealing (Kirkpatrick et al., 1983), genetic algorithms (Holland, 1992), tabu search (Glover, 1986), harmony search (Geem et al., 2001), ant colony optimization (Dorigo and Gambardella, 1997), among others. However, the integration of these powerful metaheuristics with black-box
prompt learning remains largely unexplored, presenting rich opportunities for further research in this direction.

heading: 3 Methods
text: In this paper, we propose Prompt learning using metaheuristic (Plum), a general paradigm that enables the application of numerous metaheuristics in the setting of discrete black-box prompt learning. Based on this framework, the flexible composition of different techniques becomes possible, leading to a “template” for designing general prompt learning algorithms, guaranteed to simultaneously satisfy the properties of automatic, discrete, blackbox, gradient-free, and interpretable.

heading: 3.1 Plum in General
text: Algorithm 1 Plum in General Required: (1) A well-defined neighbor set N (p) for ∀p ∈ Ω; (2) a metaheuristics algorithm A; (3) metaheuristics-dependent hyperparameters θ and functions F .
Input: An initialized prompt p0 and an objective f(p) to be maximized (or minimized).
1: p∗ ←AN ,θ,F ,f (p0,M) 2: return p∗
The framework of Plum is outlined in Algorithm 1, which encompasses four fundamental and orthogonal elements. The first element is a welldefined neighborhood N (·) in the discrete prompt search space. In practice, any operations that transform a prompt into another prompt can be used to define such neighborhoods. For instance, edit operations in GrIPS (Prasad et al., 2022) can convert a prompt p = [Let, us, think] to a new prompt p′ = [Let, us, brainstorm] via paraphrase, and all such possible conversions form prompt p’s neighborhood
N (p) = {p′|∃e ∈ E , s.t. p′ = edit(p, e)}. This abstraction of neighborhood decouples prompt transformations and search algorithms, making the flexible combination of those techniques possible.
The second part is the core metaheuristics A, such as simulated annealing (Kirkpatrick et al., 1983), tabu search (Glover, 1986), genetic algorithms (Holland, 1992). Any discrete optimization metaheuristics can fit into this part.
Notably, various metaheuristics come with their own specific hyperparameters and functions that need to be considered within the Plum paradigm. For instance, in the case of simulated annealing, a temperature scheduler is required for regulating the annealing speed, where an appropriate schedule can greatly improve the search efficiency. Similarly, general genetic algorithms incorporate a “crossover” operation, which swaps segments of two prompts and serves as a global search mechanism to expand its search scope.
During the optimization process, the objective is to maximize (or minimize) a provided function f(p), which typically corresponds to task-specific performance metrics of the target LLM. For instance, under the setting of GPT-3-babbage (Brown et al., 2020) on Commonsense Question Answering (Talmor et al., 2019), the function f(p) represents the final accuracy achieved by augmenting GPT-3-babbage with the prompt p. The primary goal of the metaheuristics is to search for the optimum p which maximizes this accuracy.
By clearly stating those key components in Plum, we are now prepared to instantiate Plum with specific metaheuristics and neighborhood definitions that can perform real-world prompt learning tasks.

heading: 3.2 Plum in Practice
text: As a proof of concept, we realize Plum with several popular metaheuristics, as listed in Table 1. Detailed pseudocodes of those methods are available in Appendix C.
All those algorithms are successful products of
metaheuristics under the setting of prompt learning. By combining the edit operations employed in GrIPS (Prasad et al., 2022) with simulated annealing (Kirkpatrick et al., 1983), we developed an algorithm capable of recovering global optimum with proper temperature scheduler (Granville et al., 1994). In this context, we define the global optimum as the prompt that maximizes the objective function f(p) within the set of reachable prompts starting from the initial prompt p0.
Furthermore, replacing simulated annealing with genetic algorithms gives birth to Plum-GA-M and Plum-GA-C, whose metaheuristics prototypes are proven to converge to global optimum given specific mutation and crossover operations (Schmitt, 2004). Compared with Plum-GA-M, Plum-GA-C adopts an extra operation of crossover, which enables more aggressive searches. Intuitively, Plum-GA-C > Plum-GA-M > Plum-SA > Plum-HC in terms of their exploration power, which is at the expense of increased search steps and the number of evaluations of the objective function f(p).
Similarly, adopting Tabu Search (Glover, 1986) and Harmony Search (Geem et al., 2001) brings forth Plum-TS and Plum-HS, both are API-efficient methods according to our experiments. Here Tabu Search introduces a Tabu list to prohibit certain candidates from being revisited, hence avoids the search process from getting stuck in a local optimum. Meanwhile, Harmony Search obtains its inspiration from musical composition, utilizing a harmony search memory with past prompts and generates the new prompt by combining segments
from them.

heading: 3.3 Plum-GA-C
text: Here we illustrate one specific algorithm Plum-GA-C as an example of instantiating a Plum algorithm. Plum-GA-C contains three stages: parsing, editing, and scoring, which are described as follows.

heading: 3.3.1 Stage 1: Parsing
text: Initially, we parse the initial prompt I into a sequence of phrases using a CRF-based constituency parser (Zhang et al., 2020a). Subsequently, we iteratively merge the leaf nodes of the obtained constituency tree until several disjoint phrase-level constituents are generated.

heading: 3.3.2 Stage 2: Editing
text: Mutation Mutation is a combination of edit operations inside the prompt. Following Prasad et al. (2022), we introduce four mutation operations including: delete, add, swap, and paraphrase. delete randomly drops some phrases while addition adds some randomly sampled phrases back at a random position. swap swaps two phrases, and paraphrase replaces a phrase with a new phrase generated by a paraphrasing model. We
apply PEGASUS (Zhang et al., 2020b) as the paraphrase model to obtain new phrases. The upper right corner of Figure 1 provides an illustrative depiction of the mutation process.
Crossover Crossover is a combination of edit operations across two prompts. To perform crossover, we divide each prompt into two sub-sequences at a randomly selected position. From each prompt, we randomly choose one sub-sequence and these selected sub-sequences are then combined to create a new sequence as the prompt. The upper left corner of Figure 1 provides a visual representation of the crossover process.

heading: 3.3.3 Stage 3: Scoring
text: The optimization process relies on performance signals acquired through the scoring process. Initially, we construct a score set S = {(x1, y1), (x2, y2), ..., (xn, yn)} by randomly sampling n examples from the training split. Subsequently, a score function is computed based on the accuracy of predictions.

heading: 3.4 Plum-HS
text: We also introduce Plum-HS, a novel algorithm that integrates Harmony Search into prompt learning.
Notably, it marks the first instance of Harmony Search being successfully applied to prompt optimization tasks. The process is quite similar to composing new songs, where new prompts are generated by combining segments from past prompts together. The procedure starts with a base prompt pending to be optimized, which is first divided into segments of words, and inserted into the harmony memory.
At each iteration, a new prompt is generated by composing ks segments from prompts in the harmony search memory, where the j-th segment is the exact copy of the j-th segment from a randomly sampled prompt in the memory. After this composition procedure, a pitch finetuning process is then applied, which edits the segment with a small probability in a similar manner as the mutation operation in Section 3.3.2. The only difference is that it utilizes two kinds of pitch change operations: small pitch changes which only do paraphrasing, and big pitch changes which include all editing operators. The generated new prompt is then evaluated and inserted back into the harmony search memory, where the memory only retains NH prompts with top scores.
Intuitively speaking, Plum-HS has a similar exploration power as Plum-GA-C and adopts a more flexible manner to replace long segments in the middle of the prompt. According to our experiments, it is also API-efficient, which achieves better performance while consuming significantly fewer API calls compared to other methods.
Besides Plum-GA-C and Plum-HS, details of other algorithms are also available in Appendix C.

heading: 4 Experiments
text: We conduct three types of experiments to demonstrate the empirical superiority of proposed algorithms. As a starting point, we first show Plum’s computational efficiency by applying them to general prompt learning tasks with white-box models, where Plum produces nontrivial improvements over baselines within a fixed time limit. We further investigate Plum on prompt learning tasks with blackbox models, where only discrete-input APIs are available. On these tasks, Plum also obtains significant performance gains with much fewer API calls. Eventually, we show that Plum is capable of discovering effective prompt patterns in both Chain-ofThought and text-to-image generation tasks, making it a promising paradigm for boosting research
in prompt learning.

heading: 4.1 Plum for White-Box Prompt Learning
text: Dataset Our experiments are conducted on a subset of the Natural-Instructions datasets v2.6 release (Mishra et al., 2022), specifically focusing on eight binary classification tasks (task 019, task 021, task 022, task 050, task 069, task 137, task 139, and task 195). This dataset evaluates the instruction following ability of models, where proper prompt design plays an important role.
Baselines We compare our method with three popular prompt learning methods:
• BDPL (Diao et al., 2022): using variancereduced policy gradient algorithm to estimate the gradients of parameters in the categorical distribution of each discrete prompt.
• BBT (Sun et al., 2022b): optimizing continuous prompts by covariance matrix adaptation evolution strategy.
• GrIPS (Prasad et al., 2022): generating discrete prompts by performing phrase-level operations, and selecting the best ones.
• APO (Pryzant et al., 2023): automatically improve the initial prompt using beam search guided by textual “gradient”.
Experiment Setup To make the comparison fair for all algorithms, we follow the same editing operators as GrIPS (Prasad et al., 2022), which generally generates discrete prompts by performing phrase-level operations and selecting the best ones. We utilize GPT2-large (Radford et al., 2019) for
the backbone model of prompt learning. To accommodate computational resource limitations, we set the batch size to 1 and the time-out threshold of the searching process to 45 minutes for all methods. All methods are given a task-agnostic instruction: You will be given a task. Read and understand the task carefully, and appropriately answer [list_of_labels]. Here [list_of_labels] serves as a placeholder to be replaced by an actual list of labels.
Results As demonstrated in Table 2, with limited time resources, Plum can provide generally on-par or better performance than baseline prompt learning methods. It is worth noticing that this is achieved even without the need for model gradients or any assistance from external LLMs, which shows the potential of Plum for offline prompt optimization scenarios where efficiency is of great importance. More experimental results are available in Appendix B.1.

heading: 4.2 Plum for Black-Box Prompt Learning
text: Experimental Setup The experimental setup follows the same setting as white-box prompt learning, except for the backbone model, where we adopt GPT3-babbage instead. Notice that BBT (Sun et al., 2022b) is no longer applicable in this setting since it requires continuous gradients in its prompt learning processes.
Results Table 3 lists the performance of Plumseries algorithms, where under both scenarios of a limited number of iterations and API calls,
Plum-HS achieves the best performance. Specifically, Plum-HS obtains this superior performance at a cost of much fewer API calls, rendering it a promising candidate for other prompt-tuning tasks. It is also worth noticing that given approximately the same amount of API calls, all Plum algorithms surpass GrIPS by a non-trivial margin.

heading: 4.3 Plum for Prompt Discovery
text: Text-to-Image Generation As demonstrated by images in Table 4, Plum can also be applied to textto-image generation tasks and obtain non-trivial improvements even without the assistance of external LLMs like GPT-3 (Brown et al., 2020). By utilizing an offline image evaluator called PickScore (Kirstain et al., 2023), optimizing image relevance to the original prompt can be formulated as a black-box prompt learning problem, where a prompt’s score is defined as the averaged PickScore of the images generated from the prompt. PickScore is a CLIP-based scoring function trained on a corpus of high-quality image text pair dataset. This makes improving the image quality quantitatively possible. For example, in the fourth image of Table 4, where the image topic is “cat in a library”, Plum-HS augments the original prompt “A cat prowling in a library at night, books and shadows, silent observer”, and replaces it with a high-score prompt “A Is that true? Not at all cat prowling in shadows And books shadows”. Notice the resultant prompt highlights the key information “cat, books, shadows” and appends a prefix “Is that true?” with a sense of mystery. To the best of our
knowledge, this pattern was unexplored before and can be beneficial for future prompt engineering development in this field. More experimental results and details are available in Appendix A.
Reasoning Another important feature of Plum is its compatibility with external LLM search operators for paraphrasing, which allows the discovery of interpretable prompt patterns. We apply this version of Plum to improve task-oriented LLMs’ performance, where the detailed cases are presented in Table 6 of Appendix A. Via studying the characteristics of the searched prompts, we found that the performance of initial prompts can be boosted by further completion of the logical chain. This way, the logical coherence of prompts can be improved while still retaining their intended meaning. For instance, ‘If there are 3 cars in the parking lot and 2 more cars arrive, how many cars are in the parking lot (total number of cars in the parking lot + 2)’, where GPT-3 completes the content inside the parentheses, leading to an easier Chain-of-Thought reasoning for LLMs.
Furthermore, it is observed that the quality of the prompts can be enhanced by incorporating a more comprehensive and lucid explanation of the statement. To illustrate, in a case of prompting for AQuA, GPT3.5 provides an initial explanation of the term ‘keystrokes’ with ‘(the act of pressing a key on a keyboard)’ to ensure clarity, and further gives a precise description for the range ‘from 1 to 500’ with ‘(inclusive)’. This suggests that knowledge-augmented prompt learning methods can be beneficial for black-box prompt tuning.

heading: 5 Discussion
text: Plum with LLM-driven Mutators LLM-based prompt tuning methods are emerging as one of the most competitive approaches in prompt learning. Recent works like Pryzant et al. (2023); Guo et al. (2023) have shown significant performance improvements through utilizing the LLM-driven prompt mutators, highlighting the success of such approaches. Compared with them, Plum offers a complementary and orthogonal paradigm. For ex-
ample, by incorporating the LLM-based operators in Plum algorithms, our Chain-of-Thought task experiments in Appendix B.2 demonstrate effectiveness in generating high-quality, interpretable fewshot-CoT prompts for reasoning tasks (as shown in Table 6). This suggests that LLMs can be a powerful tool for prompt exploration within the Plum paradigm, particularly when computational resources are abundant.
Strengths of Different Plum Algorithms In both white-box and black-box settings, our experiments demonstrate that Plum-HS achieves significant performance gains on prompt tuning tasks against baselines and the other Plum algorithms. This is because it takes into account the composability of different prompt segments, which can be seen as an extension of evolutionary algorithms. Notably, in tasks like image generation (detailed in Section 4.3), we observe that updates to different segments can occur in parallel, highlighting this approach’s efficiency.
On the other hand, while two baseline methods may have superior performance over most other Plum algorithms, those metaheuristics possess nice properties that can be beneficial for the community.
• Plum-HC stands out for its simplicity and efficiency among Plum family. It shares the same form of GrIPS (Prasad et al., 2022) and is one of the most commonly adopted paradigms in LLM-based methods. Plum-HC can serve as a foundational algorithm for developing new mutators. Its straightforward nature allows for easier debugging and comparison when developing more complex algorithms in the Plum family.
• For Plum-SA, it boasts a quite simple form, which only differs one line from Plum-HC. Crucially, it guarantees to find the global optimum with the infinite time horizon. This characteristic makes it valuable for future theoretical analyses of discrete prompt learning.
• Plum-GA-M and Plum-GA-C, the genetic algorithm, can be considered one of the most widely adopted algorithms in evolutionary search, whose results can be useful for future comparisons on the same tasks for other researchers. Additionally, our experiments also demonstrate that these algorithms achieve superior performance compared to baseline approaches.
• For Plum-TS, although its performance is not the best, it achieves on-par performance as GrIPS in 4 times fewer API calls (as shown in Table 3). This advantage makes Plum-TS an extremely favorable choice for applications with limited budgets or API call quotas.

heading: 6 Conclusion
text: In this paper, a novel paradigm of prompt learning is proposed. By formulating prompt learning as a black-box discrete optimization problem, we are able to apply various metaheuristics algorithms to help discover effective prompts, while guaranteeing the whole process to be automatic, discrete, gradient-free, interpretable, and applicable to black-box models. Six typical Plum algorithms, Hill Climbing, Simulated Annealing, Genetic Algorithm (Mutation Only), Genetic Algorithm (with Crossover), Tabu Search, and Harmony Search all obtain non-trivial improvements over GrIPS on instruction following tasks. Furthermore, with the assistance of metaheuristics-based prompt learning, we are capable of discovering new patterns of effective prompts unexplored in the past.
Limitations
As an early step of bridging metaheuristics and prompt learning in a unified paradigm, we only implement six algorithms to prove the validity of this concept. Nevertheless, there are still a bunch of other potential metaheuristics that can be incorporated into this framework, producing more empirically useful methods for solving real-world problems. We hope Plum is just a starting point, more metaheuristics, such as Particle Swarm Optimization (Kennedy and Eberhart, 1995), Ant Colony Optimization (Dorigo and Gambardella, 1997) and Cuckoo search (Yang and Deb, 2009) can be combined with more neighborhood definitions, like retrieval-based completions and multimodal, to inspire more practical algorithms, which can eventually help us find a way to properly communicate with Large Language Models.
The potential risks of this work are the general risks of black prompt tuning methods, where more powerful prompt learning algorithms enable more powerful prompt attack approaches. Nevertheless, the silver lining is that most prompt attack methods normally in term generate adversarial samples that help improve the robustness of models, which can be considered beneficial in the long term.

heading: A Discovered Prompt Patterns
text: Text-to-Image Generation Several detailed prompts in text-to-image generation tasks are presented in Table 5. It can be observed that those improved prompts have a tendency to focus more on key descriptive elements in the image, such as “shadows”, “shimmer in petals” or “sunset”. The augmented prompts also use “There is” or “They have” to emphasize the main theme of the image and remove other distractive words.
Reasoning Here we present several typical cases of discovered prompt patterns for reasoning tasks, as shown in Table 6. It turns out that 1) completing the logical chain, 2) explaining the meaning of a term, or 3) providing additional clarification of a statement all help LLMs improve performance in reasoning tasks.

heading: B Additional Experimental Results
text: B.1 Plum for White-Box Prompt Learning
More experimental results on Phi-2 (Javaheripi and Bubeck, 2024) and TinyLlama (Zhang et al., 2024) are shown in Table 7, where we still observe on-par or better performance when compare Plum with baseline methods. Additionally, Figure 2 illustrates the performance (accuracy) of Plum-HS on the Natural-Instructions dataset subtasks as the number of iterations increases, highlighting the effective-
ness and efficiency of our proposed paradigm.
B.2 Plum for Chain-of-Thought
The Plum paradigm offers a natural approach to obtaining high-quality few-shot-CoT prompts for improving LLM’s reasoning performances. To ensure that this class of prompts still remains logical and interpretable after optimization, we have made further improvements to the search process by introducing an additional LLM. Specifically, we encourage the language models to provide additional
information and details in the add operation and introduce a teaching mechanism to guide the language models in revising prompts. On top of that, we empower the language models to autonomously split the prompts into phrases based on their own comprehension of the underlying text structure.
As depicted in Figure 3), these enhancements utilize the power of modern LLMs, aiming to accelerate the searching process of interpretable and target-model-friendly prompts via improved neighborhood definitions.
Dataset We test our methods on math word solving and commonsense reasoning tasks, including GSM8k (Cobbe et al., 2021), ASDiv (Miao et al., 2020), SVAMP (Patel et al., 2021), AQuA (KoncelKedziorski et al., 2016), CSQA (Talmor et al., 2019), and StrategyQA (Geva et al., 2021). Those tasks evaluate models’ reasoning ability, where prompts are expected to elicit their logical thinking skills.
Experiment Setup Plum is applied to improving the performance of two backbones: GPT3 (textcurie-001) and GPT3.5 (gpt-turbo-3.5-0301) on aforementioned five reasoning tasks. Specifically, we employ Plum-GA-M* and Plum-HS* for this setting, majorly based on two reasons: 1) The effectiveness of both algorithms has been demonstrated in the previous general prompt learning setting, making them reasonable candidates for further extensions; 2) Compared with Plum-GA-C, both algorithms have lower computational complexity and similar exploration ability. Moreover, considering the API budgets and the extensive size of the test set for both math word solving and commonsense reasoning tasks, we chose to exclusively showcase the optimization performance of Plum-GA-M* and
Plum-HS* on the training sets for reference.
Results In this section, we present the enhancement achieved by the few-shot-CoT prompts searched by Plum across five demanding downstream tasks, as shown in Table 8. These results clearly demonstrate the superiority of Plum-GA-M* and Plum-HS* over standard few-shotCoT prompts used for GPT3 and GPT3.5. In particular, our methods achieve universal improvements across those five tasks on almost all the backbone models.
In addition, despite the already impressive performance of GPT3.5 with standard few-shot-CoT prompts, we continue to observe even greater average improvements compared to GPT3. This is due to two key factors. Firstly, the edit operations we have developed in this section allow GPT3.5 to generate better prompts for itself. Secondly, GPT3.5 exhibits stronger capabilities in comprehending and leveraging the enhanced prompts, further improving its overall performance.
Experimental Details The initial few-shot-CoT prompts provided followed the Instruction + Examples format, consisting of a task-agnostic instruction and several positive examples (details are shown in Figure 4, which is consistent with the setting described in (Prasad et al., 2022). The Examples part of the initial prompts is the same as the few-shot exemplars by Wei et al. (2022b). The Instruction provides a general description of the downstream task, although its specific content may vary depending on the backbone models used. The new prompt candidates are generated by applying edits to both the instruction and examples with equal probability during the searching process, i.e Prob{edit the instruction} = Prob{edit the examples} = 1/2.
For GPT3, the Instruction part of the prompt follows the zero-shot-CoT approach proposed by Kojima et al. (2022). Meanwhile, for GPT3.5, the Instruction part of the initial prompts is manu-
ally crafted by referring to the API reference documentation available on the GPT3.5 in the OpenAI Platform1.
Moreover, Figure 5 illustrates the template for the prompt used of the improved add operation and revisions, leveraging the GPT3.5 as the underlying model.
Experimental Cost While producing few-shotCoT prompts, we do not consider the API call limit; however, we still take into account the estimated cost of the experiments. The costs for GPT3 (text-curie-001) and GPT3.5 (gpt-3.5-turbo-0301) are nearly identical as they share the same pricing structure ($0.002 per 1k tokens). On average, each task typically involves a cost of approximately $25 and concludes after approximately 12 hours of execution when searching for few-shot-CoT prompts with a single thread.

heading: C Algorithm Details
text: To help users better understand and utilize the package, here we present the main idea and pseudocodes of all included algorithms so far.
C.1 Hill Climbing
Hill climbing is a greedy algorithm that aims to find the local optimum of a given objective, e.g. the score function. Each time the algorithm explores a fraction of possible prompt changes and selects one that improves the score.
In our version, we select the one with the most improvements to further speed up the optimization process, as shown in Algorithm 2. Notice that in the paradigm of EvoPrompt, GrIPS can also be viewed as a special version of Hill Climbing, given it greedily improves the prompt via edit operations.
1https://platform.openai.com/docs/guides/gpt/chatcompletions-api
C.2 Simulated Annealing
Different from hill climbing, simulated annealing (Kirkpatrick et al., 1983) allows a small probability of deviation when the prompt change does not yield instant improvement, thus capable of escaping from local optimums and finding better solutions. The probability of deviation is controlled by a hyperparameter called “temperature” T (i) at iteration i, where the lower temperature is, smaller the deviation probability is, less random the algorithm behavior becomes. When the temperature T (i) = 0, simulated annealing becomes deterministic and degenerates to hill climbing.
As shown in Algorithm 3, it only differs from hill climbing in one line, where it utilizes the provided temperature T (i) to decide the deviation probability. We use exponential decay T (i) = 10 exp(−i/5) as our default temperature schedule
Algorithm 2 GrIPS / Hill Climbing 1: base← init ▷ Initialize base candidate 2: sbase ← score(base) ▷ Score using examples in S 3: Ω← {del, swap, par, add} ▷ Set of edit operations 4: ρ← P ▷ Patience for early-stop 5: for i = 1, · · · , n do ▷ n: number of iterations 6: for j = 1, · · · ,m do ▷ m: number of candidates 7: Sample e1, · · · , el ∈ Ω ▷ l edits per candidate 8: C[j]← edit(base, e1 ◦ · · · ◦ el) 9: s[j]← score(C[j]) ▷ Score above candidate
10: k ← argmaxjs[j] 11: best← C[k] ▷ Chosen Candidate 12: sbest ← s[k] ▷ Score of chosen candidate 13: if sbest > sbase then ▷ Candidate better than base 14: base← best ▷ Use this candidate in the next
step 15: sbase ← sbest ▷ Update base score 16: ρ← P ▷ Refresh patience 17: else 18: if ρ > 0 then ▷ Patience not exhausted 19: decrement ρ 20: continue ▷ Continue search with same base 21: else 22: return base ▷ Early-stop criteria met 23: return base ▷ Search terminates after the last iteration
in the package.
C.3 Genetic Algorithm (Mutation Only)
Genetic algorithm (Holland, 1992) was inspired by biological evolution during the nature selection process, where each prompt is treated as a “gene” and the score function serves as the criterion of nature selection.
In a simplified version of genetic algorithm, as illustrated in Algorithm 4, only mutation operations are adopted to change the prompt. A population of prompts W are maintained during the optimization process, indicating all the possible exploring directions of the algorithm. At each iteration, we select a high-quality prompt from all those exploring direc-
tions via tournament selection. Intuitively, the more tournament round k we adopt, the higher quality of the selected prompt, but the less explorative the algorithm behavior becomes. The selected prompt is then mutated to generate a new prompt.
As the algorithm proceeds, the average quality of prompts in the population is expected to improve. In fact, given sufficient number of iterations, the algorithm is capable of finding the global optimum.
C.4 Genetic Algorithm (with Crossover)
The general version of genetic algorithm (Holland, 1992) is realized in Algorithm 5. Besides the mutation operation, a different operation called “crossover” is introduced, where we allow new prompts to be generated from two “parent prompts”. The new prompts will inherit partial segments from both parent prompts and be further changed with mutation operations.
Intuitively speaking, crossover operation represents “global” changes in the search space, while mutation operation represents “local” changes, hence it is to be expected that the genetic algorithm with crossover is less efficient than its simplified counterpart, but more explorative and capable of finding better solutions with sufficient iterations.
C.5 Tabu Search
Greedy search algorithms tend to be stuck in a local optimum or near a local optimum. Tabu Search (Glover, 1986) effectively compensates for this shortcoming by introducing the Tabu list: any candidate that was visited recently is forbidden to visit again. This allows the optimization process to leave the local “tabu” area and explore more.
In Algorithm 6 we adopt a simple Tabu rule of direct comparison: with a high probability, a generated prompt that exactly matches any prompts in the Tabu list will be treated as invalid and discarded, where the Tabu list only retains the Ntabu latest generated prompts.
C.6 Harmony Search
Harmony Search (Geem et al., 2001) mimics the improvision procedure of musicians, where a new prompt is generated via combining segments of all stored prompts in the harmony memory. A pitch finetuning process is then conducted, allowing further refinement of the prompt. The full details are available in Algorithm 7.
Algorithm 3 Simulated Annealing 1: base← init ▷ Initialize base candidate 2: sbase ← score(base) ▷ Score using examples in S 3: Ω← {del, swap, par, add} ▷ Set of edit operations 4: ρ← P ▷ Patience for early-stop 5: for i = 1, · · · , n do ▷ n: number of iterations 6: for j = 1, · · · ,m do ▷ m: number of candidates 7: Sample e1, · · · , el ∈ Ω ▷ l edits per candidate 8: C[j]← edit(base, e1 ◦ · · · ◦ el) 9: s[j]← score(C[j]) ▷ Score above candidate
10: k ← argmaxjs[j] 11: best← C[k] ▷ Chosen Candidate 12: sbest ← s[k] ▷ Score of chosen candidate 13: if sbest > sbase or exp( sbest−sbaseT (i) ) ≥
Random(U(0, 1)) then ▷ Candidate better than base 14: base← best ▷ Use this candidate in the next
step 15: sbase ← sbest ▷ Update base score 16: ρ← P ▷ Refresh patience 17: else 18: if ρ > 0 then ▷ Patience not exhausted 19: decrement ρ 20: continue ▷ Continue search with same base 21: else 22: return base ▷ Early-stop criteria met 23: return base ▷ Search terminates after the last iteration

heading: D Hyperparameter Setting
text: For all the experiments, the number of edits that performs on the candidate prompts (num_compose) and the number of the newly generated candidates (num_candidate) in each iteration follows num_compose ∈ {1, 2} and num_candidate ∈ {5, 10}. The population sizes, and tournament selections k of Plum-GA-M, Plum-GA-C and Plum-GA-M* are all set to 10 and 3 respectively. Meanwhile, the mutation rate pmutation is set to 0.5 for Plum-GA-C. The Ntabu is configured to 5 for Plum-TS. Additionally, the configuration for the harmony search memory (NH ), the number of segments (ks), harmony memory considering rate (HMCR), and pitching adjust rate (PAR) of Plum-HS and Plum-HS* is as follows: 10, 5, 0.4, and 0.5, respectively. Furthermore, the temperatures of GPT-3 are configured to 0 in our experiments.
In the context of general prompt learning, disregarding the API call limit allows us to set the maximum iteration and patience parameters to 50 and 7 respectively, which ensures that Plum-GA-M and Plum-GA-C have a sufficient number of iterations to generate relatively optimal solutions.
When it comes to few-shot-CoT prompt learning, setting a very large value for the maximum iterations is not recommended due to the potential cost implications, which is primarily because the num-
Algorithm 4 Genetic Algorithm (mutation only) 1: base← init ▷ Initialize base candidate 2: sbase ← score(base) ▷ Score using examples in S 3: Ω← {del, swap, par, add} ▷ Set of edit operations 4: ρ← P ▷ Patience for early-stop 5: W← {⟨base, sbase⟩} 6: result← base ▷ Initialize population 7: for i = 1, 2, · · · , n do ▷ n: number of iterations 8: S← ∅ ▷ Examples that are going to battle in the
tournament 9: for j = 1, 2, · · · , k do ▷ k: tournament selection
hyperparameter 10: ⟨parent, sparent⟩ ← Random(W) 11: S← S+ {⟨parent, sparent⟩} 12: base← argmax{parent|⟨parent,sparent⟩∈S}sparent 13: for j = 1, · · · ,m do ▷ m: number of candidates 14: Sample e1, · · · , el ∈ Ω ▷ l edits per candidate 15: C[j]← edit(base, e1 ◦ · · · ◦ el) 16: s[j]← score(C[j]) ▷ Score above candidate 17: k ← argmaxjs[j] 18: best← C[k] ▷ Chosen Candidate 19: sbest ← s[k] ▷ Score of chosen candidate 20: W←W + {⟨best, sbest⟩} 21: if sbest > sresult then ▷ Candidate better than best
result so far 22: result← best 23: sresult ← sbest 24: ρ← P ▷ Refresh patience 25: else 26: if ρ > 0 then ▷ Patience not exhausted 27: decrement ρ 28: continue ▷ Continue search with same base 29: else 30: return result ▷ Early-stop criteria met 31: return result ▷ Search terminates after the last iteration
ber of tokens generated in each iteration tends to be substantial. And for Plum-GA-M* and Plum-HS*, effective results can be obtained with maximum iterations ranging from 5 to 15, coupled with a patience level ranging from 2 to 5. For the experimental results shown in Table 8, we configured the maximum iterations to be 10 and deactivated the patience parameter.
For text-to-image generation tasks, to accommodate a wide range of application scenarios, the initial prompts for image generation are kept short (merely 5-15 words). Given that stable diffusion requires approximately 15 seconds (on a NVIDIA A40 GPU) to produce each image, and the scoring process necessitates the generation of two images for direct comparison, it has been observed that after 6-7 iterations, most prompts reach a point of convergence where the best score is achieved. Consequently, it is advisable to limit the number of iterations to 5-15 and set the patience parameter between 2-5. In the experimental results presented in Table 5, we set the maximum iterations to 10 and deactivated the patience parameter. Further-
Algorithm 5 Genetic Algorithm (with crossover) Hyperparameters: Number of iterations n, population size Np, number of offsprings each generation Noffspring, mutation probability pmutation. 1: base← init ▷ Initialize base candidate 2: sbase ← score(base) ▷ Score using examples in S 3: Ω← {del, swap, par, add} ▷ Set of edit operations 4: ρ← P ▷ Patience for early-stop 5: W0 ← {⟨base, sbase⟩ ×Np} ▷ Np copies of the initial prompt 6: result← base 7: for i = 1, 2, · · · , n do ▷ n: number of iterations 8: Wi ←Wi−1 9: for j = 1, 2, · · · , Noffspring do ▷ ===== Crossover 10: ⟨parent1, sparent1 ⟩ ← BI-
NARY_TOURNAMENT_SELECTION(Wi−1) 11: ⟨parent2, sparent2 ⟩ ← BI-
NARY_TOURNAMENT_SELECTION(Wi−1) 12: offspring← CROSSOVER(parent1, parent2) 13: if offspring ̸∈Wi then 14: soffspring ← score(offspring) 15: Wi ←Wi + {⟨offspring, soffspring⟩} 16: Wi ← top-Np(Wi) ▷ Removes prompts with bad scores,
reserves only best Np prompts 17: ρ, result, flagstop ← UPDATE_RESULT(ρ, result,Wi) 18: if flagstop then return result 19: 20: W′i ←Wi 21: for ∀⟨base, sbase⟩ ∈Wi do ▷ ===== Mutation 22: if pmutation ≥ Random([0, 1]) then 23: Sample e1, · · · , el ∈ Ω ▷ l edits per candidate 24: mutated← edit(base, e1 ◦ · · · ◦ el) 25: smutated ← score(mutated) ▷ Score above candidate 26: W′i ←W′i + {⟨mutated, smutated⟩} 27: Wi ← top-Np(W′i) ▷ Removes prompts with bad scores,
reserves only best Np prompts 28: ρ, result, flagstop ← UPDATE_RESULT(ρ, result,Wi) 29: if flagstop then return result 30: return result ▷ Search terminates after last iteration 31: 32: function BINARY_TOURNAMENT_SELECTION(W) 33: S← ∅ 34: for i = 1, 2 do 35: ⟨parent, sparent⟩ ← Random(W) 36: S← S + {⟨parent, sparent⟩} 37: return argmax{parent|⟨parent,sparent⟩∈S}sparent 38: 39: function CROSSOVER(parent1, parent2) 40: [t1,1, t1,2, . . . , t1,L1 ]← parent1 41: [t2,1, t2,2, . . . , t2,L2 ]← parent2 42: split← Random({0, 1, 2, . . . ,max(L1, L2)}) 43: offspring← [t1,1, . . . , t1,split, t2,split+1, . . . , t2,L2 ] 44: return offspring 45: 46: function UPDATE_RESULT(ρ, result,W) 47: flagstop ← false 48: best← argmax{prompt|⟨prompt,sprompt⟩∈W}sprompt 49: if sbest > sresult then ▷ Candidate better than best result so far 50: result← best 51: sresult ← sbest 52: ρ← P ▷ Refresh patience 53: else 54: if ρ > 0 then ▷ Patience not exhausted 55: decrement ρ 56: else 57: flagstop ← true ▷ Early-stop criteria met 58: return ρ, result, flagstop
more, considering the prompt’s brevity, the initial ks value was set to 2 in the experiments to guarantee that each segment targeted for modification includes at least one tag. Additionally, the Harmony Memory Considering Rate (HMCR), Pitch Adjustment Rate (PAR), and Harmony Search Memory (nH ) were set to 0.4, 0.5, and 10 respectively.
Algorithm 6 Tabu Search 1: base← init ▷ Initialize base candidate 2: sbase ← score(base) ▷ Score using examples in S 3: Ω← {del, swap, par, add} ▷ Set of edit operations 4: ρ← P ▷ Patience for early-stop 5: result← base ▷ Initialize population 6: w0 ← base 7: T ← {w0} ▷ Initialize T 8: for i = 1, 2, · · · , K do ▷ n: number of iterations 9: for j = 1, 2, · · · , n do ▷ k: number of the new candidate
10: Sample e1, · · · , el ∈ Ω ▷ l edits per candidate 11: C[j]← edit(base, e1 ◦ · · · ◦ el) 12: s[j]← score(C[j]) ▷ Score above candidate 13: W′j ← TABU(T , s[j], TEMP ) = 0 14: k ← argmaxl∈W′
j s[j]
15: best← C[k] ▷ Chosen Candidate 16: sbest ← s[k] ▷ Score of chosen candidate 17: T ← T + s[k] ▷ Update T 18: if sbest > sresult then ▷ Candidate better than best result so far 19: result← best 20: sresult ← sbest 21: ρ← P ▷ Refresh patience 22: else 23: if ρ > 0 then ▷ Patience not exhausted 24: decrement ρ 25: continue ▷ Continue search with same base 26: else 27: return result ▷ Early-stop criteria met 28: if |T | > Ntabu then 29: T ← T − wk−Ntabu ▷ Keep Short-term tabus 30: return result ▷ Search terminates after last iteration 31: function TABU(T ,w, TEMP ) 32: if w ∈ T then 33: if TEMP ≥ Random(U(0, 1)) then 34: return 0 35: else 36: return 1 37: else 38: return 0
Sensitiveness of Plum-HS Hyperparameters To gain deeper insights into the impact of key hyperparameters in Harmony Search and guide their effective tuning, we conducted an ablation study on HMCR, ks and NH for Plum-HS with GPT2large as the backbone model. As shown in Figure 6, both the Harmony Memory Consideration
Algorithm 7 Harmony Search 1: base← init ▷ Initialize base candidate 2: sbase ← score(base) ▷ Score using examples in S 3: Ω← {del, swap, par, add} 4: Ωsmall ← {par} ▷ Set of edit operations 5: ρ← P ▷ Patience for early-stop 6: W0 ← {⟨base, sbase⟩} 7: result← base ▷ Initialize population 8: for i = 1, 2, · · · , n do ▷ n: number of iterations 9: W′ ← ∅ ▷ Candidates generated at this iteration
10: for c = 1, 2, · · · , k do 11: w′ ← GENERATE_CANDIDATE(Wi−1,Ωsmall,Ω) 12: sw′ ← score(w′) ▷ Score the new candidate 13: W′ ←W′ + {⟨w′, sw′ ⟩} 14: ρ, result, flagstop ← UPDATE_RESULT(ρ, result,W′) 15: if flagstop then return result 16: Wi ← top-NH(Wi−1 ∪W′) ▷ Removes prompts with bad
scores, reserves only best NH prompts 17: return result ▷ Search terminates after last iteration 18: function GENERATE_CANDIDATE(W,Ωsmall,Ω) 19: w′ ← [ ] ▷ Empty string/list 20: for j = 1, 2, · · · , ks do ▷ ks: number of segments (pitches) 21: ⟨w, ∗⟩ ← Random(W) ▷ Randomly sample an existing
prompt for j-th segment 22: L← |w| ▷ Length of the reference prompt 23: [w0, w1, . . . , wL−1]← w 24: start← ⌈ j−1ks · L⌉ 25: end← ⌈ jks · L⌉ − 1 26: wsegment ← [wstart, wstart+1, . . . , wend] 27: if HMCR ≥ Random(U(0, 1)) then 28: if PAR ≥ Random(U(0, 1)) then ▷ A little different
random segment (pitch) 29: Sample e1, · · · , el ∈ Ωsmall ▷ Only replace phrases
with their synonyms 30: wsegment ← edit(wsegment, e1 ◦ · · · ◦ el) 31: else ▷ A largely different random segment (pitch) 32: Sample e1, · · · , el ∈ Ω ▷ A big change in the original
segment 33: wsegment ← edit(wsegment, e1 ◦ · · · ◦ el) 34: w′ = w′ + wsegment ▷ Here “+” means concatenation. 35: return w′ 36: function UPDATE_RESULT(ρ, result,W) 37: flagstop ← false 38: best← argmax{prompt|⟨prompt,sprompt⟩∈W}sprompt 39: if sbest > sresult then ▷ Candidate better than best result so far 40: result← best 41: sresult ← sbest 42: ρ← P ▷ Refresh patience 43: else 44: if ρ > 0 then ▷ Patience not exhausted 45: decrement ρ 46: else 47: flagstop ← true ▷ Early-stop criteria met 48: return ρ, result, flagstop
Rate HMCR and the Harmony Memory size NH achieve optimal performance with a moderate value. This trend aligns with the core principles of Harmony Search (Algorithm 7). Low HMCR encourages the algorithm perform random modifications, which results in introducing too much difference. Conversely, high HMCR over-relies on past prompts in the harmony memory, which lacks diversity. As for Harmony Memory size NH , it is similar to the population size in genetic algorithms. Small NH restricts exploration such that the algorithm behaves similarly to greedy search and lacks diversity. While, large NH deteriorates the quality of memorized prompts, and slows down the search efficiency. Notably, the performance
exhibits stable and high performance when PAR is set to 0.4 or above.
For ks, we find that a larger value introduces superior performance. This is because larger ks leads to finer granularity of segments, which allows a finer level of tuning on prompts. In this paper, the upper bound of ks is considered as 5 since the shortest sample in our datasets only contains 5 segments of words.

heading: E Licenses
text: For general prompt learning tasks, the dataset Natural-Instructions v2.6 (Mishra et al., 2022) is released under Apache-2.0 license. While for Chain-of-Thought prompt learning, the datasets ASDiv (Miao et al., 2020), SVAMP (Patel et al., 2021), CSQA (Talmor et al., 2019) are released under CC-BY-NC 4.0, MIT, and CC BY-SA 4.0 licenses respectively.

heading: Future Work
text: This characteristic makes it valuable for future theoretical analyses of discrete prompt learning. • Plum-GA-M and Plum-GA-C, the genetic algorithm, can be considered one of the most widely adopted algorithms in evolutionary search, whose results can be useful for future comparisons on the same tasks for other researchers. Additionally, our experiments also demonstrate that these algorithms achieve superior performance compared to baseline approaches. • For Plum-TS, although its performance is not the best, it achieves on-par performance as GrIPS in 4 times fewer API calls (as shown in Table 3). This advantage makes Plum-TS an extremely favorable choice for applications with limited budgets or API call quotas