Title: Teach Better or Show Smarter? On Instructions and Exemplars in Automatic Prompt Optimization
Venue: NeurIPS 2024
Accepted: yes
Keywords: instruction optimization

prompting

in-context learning
Abstract: Large language models have demonstrated remarkable capabilities but their performance is heavily reliant on effective prompt engineering. Automatic prompt optimization (APO) methods are designed to automate this and can be broadly categorized into those targeting instructions (instruction optimization, IO) vs. those targeting exemplars (exemplar optimization, EO). Despite their shared objective, these have evolved rather independently, with IO receiving more research attention recently. This paper seeks to bridge this gap by comprehensively comparing the performance of representative IO and EO techniques both isolation and combination on a diverse set of challenging tasks. Our findings reveal that intelligently reusing model-generated input-output pairs obtained from evaluating prompts on the validation set as exemplars, consistently improves performance on top of IO methods but is currently under-investigated. We also find that despite the recent focus on IO, how we select exemplars can outweigh how we optimize instructions, with EO strategies as simple as random search outperforming state-of-the-art IO methods with seed instructions without any optimization. Moreover, we observe a synergy between EO and IO, with optimal combinations surpassing the individual contributions. We conclude that studying exemplar optimization both as a standalone method and its optimal combination with instruction optimization remain a crucial aspect of APO and deserve greater consideration in future research, even in the era of highly capable instruction-following models.

Teach Better or Show Smarter? On Instructions and Exemplars in Automatic Prompt Optimization

Abstract

Large language models have demonstrated remarkable capabilities but their performance is heavily reliant on effective prompt engineering. Automatic prompt optimization (APO) methods are designed to automate this and can be broadly categorized into those targeting instructions (instruction optimization, IO) vs. those targeting exemplars (exemplar optimization, EO). Despite their shared objective, these have evolved rather independently, with IO receiving more research attention recently. This paper seeks to bridge this gap by comprehensively comparing the performance of representative IO and EO techniques both isolation and combination on a diverse set of challenging tasks. Our findings reveal that intelligently reusing model-generated input-output pairs obtained from evaluating prompts on the validation set as exemplars, consistently improves performance on top of IO methods but is currently under-investigated. We also find that despite the recent focus on IO, how we select exemplars can outweigh how we optimize instructions, with EO strategies as simple as random search outperforming state-of-the-art IO methods with seed instructions without any optimization. Moreover, we observe a synergy between EO and IO, with optimal combinations surpassing the individual contributions. We conclude that studying exemplar optimization both as a standalone method and its optimal combination with instruction optimization remain a crucial aspect of APO and deserve greater consideration in future research, even in the era of highly capable instruction-following models.

1 Introduction

Significant advancements in large language models (LLMs) have revolutionized various natural language processing tasks [9, 4, 1, 15]. One notable aspect of LLMs, however, is their sensitivity to the input "prompts," which has given rise to the burgeoning field of prompt engineering [26, 38]. On black-box LLMs where we can neither modify or access internal parameters, prompt engineering involves crafting input prompts that effectively guide LLMs to generate desired outputs. Starting from manual processes requiring human expertise, the complexity and volume of prompts have necessitated the development of automatic prompt optimization (APO) methods aiming to streamline and automate prompt generation, thereby alleviating the burden of manual intervention. Broadly, since prompts consist of instructions and exemplars, we may roughly categorize APO into instruction optimization (IO) and exemplar optimization (EO) approaches. IO focuses

Initial instruction

+  No exemplars

Optimized instruction

+  No exemplars

Optimized instruction

+  Random exemplars

Initial instruction

+

Optimized exemplars

Optimized instruction

+

Optimized exemplars

60.3

68.1

70.8

72.9

76.1

Most

existing IO

works

Optimizing

exemplars

only

outperforms

SoTA IO

on refining the textual instructions provided to LLMs that contain task-specific information (i.e., to teach ), whereas EO emphasizes the selection of relevant examples to guide model behavior (i.e., to show ). Partially driven by the improved instruction-following ability of LLMs, the research attention has increasingly shifted towards IO, especially using LLMs themselves as optimizers [66, 35, 48].

While EO and IO approaches address the similar overarching problem, they have evolved somewhat independently, with a few exceptions [14, 47]. Indeed, as we elaborate in §2, EO approaches are often based on simple, handcrafted templates without explicit instruction optimization [21, 45], while IO methods seldom optimize exemplars and often rely on random validation set samples [35], require additional fixed exemplars on top of the validation set [66, 16], or consider the 'zero-shot' setup with no exemplars at all [48]. Whereas the lack of IO in EO methods is somewhat understandable as many EO approaches predate instruction finetuning [49] and, subsequently, instruction-following models that are sensitive to instructions, the inverse is much less so: concretely, almost all existing IO approaches already require a labeled dataset as the validation set, and are therefore, by definition, not 'zero-shot'. With the inputs, labels, and, if applicable, model-generated intermediate outputs (e.g., reasoning steps) on the subset of the validation set that the model has answered correctly, we already have a set of exemplars as a free side-product whenever we perform IO, independent from and on top to any additionally provided, human-annotated exemplars. A common argument for not focusing on EO, such as mentioned in Pryzant et al. [35], is the goal to focus on one objective at a time 1 . However, given the common practical goal of and the interplay between EO and IO [30], we argue they should not be treated separately - it is instead critical to understand their relative importance and combined impact, and, where necessary, optimize them jointly for the best performance-cost balance.

This is, to our knowledge, where there is a gap in the literature that we aim to bridge. To do so, on a diverse suite of challenging BIG-Bench and MMLU tasks, we compare the performance gain brought by various representative, state-of-the-art (SoTA) IO and EO methods on the fairground with PaLM 2, Gemini (1.0/1.5) and GPT models to foster better scientific understanding of different APO techniques. While IO comfortably improves the baseline prompts before any instruction or exemplar optimization, this, at best, portrays an incomplete picture. Under the same setup, with simple yet effective EO methods on the model-generated exemplars on the validation set, we show:

Intelligently incorporating exemplars generated by the target model itself on the validation set significantly and consistently improves performance on top of recently proposed IO methods;

The performance gains realized by choosing appropriate exemplars via methods as simple as random search can eclipse the improvements brought by SoTA instruction optimization. As a concrete example, as shown in Fig. 1, with a simple optimization routine on exemplars, seed instructions before any optimization outperform optimized instructions obtained with complex IO but with random exemplars most commonly used.

There exists a synergy between EO and IO, and optimally mixing-and-matching IO and EO is greater than the sum of its parts under a comparable computational budget.

SoTA IO might be itself implicitly generating exemplars, and these exemplars, while somewhat unintentional, contribute more to the performance than the rest of the instruction that IO methods are meant to optimize.

While arguably receiving less research attention recently, exemplar optimization remains a crucial design consideration in APO. Even in an era of highly capable instruction-following LLMs, the significance of exemplar optimization should not be relegated to an afterthought, and better exemplar optimization both as a standalone tool and as a combinable component with IO is crucial for APO.

2 Preliminaries

Prompts are natural language inputs to LLMs. Denoting an input task query as x , a few-shot prompt P ( x ) may be represented as P ( x ) = [ I, e 1 , ..., e k , x ] where I denotes an instruction and { e 1 , ..., e k } denote k exemplars (or interchangeably, demonstrations ), each of which is a concatenation of other queries and their outputs (including both the final answer and any possible intermediate outputs) which resemble the current query x or may otherwise guide the LLM to better handle the current task, and we show a common prompt template organizing these components in Fig. 2 - note that not all components are required: e.g., zero-shot prompts feature no exemplars.

Automatic prompt optimization (APO) aims to automatically design P ( x ) via optimization. We broadly consider black-box API-only LLMs 2 . The proposed framework assumes a validation dataset D val := { ( x i , y i ) } n val i =1 , where x i and y i represent validation inputs and targets, a performance metric (e.g., accuracy) g ( · , · ) , and aims to find the optimal prompt P ∗ ( x ) to be used at test time, which is empirically the maximizer on D val :

where f LLM ( · ) denotes a textual LLM output given input and P denotes the search space , whose definition allows a broad categorization of APO methods into instruction optimization methods targeting instructions in Fig. 2, exemplar optimization methods targeting exemplars in Fig. 2 and approaches that tackle both.

Exemplar optimization (EO). Efforts to optimize exemplars started soon after the discovery of in-context learning (ICL) [6] via retrieval-based approaches to identify the closest labeled examples [28, 61, 51, 64], influences and sensitivity [31, 8], and learning-based approaches [58, 55]. Toolkits like DSPy [21] adopt EO as the main APO paradigm. Works [45, 46, 60, 22, 41, 59] have also extended EO to model-generated exemplars in LLMs and multimodal models. Lastly, by framing EO from an active learning angle, Margatina et al. [29] provide a comprehensive understanding and comparative analyses. These works, however, principally analyze different EO strategies only, nor do they analyze from the angle of APO. Many of these works also primarily focus on and draw findings from earlier and simpler tasks that are arguably less challenging to SoTA LLMs.

Solve the following math problems by following the steps:

When multiplying or dividing two negative numbers, the result will be positive.

When multiplying or dividing a

positive and a negative number, the result will be negative.

When adding or subtracting a negative number, it is the same as adding or subtracting its positive counterpart.

((6 - 0 * 5

+  -3) * (6 - -7

+  -2 - -7))  =

(6 - 0

+  -3) * (6 - -7

3 * (13

+  -2 - -7)

+  -2

+  7)

3 * 18

< answer

> 54

< /answer >

==

Figure 2: An example prompt: instruction I describes the task; exemplars ( e 1 , ..., e k , k = 1 in the figure) provide demonstrations and enable ICL; both are prepended to the query x before receiving the LLM responses.

Instruction optimization (IO). On black-box LLMs, the origin of IO may be traced to discrete prompt search [12, 40, 53, 62, 34] which prepend optimized tokens, which can be viewed as a form of 'instructions', to inputs. However, these approaches do not necessarily yield interpretable prompts and most of them require output signals (such as logits) beyond strictly black-box access. Thus, recent advances have shifted towards utilizing an LLM itself to generate natural language instructions for iterative optimization on D val in Eq. 1. The seminal works is APE [66], which employs the LLM to iteratively cull top-performing instructions on D val and paraphrase them until convergence. Similar evolutionary frameworks are widely used in follow-up works [16, 19, 63] and alternative formulations like Bayesian optimization (BO) [7] and neural bandits [25] were also used. Another line of works [35, 48, 56, 42] employ reflection , directing an LLM to articulate reasons for errors to iteratively improve instructions. Other approaches like OPRO and its variants [54, 27], treat the LLM as a black-box optimizer, tasking it with generating new instructions based on the trajectory of previously evaluated instructions and their performances without explicit meta-instructions.

Combining EO and IO. As discussed in §1, there is a relative dearth of work combining EO and IO despite their shared objective. Specifically, even when the labeled dataset D val is a prerequisite of virtually all IO methods, it is primarily used to estimate the expectation in Eq. 1 only rather than to construct exemplars in a principled way: For instance, ProTeGi [35] randomly samples exemplars from D val , while OPRO [54] uses them only for instruction induction [18]. Other works [16, 66, 48] either use no exemplars or fix exemplars and only optimize the instructions - for challenging reasoning tasks, these methods require human-annotated chain-of-thought (CoT) exemplars [50] in addition to D val , which arguably runs counter to the goal of automatically designing prompts without human intervention in APO. A few exceptions exist: PromptBreeder [14] employs 'context shuffling' to co-evolve exemplars and instructions, while Mixture-of-Prompts (MoP) [47] aligns exemplars with multiple prompting 'experts" for joint optimization. However, these works still focus their optimization effort on instructions: PromptBreeder emphasizes complex mutation operators for IO while providing only basic EO frameworks, whereas MoP chiefly focuses on IO with the bulk of its contribution being assigning optimized instructions to different exemplar groups, rather

than optimizing the exemplars themselves. Other works [13, 57] also include both exemplars and instructions in the search space, but they require information beyond strictly black-box outputs to some extent. Lastly, several works have analyzed the interplay between ICL and instructions [30] or prompt templates [39], but they mainly characterize the performance variation as an issue deserving attention. We, however, consider the APO setup specifically, and argue that such an interdependence presents an opportunity through holistically considering instructions and exemplars. Concurrent to our work, Agarwal et al. [2] and Opsahl-Ong et al. [32] also study the joint optimization of instructions and exemplars, and in many cases reached conclusions corroborating our findings, demonstrating the community's growing awareness on the importance of the subject of focal interest to this paper.

3 Understanding Instruction Optimization and Exemplar Optimization

While studying IO and EO independently has academic value, the practical goal ultimately for both is to optimize the performance of LLMs. Hence, IO and EO, as two dominant genres of APO methods, present practitioners with the challenge of selecting or combining them to maximize cost-performance benefits. We aim to meet this by evaluating EO and IO in the context of APO by answering the following: 1) What is the relative importance and performance impact of EO and IO, both in isolation and when combined together? 2 ) How do we make the optimal use of the limited data and computational budget under the current APO framework?

3.1 Experimental Setup

We perform thorough experiments employing various EO and IO methods individually and in combination. We use the PaLM 2 text-bison-002 [4] and Gemini 1.0 Pro/1.5 Flash [15, 36] as the target models, but we will also validate key findings on GPT-3.5. Modern IO methods often employ another, usually more potent optimizer model for to generate and/or critique instructions; we use PaLM 2 text-unicorn-001 (for text-bison target model), Gemini 1.0 Ultra (for Gemini 1.0 Pro target model) or Gemini 1.5 Pro (for Gemini 1.5 Flash target model). We evaluate on tasks selected from BIG-Bench Hard (BBH) [43], a collection of diverse tasks considered to be challenging to LLMs - the suite itself and datasets of similar task types are frequently used in many recent APO works [48, 54, 19, 14, inter alia ]: the tasks include numerical reasoning, commonsense problemsolving, logical deduction, linguistic manipulation, machine translation, and tabular reasoning, among others. For all tasks, we use 20% of data as validation set and the remaining 80% for testing, the latter of which is held-out and unavailable to the LLM at search time (see App. A for implementation details). We also test some of our key findings on the MMLU benchmark [17], a set of 57 tasks frequently used to gauge the general problem-solving abilities of LLMs - we use the official val and test splits for validation and testing, respectively. We consider the following IO strategies:

No IO : we use the seed instruction I 0 ' Let's think step by step .' [23] without any optimization.

APE [66] is the seminal work for LLM-as-an-instruction-optimizer and uses an evolutionary algorithm design: at each iteration, we evaluate a population of instructions on the validation set and the optimizer is asked to generate a new population by paraphrasing the top-performing instructions. This process iterates until convergence.

ProTeGi [35] collects samples that the target LLM answers incorrectly on D val under the current instruction and directs the optimizer LLM to reflect and critique it. The optimizer model is then asked to update the instruction by summarizing and abstracting the feedback. Additionally, at each iteration, ProTeGi also paraphrases instructions similar to APE (referred to as 'Monte Carlo sampling') and uses beam search to identify the most promising instructions.

PromptAgent [48] is similar to ProTeGi but it features a more advanced planning agent using Monte Carlo tree search [10].

OPRO [54] implicitly optimizes instructions. Starting from the seed instruction, at each iteration, OPRO provides the optimizer model a concatenation of previously evaluated instructions and their validation scores. Instead of explicitly requesting paraphrasing or reflection, OPRO treats the optimizer LLM as a black-box optimizer and simply asks the optimizer model to 'come up with a better instruction' given these information.

The above methods are selected as each of them represents the state of the art of a genre of approaches as outlined in §2 and collectively represents IO techniques as a whole. We initialize each method at the seed instruction and ensure they consume the same amount of compute measured by the

⊗}(〉⋃

⊕√√/

⊗}(〉⋃

⊗}(〉⋃

⊕√√/

〉⌋





















⋂∐√⌋(]{̂˜glyph[arrowtp]

〉⌋

〉⌋

〉⌋

〉⌋

〉⌋

〉⌋

〉⌋

〉⌋

〉⌋

〉⌋

〉⌋

〉⌋

〉⌋

〉⌋

〉⌋

〉⌋

〉⌋

/

/

⊗}(∮⊗





















⋂∐√⌋(]{̂˜glyph[arrowtp]

〉⌋

〉⌋

∖⌋

〉⌋

〉⌋

〉⌋

〉⌋

〉⌋

〉⌋

〉⌋

〉⌋

〉⌋

〉⌋

〉⌋

〉⌋

〉⌋

∖⌋

〉⌋

〉⌋

/

/

∫/⋃/

∑〉





















⋂∐√⌋(]{̂˜glyph[arrowtp]

〉⌋

〉⌋

∖⌋

〉⌋

∖⌋

⌋

〉⌋

〉⌋

〉⌋

〉⌋

〉⌋

〉⌋

〉⌋

〉⌋

〉⌋

/

/

∑√}⋂/

number of prompt evaluations on D val m (we cap m = 32 except for 'No IO' which requires no iteration). We also compare against PromptBreeder [14] in a later section, as it features a much more expansive search space and requires significantly more than 32 iterations before convergence. After obtaining the optimized instruction I ∗ (or I 0 if no IO is performed), we perform EO. At this point, we emphasize that our setup should not be confused with the 'few-shot' setup considered by some prior works [66, 16] which require additional human-annotated exemplars with reasoning traces to elicit CoT behavior. We perform EO only from the exemplars self-generated by the target model (also referred to as 'bootstrapped few-shot' in DSPy [21] and 'reinforced ICL' in concurrent works like Agarwal et al. [3]) and do not assume exemplars are given at the start of APO (i.e., we do not assume the presence of initial { e 1 , ..., e k } in P ( x ) ). We consider the following EO strategies:

No EO : no exemplars are used; this is typically referred to as 'zero-shot' in the APO literature.

Random : we randomly sample k input-output pairs from D c ( I ∗ ) ⊆ D val , the subset to the validation set that the target LLM predicted correctly under I ∗ and the output in this case includes any intermediate output the LLM generates before the final answer.

Nearest : We use the same D c ( I ∗ ) as above, but instead of sampling randomly, we retrieve topk input-output pairs whose inputs are most similar to the current test input based on text embedding cosine similarity. We use the Gecko embedding [24].

Diversity : We use D c ( I ∗ ) but select the k input-output pairs closest to the centroids via k -means clustering, similar to the approach in Zhang et al. [60] to promote diversity in exemplars.

All exemplars ( Gemini 1.5 target models only ): With the advent of long-context models like Gemini 1.5, we may also fit the entire set of D c into the context and perform no selection at all.

The above heuristic -based EO strategies do not use D val , whereas optimization -based EO can utilize it similarly to IO. Instead of generating instructions , we select the exemplar combinations with the highest validation accuracy for testing [33, 20, 21]. Unlike IO, which creates new instructions via an optimizer model, EO selects from pre-generated outputs and does not require an optimizer model. Formally, we focus on optimizing exemplars conditional on I ∗ from IO (or I 0 if no IO is involved) 3 :

We include the following optimization-based EO methods that differ in search strategy:

Random search : Following the EO procedure in DSPy [21], we randomly sample m combinations of k exemplars: { E 1 , ..., E m } where each E glyph[lscript] = { e glyph[lscript] j } k j =1 ∀ glyph[lscript] ∈ { 1 , ..., m } . We evaluate each combination on the validation set and use the best for testing.

Mutation : We also implement a mutation-based baseline, initiating with a population of Q combinations for T = m/Q generations, where Q = 8 . Each generation starts with a randomly

initialized first generation, similar to random search. For subsequent generations, we populate with Q mutations of the best-performing combination from the previous generation, E ∗ ≤ t . Each mutation involves swapping one exemplar for another input-output pair from D c .

For all EO methods except for ' All exemplars ', we use m = 32 , k = 3 for all main experiments, but we also test with k = { 1 , 3 , 5 , 10 , 20 } in App. B.4. Given the IO and EO strategies, we experiment on each of the IO-EO combinations.

3.2 Results and Analyses

On BBH, we aggregate the results in Table 1 for PaLM 2 models. We also experiment on Gemini models with no IO and with ProTeGi (best overall IO technique from the PaLM 2 results) in Table 2 (Gemini 1.0) and Table 4 (Gemini 1.5). We also validate key findings in representative datasets on GPT-3.5 (Table 13, App B). On MMLU, we present results in Table 3 (PaLM 2) and App. B.1 (Gemini). Below, we highlight and discuss the key insights.

Insight 1 : We should almost always perform EO whenever we perform APO.

One of the immediate findings from the tables is that comparing the first column against others, any EO consistently improves test performance, with any or no instruction optimization. In Fig. 3, we further show that the EO not only benefits at an aggregated level but also leads to significant and almost unanimous improvements across diverse tasks.

While exemplars improving performance may not seem surprising, it is worth noting that as mentioned, in this case they are side-products of evaluating instructions on D val with no additional data costs. Thus, we argue that for practical purposes under the framework of Eq. 1, barring unusual constraints

6.49

6.49

5.82

9.56

6.70

0.07

11.06

-0.05

1.43

9.07

-1.73

-1.46

9.59

13.03

13.16

10.81

PaLM 2 / BBH

Gemini 1.0 Pro / BBH

Gemini 1.5 Flash / BBH

PaLM 2 / MMLU

Gemini 1.0 Pro / MMLU

Val/Test Acc.

80

77.7

79.6

80

84.8

90

86.9

89.1

80

82.8

82.3

80

85

81.8

82.2

72.9

77.0

75.8

85

83.2

60

70

61.7

60.3

68.1

70

63.263.1

65.9

80

75

75.075.1

80.4

70

64.0

65.8

69.7

72.8

70

75

67.6

69.1

68.6

71.4

65

Base

Base

Base

Base

Base

like extreme restrictions in context length, which This might restrict applicability of IO methods too as many SoTA IO methods also generate long prompts, and/or extreme long-context tasks where it is not possible to fit exemplars in the context window, there is little incentive to consider the 'zero-shot' setup without exemplars and little incentive not to perform EO , given that current APO setup requires D val anyway, regardless of whether we use them as exemplars. Thus, it is by definition, not 'zero-shot' and is not directly comparable to true zero-shot methods requiring no labeled data. Furthermore, there is also the risk that 'zero-shot' results neither reflect nor accurately predict the full potential of the LLM, as what performs well under zero-shot does not necessarily performs well when a better EO strategy (e.g., PromptAgent in Table 1 and ProTeGi in Table 3) is used. Lastly, since obtaining labeled data can be costly, intelligently reusing them as exemplars also represents a more judicious use of scarce resources compared to only using them to evaluate a metric for optimization.

Insight 2 : How we select exemplars may outweigh how we optimize instructions, and selecting exemplars via iterative optimization consistently outperforms alternative strategies.

Exemplar optimization outweighs instruction optimization. Despite the recent focus the community places on IO, we find that how we select exemplars outweighs how we optimize instructions in the model-task combinations we investigate. With reference to Tables 1 - 4 (and task-specific breakdown in Fig. 5), we find that if we optimize instructions or exemplars (i.e., the blue cells) under a roughly compute-matched budget, prompts without instruction optimization but with optimized exemplars (e.g., the 'No IO + Mutation' combination) outperform prompts with SoTA instruction optimization but without optimized exemplars (e.g., the 'ProTeGi + Random' combinations) in an overwhelming majority of cases. In fact, on a separate set of experiments performed on the PaLM 2 models, we find this to be true even after halving the evaluation budget of EO (see App. B.6), and optimization on exemplars as naïve as random search can outperform IO methods that are significantly more complicated and expensive. Further substantiating this argument are that:

1) In isolation , EO boosts performance more effectively than IO: for example, with reference to Table 2, compared to the seed prompt, using the best EO strategy (Mutation, first row ) alone increases the average performance by >11%, compared to approximately 8% using the best IO strategy (ProTeGi, first column );

2) When combined , benefits of EO and IO stack up but are largely attributable to EO: under 'Mutation' ( last column ), the best EO strategy, the performance gap between the best and worst IO strategies shrinks to less than 4%, suggesting that instructions might be less critical if the LLM is provided with good exemplars after all.

We observe similar conclusions for different models and task combinations. In fact, on MMLU (Table 3) featuring much smaller validation splits, we observe that judicious exemplar optimization completely eliminates the performance gap caused by IO under zero-shot, with No IO even surpassing SoTA IO. Interestingly, as we show in Fig. 4 where we further consider the difference between validation accuracy, which is the empirical objective function in Eq. 1, and the test accuracy, which is the reported metric that represents the generalization power of the optimized prompt, optimized exemplars generalize better than optimized instructions under all model-task combinations considered. On MMLUtasks (two rightmost plots in Fig. 4), IO even improves validation performance comparable to or better than EO, but the validation improvement does not generalize to the test set. These imply that the superior test performance of EO cannot be solely attributed to a more effective search space P or optimization strategy in Eq. 1, and the performance gap might not be completely closed by advancing optimization only.

Optimization-based EO outperforms heuristics. Between the different EO strategies, we find that optimization-based EO vastly outperform the alternatives: e.g. in all tables, ProTeGi with optimized























IO:







ES:











































































(a) web_of_lies (55 %

→

97 % )

Q: Maybelle tells the truth. Antwan says

Maybelle tells the truth. Audrie says Antwan

tells the truth. Ryan says Audrie lies. Delna

says Ryan lies. Does Delna tell the truth?

A: Let's think step by step.

Maybelle tells the truth.

Antwan says Maybelle tells the truth. So

Antwan tells the truth.

Audrie says Antwan tells the truth. So Audrie

tells the truth.

Ryan says Audrie lies. So Ryan lies.

Delna says Ryan lies. So Delna tells the

truth.

< answer

> True

< /answer >

Q: Which statement is sarcastic?

Options:

(A) If it's on Britannica it must be True..

(B) If it's on reddit it must be True..

A: We should move forward in stages.

< answer

> (B)

< /answer >

Q: Alice, Bob, Claire, Dave, Eve, Fred, and Gertrude are dancers at a

square dance. At the start of a song, they each have a partner: Alice is

dancing with Ophelia, Bob is dancing with Melissa, ...

(text omied)

, and

Gertrude is dancing with Karl.

Throughout the song, the dancers oen trade partners. First, Dave and

Claire switch partners. Then, Alice and Eve switch partners. ...

(text

omied)

At the end of the dance, Fred is dancing with

Options:

(A) Ophelia

...

(text omied)

A: Let's follow a step-by-step process. Let's follow the steps one by

one:

1. Dave and Claire switch partners.

Dave was dancing with Sam, and Claire was dancing with Jamie. So

now, Dave is dancing with Jamie, and Claire is dancing with Sam.

2. Alice and Eve switch partners.

Alice was dancing with Ophelia, and Eve was dancing with Patrick. So

now, Alice is dancing with Patrick, and Eve is dancing with Ophelia.

...

At the end of the dance, Fred is dancing with Melissa.

< answer

> (B)

< /answer >

(b) tracking_shued_objects (7) (38 %

→

86 % )

(c) snarks (73 %

→

87 % )

exemplars outperforms random exemplars, which is the default design in Pryzant et al. [35], by more than 6% in both Table 1 and 2 and more than 2% in Table 4. Interestingly, as shown by the 'All' column in Table 4 for Gemini 1.5 and App. B.4 for Gemini 1.0, naïvely scaling the number of exemplars may not be the most effective - the fact using the entire D c underperforms 3 optimized exemplars, which are a subset of D c , highlights the relevance of EO even for modern LLMs capable of handling long context windows. On the other hand, heuristic-based selection like Diversity and Nearest do not consistently outperform simple random baseline, echoing previous findings [33].

Imitation of task-dependent winning patterns outweighs elaborate descriptions. We further present representative prompts in Fig 6 and how LLM responds to them in App. B.9: Generally, we find that even detailed and task-specific instructions do not regulate the LLM's behavior as effectively as exemplars, which enable imitation . For example, for challenging tasks like tracking_shuffled_objects and web_of_lies , optimization-based EO discover 'winning' templates that, when followed, improve performance massively. Even when SoTA IO methods may often state the answer steps equivalently in words, we find LLMs to simply respond better to exemplars from which they can copy behaviors. On the other hand, for tasks where CoT-style answering are known to be unhelpful (e.g., snarks ) [43], the optimized exemplars are invariably those giving direct answers; when prepended to the test queries, LLMs tend to prioritize exemplar imitation over instruction following to answer directly despite triggers like 'We should move forward in stages'. These highly task-dependent 'winning' patterns that vary from elaborate step-to-step reasoning to direct answering may also explain why heuristic-based EO fares worse to data-driven approaches, since there might not be a single heuristic universally useful for all tasks.

Concluding remarks. We argue that the findings are highly significant for the future of APO. First, they point to a need of re-balancing: without disparaging the value of IO, we argue that EO is at least



























































PaLM 2

(

text-bison-002

)

Eval. budget

m

32

64

IO Budget

m

IO

32

24

16

8

0

32

EO Budget

m

EO

0

8

16

24

32

32

Avg. test acc. (%)

↑

70.81

†

73.26

74.49

76.14

72.92

76.25

Avg. test rank

↓

4.50

3.63

3.44

2.88

3.60

2.94

Gemini 1.5

Flash

Eval. budget

m

32

64

IO Budget

m

IO

32

24

16

8

0

32

EO Budget

m

EO

0

8

16

24

32

32

Avg. test acc. (%)

↑

83.25

†

84.90

85.17

84.82

83.71

85.04

Avg. test rank

↓

4.04

3.65

3.29

3.00

3.58

3.44

equally crucial and should not be relegated to an afterthought. Second, we note that the EO strategies studied are in no way exhaustive. In fact, in contrast to the sophisticated search and instruction generation approaches adopted by IO methods, they can even be considered elementary. Yet, they deliver comparable or more significant improvements. Thus, we anticipate advanced methods that more effectively optimize exemplars would yield even greater enhancements: some techniques in IO may be adapted to EO with little-to-no modifications. For example, many recent advances in IO adopt an evolutionary search framework - while our 'Mutation' baseline can be seen as an elementary version of it in EO, it should be also straightforward to use more advanced search strategies and operators or use techniques like LLM-generated paraphrasing on top of selected exemplars. Other search techniques, such as sample-efficient combinatorial BO [5, 44, 11, 65], can be uniquely suitable for the EO setup, which is itself a combinatorial optimization problem 4 . Furthermore, while we used a fixed set of exemplars (i.e., task -wise selection), it might also be fruitful to further explore in the direction of instance -wise selection [51, 37]. Lastly, as discussed, the presence of (often large) generalization gap also suggests the importance to consider generalization alongside optimization, which seems to be the chief focus thus far; it might be promising to investigate analogies of well-tested in classical machine learning like regularization and cross-validation in APO.

Insight 3 : Optimizing both instructions and exemplars is greater than the sum of its parts, even under a comparable computational budget.

For most of the results obtained, we note that iteratively optimizing both instructions and exemplars led to the best performance. This naturally leads us to answer the second research question: whereas experiments in Tables 1 and 2 expend additional cost by optimizing exemplars on top of the optimized instructions, we show that 1) such a combinable benefit does not simply root from the additional compute and 2) optimally mixing-and-matching IO and EO leads to significant performance improvement with negligible additional overhead.

Concretely, we budget a total m = 32 prompt evaluations on D val where we use first m IO ={0,8,16,24,32} iterations optimizing instructions and the remaining m EO optimizing exemplars. We summarize the results in Fig. 7, where we find that 1) any mix-and-match outperforms IO or EO only (i.e., m IO =0 or 32), and 2) the best allocation bridges the gap or almost bridges the gap compared to the combination that uses twice as many prompt evaluations (last column) - interestingly, in this case the optimal allocation also roughly reflects the relative contribution of IO and EO to the overall performance improvement in Table 1. We show in App. B.7 that the above findings hold for other target models and instruction optimizers, and we also give detailed examples and explanations of the mechanism leading to this synergy. Additionally, in App. B.10, we compare this simple routine against PromptBreeder [14], which is one of the few existing IO methods that supports EO via an

optional 'context shuffling' routine. It, however, mutates the exemplars purely stochastically rather than optimizing from the validation metric. We show that despite the simplicity, our algorithm converges to comparable or better solutions while incurring a fraction of the PromptBreeder cost, which often requires hundreds of evaluations before convergence. Finally, we note that the presented way to combine optimization-based IO and EO is a proof of concept and room for future improvement can be vast and we experiment several other alternative ways to combine them in App. B.8, but we defer thorough investigations to a future work.

Insight 4 : SoTA IO methods may be inadvertently relying on exemplars already.

Beyond inspecting performance metrics only, we also examine the actual instructions and exemplars discovered. While detailed prompts are available in App. C, we highlight a key observation that adds a new dimension to our discussion: SoTA IO strategies may inadvertently utilize exemplars already. Despite IO methods not typically explicitly focusing on exemplars, we find them to frequently generate texts resembling exemplars within instructions through feedback and reflection processes. For instance, PromptBreeder employs 'Lamarckian mutation' to reconstruct instructions from inputoutput pairs, while ProTeGi prompts the optimizer model to analyze target model errors. These operators, though varied, all involve taking actual validation exemplars as inputs to optimizer models. As exemplified by Fig. 8, while the original intention may have been to abstract task-level instructions, the model occasionally incorporates these exemplars verbatim in the instructions. Whereas these 'quasi-exemplars' may seem unintentional, we observe that they are surprisingly common in highperforming instructions and often contribute more to the performance than the actual instructions themselves. Full prompt: 84.5 %

We argue that the findings here provide further evidence corroborating our insights obtained so far and our suggestions advocating explicit EO. Indeed, in contrast to explicit optimization of the exemplars, the aforementioned mechanism of exemplar discovery via IO is entirely opportunistic and, depending on interpretation, an unintentional artifact. For example, the quasi-exemplars in Fig. 8 almost certainly originate from the optimizer model in ProTeGi taking a convenient shortcut by incorporating a critique into the instruction verbatim (note the presence of traces like ' Label: a Prediction: b ' which suggests a previous mistake by the target model), which should not happen if the optimizer model perfectly executes the intended task of abstracting these critiques. Thus, we argue that instead of relying on the opportunistic exemplar generation via IO, explicitly optimizing for exemplars can be more preferable, as shown throughout this study.

4 Conclusion

We present comprehensive evaluations on the SoTA IO and EO methods, both individually and combined. We demonstrate that EO can be a potentially more crucial element in APO, revealing a beneficial synergy through joint optimization with IO. We also find

that the high performance of SoTA IO methods can themselves be driven by implicit yet spontaneous exemplar discovery. Overall, we advocate for further research into EO, both as an independent approach and in conjunction with IO, even for highly-capable instruction-following modern models. One limitation is that although the tasks we consider are fairly diverse and findings on it are already of value given the widespread interest just on these tasks only , they are not exhaustive, omitting tasks like open-ended longer-form generation and metrics like safety & harmfulness which are important for responsible use of LLMs. As is the case for any inductive study deriving insights from experiments, there is also a possibility that the findings may not fully generalize to other tasks and/or models. Expanding to include these aspects would be important for future work.

References

Achiam, J., Adler, S., Agarwal, S., Ahmad, L., Akkaya, I., Aleman, F. L., Almeida, D., Altenschmidt, J., Altman, S., Anadkat, S., et al. GPT-4 technical report. arXiv preprint arXiv:2303.08774 , 2023.

Agarwal, E., Dani, V., Ganu, T., and Nambi, A. Promptwizard: Task-aware agent-driven prompt optimization framework. arXiv preprint arXiv:2405.18369 , 2024.

Agarwal, R., Singh, A., Zhang, L. M., Bohnet, B., Chan, S., Anand, A., Abbas, Z., Nova, A., Co-Reyes, J. D., Chu, E., et al. Many-shot in-context learning. arXiv preprint arXiv:2404.11018 , 2024.

Anil, R., Dai, A. M., Firat, O., Johnson, M., Lepikhin, D., Passos, A., Shakeri, S., Taropa, E., Bailey, P., Chen, Z., et al. Palm 2 technical report. arXiv preprint arXiv:2305.10403 , 2023.

Baptista, R. and Poloczek, M. Bayesian optimization of combinatorial structures. In International Conference on Machine Learning , pp. 462-471. PMLR, 2018.

Brown, T., Mann, B., Ryder, N., Subbiah, M., Kaplan, J. D., Dhariwal, P., Neelakantan, A., Shyam, P., Sastry, G., Askell, A., et al. Language models are few-shot learners. Advances in neural information processing systems , 33:1877-1901, 2020.

Chen, L., Chen, J., Goldstein, T., Huang, H., and Zhou, T. Instructzero: Efficient instruction optimization for black-box large language models. arXiv preprint arXiv:2306.03082 , 2023.

Chen, Y., Zhao, C., Yu, Z., McKeown, K., and He, H. On the relation between sensitivity and accuracy in in-context learning. arXiv preprint arXiv:2209.07661 , 2022.

Chowdhery, A., Narang, S., Devlin, J., Bosma, M., Mishra, G., Roberts, A., Barham, P., Chung, H. W., Sutton, C., Gehrmann, S., et al. Palm: Scaling language modeling with pathways. Journal of Machine Learning Research , 24(240):1-113, 2023.

Coulom, R. Efficient selectivity and backup operators in monte-carlo tree search. In International conference on computers and games , pp. 72-83. Springer, 2006.

Daulton, S., Wan, X., Eriksson, D., Balandat, M., Osborne, M. A., and Bakshy, E. Bayesian optimization over discrete and mixed spaces via probabilistic reparameterization. Advances in Neural Information Processing Systems , 35:12760-12774, 2022.

Deng, M., Wang, J., Hsieh, C.-P., Wang, Y., Guo, H., Shu, T., Song, M., Xing, E., and Hu, Z. RLPrompt: Optimizing discrete text prompts with reinforcement learning. In Goldberg, Y., Kozareva, Z., and Zhang, Y. (eds.), Proceedings of the 2022 Conference on Empirical Methods in Natural Language Processing , pp. 3369-3391, Abu Dhabi, United Arab Emirates, December 2022. Association for Computational Linguistics. doi: 10.18653/v1/2022.emnlp-main.222. URL https://aclanthology.org/2022.emnlp-main.222 .

Do, X. L., Zhao, Y., Brown, H., Xie, Y., Zhao, J. X., Chen, N. F., Kawaguchi, K., Xie, M. Q., and He, J. Prompt optimization via adversarial in-context learning. arXiv preprint arXiv:2312.02614 , 2023.

Fernando, C., Banarse, D., Michalewski, H., Osindero, S., and Rocktäschel, T. Promptbreeder: Self-referential self-improvement via prompt evolution. arXiv preprint arXiv:2309.16797 , 2023.

Gemini team, Anil, R., Borgeaud, S., Wu, Y., Alayrac, J.-B., Yu, J., Soricut, R., Schalkwyk, J., Dai, A. M., Hauth, A., et al. Gemini: a family of highly capable multimodal models. arXiv preprint arXiv:2312.11805 , 2023.

Guo, Q., Wang, R., Guo, J., Li, B., Song, K., Tan, X., Liu, G., Bian, J., and Yang, Y. Connecting large language models with evolutionary algorithms yields powerful prompt optimizers. International Conference on Learning Representations , 2024.

Hendrycks, D., Burns, C., Basart, S., Zou, A., Mazeika, M., Song, D., and Steinhardt, J. Measuring massive multitask language understanding. arXiv preprint arXiv:2009.03300 , 2020.

Honovich, O., Shaham, U., Bowman, S. R., and Levy, O. Instruction induction: From few examples to natural language task descriptions. In Rogers, A., Boyd-Graber, J., and Okazaki, N. (eds.), Proceedings of the 61st Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers) , pp. 1935-1952, Toronto, Canada, July 2023. Association for Computational Linguistics. doi: 10.18653/v1/2023.acl-long.108. URL https://aclanthology.org/2023.acl-long.108 .

Hsieh, C.-J., Si, S., Yu, F. X., and Dhillon, I. S. Automatic engineering of long prompts. arXiv preprint arXiv:2311.10117 , 2023.

Khattab, O., Santhanam, K., Li, X. L., Hall, D., Liang, P., Potts, C., and Zaharia, M. Demonstrate-search-predict: Composing retrieval and language models for knowledge-intensive nlp. arXiv preprint arXiv:2212.14024 , 2022.

Khattab, O., Singhvi, A., Maheshwari, P., Zhang, Z., Santhanam, K., Vardhamanan, S., Haq, S., Sharma, A., Joshi, T. T., Moazam, H., et al. DSPy: Compiling declarative language model calls into self-improving pipelines. International Conference on Learning Representations , 2024.

Kim, H. J., Cho, H., Kim, J., Kim, T., Yoo, K. M., and Lee, S.-g. Self-generated in-context learning: Leveraging auto-regressive language models as a demonstration generator. arXiv preprint arXiv:2206.08082 , 2022.

Kojima, T., Gu, S. S., Reid, M., Matsuo, Y., and Iwasawa, Y. Large language models are zero-shot reasoners. Advances in neural information processing systems , 35:22199-22213, 2022.

Lee, J., Dai, Z., Ren, X., Chen, B., Cer, D., Cole, J. R., Hui, K., Boratko, M., Kapadia, R., Ding, W., et al. Gecko: Versatile text embeddings distilled from large language models. arXiv preprint arXiv:2403.20327 , 2024.

Lin, X., Wu, Z., Dai, Z., Hu, W., Shu, Y., Ng, S.-K., Jaillet, P., and Low, B. K. H. Use your instinct: Instruction optimization using neural bandits coupled with transformers. arXiv preprint arXiv:2310.02905 , 2023.

Liu, P., Yuan, W., Fu, J., Jiang, Z., Hayashi, H., and Neubig, G. Pre-train, prompt, and predict: A systematic survey of prompting methods in natural language processing. ACM Computing Surveys , 55(9):1-35, 2023.

Liu, S., Chen, C., Qu, X., Tang, K., and Ong, Y.-S. Large language models as evolutionary optimizers. arXiv preprint arXiv:2310.19046 , 2023.

Lu, Y., Bartolo, M., Moore, A., Riedel, S., and Stenetorp, P. Fantastically ordered prompts and where to find them: Overcoming few-shot prompt order sensitivity. In Proceedings of the 60th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers) , pp. 8086-8098, 2022.

Margatina, K., Schick, T., Aletras, N., and Dwivedi-Yu, J. Active learning principles for in-context learning with large language models. In Bouamor, H., Pino, J., and Bali, K. (eds.), Findings of the Association for Computational Linguistics: EMNLP 2023 , pp. 5011-5034, Singapore, December 2023. Association for Computational Linguistics. doi: 10.18653/v1/2023. findings-emnlp.334. URL https://aclanthology.org/2023.findings-emnlp.334 .

Mizrahi, M., Kaplan, G., Malkin, D., Dror, R., Shahaf, D., and Stanovsky, G. State of what art? a call for multi-prompt llm evaluation. arXiv preprint arXiv:2401.00595 , 2024.

Nguyen, T. and Wong, E. In-context example selection with influences. arXiv preprint arXiv:2302.11042 , 2023.

Opsahl-Ong, K., Ryan, M. J., Purtell, J., Broman, D., Potts, C., Zaharia, M., and Khattab, O. Optimizing instructions and demonstrations for multi-stage language model programs. arXiv preprint arXiv:2406.11695 , 2024.

Perez, E., Kiela, D., and Cho, K. True few-shot learning with language models. Advances in neural information processing systems , 34:11054-11070, 2021.

Prasad, A., Hase, P., Zhou, X., and Bansal, M. GrIPS: Gradient-free, edit-based instruction search for prompting large language models. In Vlachos, A. and Augenstein, I. (eds.), Proceedings of the 17th Conference of the European Chapter of the Association for Computational Linguistics , pp. 3845-3864, Dubrovnik, Croatia, May 2023. Association for Computational Linguistics. doi: 10.18653/v1/2023.eacl-main.277. URL https://aclanthology.org/2023. eacl-main.277 .

Pryzant, R., Iter, D., Li, J., Lee, Y ., Zhu, C., and Zeng, M. Automatic prompt optimization with 'gradient descent' and beam search. In Bouamor, H., Pino, J., and Bali, K. (eds.), Proceedings of the 2023 Conference on Empirical Methods in Natural Language Processing , pp. 7957-7968, Singapore, December 2023. Association for Computational Linguistics. doi: 10.18653/v1/2023. emnlp-main.494. URL https://aclanthology.org/2023.emnlp-main.494 .

Reid, M., Savinov, N., Teplyashin, D., Lepikhin, D., Lillicrap, T., Alayrac, J.-b., Soricut, R., Lazaridou, A., Firat, O., Schrittwieser, J., et al. Gemini 1.5: Unlocking multimodal understanding across millions of tokens of context. arXiv preprint arXiv:2403.05530 , 2024.

Rubin, O., Herzig, J., and Berant, J. Learning to retrieve prompts for in-context learning. In Carpuat, M., de Marneffe, M.-C., and Meza Ruiz, I. V. (eds.), Proceedings of the 2022 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies , pp. 2655-2671, Seattle, United States, July 2022. Association for Computational Linguistics. doi: 10.18653/v1/2022.naacl-main.191. URL https:// aclanthology.org/2022.naacl-main.191 .

Sahoo, P., Singh, A. K., Saha, S., Jain, V., Mondal, S., and Chadha, A. A systematic survey of prompt engineering in large language models: Techniques and applications. arXiv preprint arXiv:2402.07927 , 2024.

Sclar, M., Choi, Y., Tsvetkov, Y., and Suhr, A. Quantifying language models' sensitivity to spurious features in prompt design or: How i learned to start worrying about prompt formatting. arXiv preprint arXiv:2310.11324 , 2023.

Shin, T., Razeghi, Y., Logan IV, R. L., Wallace, E., and Singh, S. Autoprompt: Eliciting knowledge from language models with automatically generated prompts. In Proceedings of the 2020 Conference on Empirical Methods in Natural Language Processing (EMNLP) , pp. 4222-4235, 2020.

Stani´ c, A., Caelles, S., and Tschannen, M. Towards truly zero-shot compositional visual reasoning with llms as programmers. arXiv preprint arXiv:2401.01974 , 2024.

Sun, H., Li, X., Xu, Y., Homma, Y., Cao, Q., Wu, M., Jiao, J., and Charles, D. Autohint: Automatic prompt optimization with hint generation. arXiv preprint arXiv:2307.07415 , 2023.

Suzgun, M., Scales, N., Schärli, N., Gehrmann, S., Tay, Y., Chung, H. W., Chowdhery, A., Le, Q. V., Chi, E. H., Zhou, D., et al. Challenging big-bench tasks and whether chain-of-thought can solve them. arXiv preprint arXiv:2210.09261 , 2022.

Wan, X., Nguyen, V., Ha, H., Ru, B., Lu, C., and Osborne, M. A. Think global and act local: Bayesian optimisation over high-dimensional categorical and mixed search spaces. In International Conference on Machine Learning , pp. 10663-10674. PMLR, 2021.

Wan, X., Sun, R., Dai, H., Arik, S., and Pfister, T. Better zero-shot reasoning with selfadaptive prompting. In Rogers, A., Boyd-Graber, J., and Okazaki, N. (eds.), Findings of the Association for Computational Linguistics: ACL 2023 , pp. 3493-3514, Toronto, Canada, July 2023. Association for Computational Linguistics. doi: 10.18653/v1/2023.findings-acl.216. URL https://aclanthology.org/2023.findings-acl.216 .

Wan, X., Sun, R., Nakhost, H., Dai, H., Eisenschlos, J., Arik, S., and Pfister, T. Universal self-adaptive prompting. In Bouamor, H., Pino, J., and Bali, K. (eds.), Proceedings of the 2023 Conference on Empirical Methods in Natural Language Processing , pp. 7437-7462, Singapore, December 2023. Association for Computational Linguistics. doi: 10.18653/v1/2023. emnlp-main.461. URL https://aclanthology.org/2023.emnlp-main.461 .

Wang, R., An, S., Cheng, M., Zhou, T., Hwang, S. J., and Hsieh, C.-J. Mixture-of-experts in prompt optimization. International Conference on Machine Learning , 2024.

Wang, X., Li, C., Wang, Z., Bai, F., Luo, H., Zhang, J., Jojic, N., Xing, E. P., and Hu, Z. Promptagent: Strategic planning with language models enables expert-level prompt optimization. International Conference on Learning Representations , 2024.

Wei, J., Bosma, M., Zhao, V., Guu, K., Yu, A. W., Lester, B., Du, N., Dai, A. M., and Le, Q. V. Finetuned language models are zero-shot learners. In International Conference on Learning Representations , 2022. URL https://openreview.net/forum?id=gEZrGCozdqR .

Wei, J., Wang, X., Schuurmans, D., Bosma, M., Xia, F., Chi, E., Le, Q. V., Zhou, D., et al. Chain-of-thought prompting elicits reasoning in large language models. Advances in neural information processing systems , 35:24824-24837, 2022.

Wu, Z., Wang, Y., Ye, J., and Kong, L. Self-adaptive in-context learning: An information compression perspective for in-context example selection and ordering. In Proceedings of the 61st Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers) , pp. 1423-1436, 2023.

Wu, Z., Lin, X., Dai, Z., Hu, W., Shu, Y., Ng, S.-K., Jaillet, P., and Low, B. K. H. Prompt optimization with ease? efficient ordering-aware automated selection of exemplars. arXiv preprint arXiv:2405.16122 , 2024.

Xu, H., Chen, Y., Du, Y., Shao, N., Wang, Y., Li, H., and Yang, Z. Gps: Genetic prompt search for efficient few-shot learning. arXiv preprint arXiv:2210.17041 , 2022.

Yang, C., Wang, X., Lu, Y., Liu, H., Le, Q. V., Zhou, D., and Chen, X. Large language models as optimizers. In International Conference on Learning Representations , 2024.

Ye, J., Wu, Z., Feng, J., Yu, T., and Kong, L. Compositional exemplars for in-context learning. In International Conference on Machine Learning , pp. 39818-39833. PMLR, 2023.

Ye, Q., Axmed, M., Pryzant, R., and Khani, F. Prompt engineering a prompt engineer. arXiv preprint arXiv:2311.05661 , 2023.

Zhang, T., Wang, X., Zhou, D., Schuurmans, D., and Gonzalez, J. E. Tempera: Test-time prompting via reinforcement learning. International Conference on Learning Representations , 2023.

Zhang, Y., Feng, S., and Tan, C. Active example selection for in-context learning. In Goldberg, Y., Kozareva, Z., and Zhang, Y. (eds.), Proceedings of the 2022 Conference on Empirical Methods in Natural Language Processing , pp. 9134-9148, Abu Dhabi, United Arab Emirates, December 2022. Association for Computational Linguistics. doi: 10.18653/v1/2022.emnlp-main.622. URL https://aclanthology.org/2022.emnlp-main.622 .

Zhang, Y., Zhou, K., and Liu, Z. What makes good examples for visual in-context learning? Advances in Neural Information Processing Systems , 36, 2024.

Zhang, Z., Zhang, A., Li, M., and Smola, A. Automatic chain of thought prompting in large language models. International Conference on Learning Representations , 2023.

Zhao, Z., Wallace, E., Feng, S., Klein, D., and Singh, S. Calibrate before use: Improving few-shot performance of language models. In International conference on machine learning , pp. 12697-12706. PMLR, 2021.

Zhou, H., Wan, X., Vuli´ c, I., and Korhonen, A. Survival of the most influential prompts: Efficient black-box prompt search via clustering and pruning. In Bouamor, H., Pino, J., and Bali, K. (eds.), Findings of the Association for Computational Linguistics: EMNLP 2023 , pp. 13064-13077, Singapore, December 2023. Association for Computational Linguistics. doi: 10.18653/v1/2023. findings-emnlp.870. URL https://aclanthology.org/2023.findings-emnlp.870 .

Zhou, H., Wan, X., Liu, Y., Collier, N., Vuli´ c, I., and Korhonen, A. Fairer preferences elicit improved human-aligned large language model judgments. arXiv preprint arXiv:2406.11370 , 2024.

Zhou, H., Wan, X., Proleev, L., Mincu, D., Chen, J., Heller, K., and Roy, S. Batch calibration: Rethinking calibration for in-context learning and prompt engineering. International Conference on Learning Representations (ICLR) , 2024.

Zhou, H., Wan, X., Vuli´ c, I., and Korhonen, A. Autopeft: Automatic configuration search for parameter-efficient fine-tuning. Transactions of the Association for Computational Linguistics , 12:525-542, 2024.

Zhou, Y., Muresanu, A. I., Han, Z., Paster, K., Pitis, S., Chan, H., and Ba, J. Large language models are human-level prompt engineers. In The Eleventh International Conference on Learning Representations , 2023.

NeurIPS Paper Checklist

1. Claims

Question: Do the main claims made in the abstract and introduction accurately reflect the paper's contributions and scope?

Answer: [Yes]

Justification: The abstract and instroduction accurately reflect our contributions and scope.

Guidelines:

The answer NA means that the abstract and introduction do not include the claims made in the paper.

The abstract and/or introduction should clearly state the claims made, including the contributions made in the paper and important assumptions and limitations. A No or NA answer to this question will not be perceived well by the reviewers.

The claims made should match theoretical and experimental results, and reflect how much the results can be expected to generalize to other settings.

It is fine to include aspirational goals as motivation as long as it is clear that these goals are not attained by the paper.

2. Limitations

Question: Does the paper discuss the limitations of the work performed by the authors?

Answer: [Yes]

Justification: We clearly discuss our limitations in the Conclusion.

Guidelines:

The answer NA means that the paper has no limitation while the answer No means that the paper has limitations, but those are not discussed in the paper.

The authors are encouraged to create a separate "Limitations" section in their paper.

The paper should point out any strong assumptions and how robust the results are to violations of these assumptions (e.g., independence assumptions, noiseless settings, model well-specification, asymptotic approximations only holding locally). The authors should reflect on how these assumptions might be violated in practice and what the implications would be.

The authors should reflect on the scope of the claims made, e.g., if the approach was only tested on a few datasets or with a few runs. In general, empirical results often depend on implicit assumptions, which should be articulated.

The authors should reflect on the factors that influence the performance of the approach. For example, a facial recognition algorithm may perform poorly when image resolution is low or images are taken in low lighting. Or a speech-to-text system might not be used reliably to provide closed captions for online lectures because it fails to handle technical jargon.

The authors should discuss the computational efficiency of the proposed algorithms and how they scale with dataset size.

If applicable, the authors should discuss possible limitations of their approach to address problems of privacy and fairness.

While the authors might fear that complete honesty about limitations might be used by reviewers as grounds for rejection, a worse outcome might be that reviewers discover limitations that aren't acknowledged in the paper. The authors should use their best judgment and recognize that individual actions in favor of transparency play an important role in developing norms that preserve the integrity of the community. Reviewers will be specifically instructed to not penalize honesty concerning limitations.

3. Theory Assumptions and Proofs

Question: For each theoretical result, does the paper provide the full set of assumptions and a complete (and correct) proof?

Answer: [NA]

Justification: The paper does not include theoretical results.

Guidelines:

The answer NA means that the paper does not include theoretical results.

All the theorems, formulas, and proofs in the paper should be numbered and crossreferenced.

All assumptions should be clearly stated or referenced in the statement of any theorems.

The proofs can either appear in the main paper or the supplemental material, but if they appear in the supplemental material, the authors are encouraged to provide a short proof sketch to provide intuition.

Inversely, any informal proof provided in the core of the paper should be complemented by formal proofs provided in appendix or supplemental material.

Theorems and Lemmas that the proof relies upon should be properly referenced.

4. Experimental Result Reproducibility

Question: Does the paper fully disclose all the information needed to reproduce the main experimental results of the paper to the extent that it affects the main claims and/or conclusions of the paper (regardless of whether the code and data are provided or not)?

Answer: [Yes]

Justification: We fully disclose the datasets (all datasets are publicly accessible) and the models used (all models have public API). We include all discussions about the implementation details in Appendix. A.

Guidelines:

The answer NA means that the paper does not include experiments.

If the paper includes experiments, a No answer to this question will not be perceived well by the reviewers: Making the paper reproducible is important, regardless of whether the code and data are provided or not.

If the contribution is a dataset and/or model, the authors should describe the steps taken to make their results reproducible or verifiable.

Depending on the contribution, reproducibility can be accomplished in various ways. For example, if the contribution is a novel architecture, describing the architecture fully might suffice, or if the contribution is a specific model and empirical evaluation, it may be necessary to either make it possible for others to replicate the model with the same dataset, or provide access to the model. In general. releasing code and data is often one good way to accomplish this, but reproducibility can also be provided via detailed instructions for how to replicate the results, access to a hosted model (e.g., in the case of a large language model), releasing of a model checkpoint, or other means that are appropriate to the research performed.

While NeurIPS does not require releasing code, the conference does require all submissions to provide some reasonable avenue for reproducibility, which may depend on the nature of the contribution. For example

(a) If the contribution is primarily a new algorithm, the paper should make it clear how to reproduce that algorithm.

(b) If the contribution is primarily a new model architecture, the paper should describe the architecture clearly and fully.

(c) If the contribution is a new model (e.g., a large language model), then there should either be a way to access this model for reproducing the results or a way to reproduce the model (e.g., with an open-source dataset or instructions for how to construct the dataset).

(d) We recognize that reproducibility may be tricky in some cases, in which case authors are welcome to describe the particular way they provide for reproducibility. In the case of closed-source models, it may be that access to the model is limited in some way (e.g., to registered users), but it should be possible for other researchers to have some path to reproducing or verifying the results.

5. Open access to data and code

Question: Does the paper provide open access to the data and code, with sufficient instructions to faithfully reproduce the main experimental results, as described in supplemental material?

Answer: [No]

Justification: While we aim to open-source our experimental code in the future, we cannot open source the codebase at the time of submission. However, many of our experiments derive from existing methods, which can be reproduced by running the respective, opensourced codebase which we outline in Appendix A.

Guidelines:

The answer NA means that paper does not include experiments requiring code.

Please see the NeurIPS code and data submission guidelines ( https://nips.cc/ public/guides/CodeSubmissionPolicy ) for more details.

While we encourage the release of code and data, we understand that this might not be possible, so 'No' is an acceptable answer. Papers cannot be rejected simply for not including code, unless this is central to the contribution (e.g., for a new open-source benchmark).

The instructions should contain the exact command and environment needed to run to reproduce the results. See the NeurIPS code and data submission guidelines ( https: //nips.cc/public/guides/CodeSubmissionPolicy ) for more details.

The authors should provide instructions on data access and preparation, including how to access the raw data, preprocessed data, intermediate data, and generated data, etc.

The authors should provide scripts to reproduce all experimental results for the new proposed method and baselines. If only a subset of experiments are reproducible, they should state which ones are omitted from the script and why.

At submission time, to preserve anonymity, the authors should release anonymized versions (if applicable).

Providing as much information as possible in supplemental material (appended to the paper) is recommended, but including URLs to data and code is permitted.

6. Experimental Setting/Details

Question: Does the paper specify all the training and test details (e.g., data splits, hyperparameters, how they were chosen, type of optimizer, etc.) necessary to understand the results?

Answer: [Yes]

Justification: We fully disclose the datasets (all datasets are publicly accessible) and the models used (all models have public API). We include all discussions about the implementation details in Appendix. A.

Guidelines:

The answer NA means that the paper does not include experiments.

The experimental setting should be presented in the core of the paper to a level of detail that is necessary to appreciate the results and make sense of them.

The full details can be provided either with the code, in appendix, or as supplemental material.

7. Experiment Statistical Significance

Question: Does the paper report error bars suitably and correctly defined or other appropriate information about the statistical significance of the experiments?

Answer: [No]

Justification: While we do not repeat each experiments more than once due to the computational constraints, we perform experiments on a large set of tasks, and insights are largely drawn from the aggregated results averaged over 70+ different tasks to improve the reliability of our results.

Guidelines:

The answer NA means that the paper does not include experiments.

The authors should answer "Yes" if the results are accompanied by error bars, confidence intervals, or statistical significance tests, at least for the experiments that support the main claims of the paper.

The factors of variability that the error bars are capturing should be clearly stated (for example, train/test split, initialization, random drawing of some parameter, or overall run with given experimental conditions).

The method for calculating the error bars should be explained (closed form formula, call to a library function, bootstrap, etc.)

The assumptions made should be given (e.g., Normally distributed errors).

It should be clear whether the error bar is the standard deviation or the standard error of the mean.

It is OK to report 1-sigma error bars, but one should state it. The authors should preferably report a 2-sigma error bar than state that they have a 96% CI, if the hypothesis of Normality of errors is not verified.

For asymmetric distributions, the authors should be careful not to show in tables or figures symmetric error bars that would yield results that are out of range (e.g. negative error rates).

If error bars are reported in tables or plots, The authors should explain in the text how they were calculated and reference the corresponding figures or tables in the text.

8. Experiments Compute Resources

Question: For each experiment, does the paper provide sufficient information on the computer resources (type of compute workers, memory, time of execution) needed to reproduce the experiments?

Answer: [Yes]

Justification: We include all necessary details in Appendix. A.

Guidelines:

The answer NA means that the paper does not include experiments.

The paper should indicate the type of compute workers CPU or GPU, internal cluster, or cloud provider, including relevant memory and storage.

The paper should provide the amount of compute required for each of the individual experimental runs as well as estimate the total compute.

The paper should disclose whether the full research project required more compute than the experiments reported in the paper (e.g., preliminary or failed experiments that didn't make it into the paper).

9. Code Of Ethics

Question: Does the research conducted in the paper conform, in every respect, with the NeurIPS Code of Ethics https://neurips.cc/public/EthicsGuidelines ?

Answer: [Yes]

Justification: We have reviewed and agreed to the NeurIPS code of ethics.

Guidelines:

The answer NA means that the authors have not reviewed the NeurIPS Code of Ethics.

If the authors answer No, they should explain the special circumstances that require a deviation from the Code of Ethics.

The authors should make sure to preserve anonymity (e.g., if there is a special consideration due to laws or regulations in their jurisdiction).

10. Broader Impacts

Question: Does the paper discuss both potential positive societal impacts and negative societal impacts of the work performed?

Answer: [Yes]

Justification: We include such discussion in the Conclusion.

Guidelines:

The answer NA means that there is no societal impact of the work performed.

If the authors answer NA or No, they should explain why their work has no societal impact or why the paper does not address societal impact.

Examples of negative societal impacts include potential malicious or unintended uses (e.g., disinformation, generating fake profiles, surveillance), fairness considerations (e.g., deployment of technologies that could make decisions that unfairly impact specific groups), privacy considerations, and security considerations.

The conference expects that many papers will be foundational research and not tied to particular applications, let alone deployments. However, if there is a direct path to any negative applications, the authors should point it out. For example, it is legitimate to point out that an improvement in the quality of generative models could be used to generate deepfakes for disinformation. On the other hand, it is not needed to point out that a generic algorithm for optimizing neural networks could enable people to train models that generate Deepfakes faster.

The authors should consider possible harms that could arise when the technology is being used as intended and functioning correctly, harms that could arise when the technology is being used as intended but gives incorrect results, and harms following from (intentional or unintentional) misuse of the technology.

If there are negative societal impacts, the authors could also discuss possible mitigation strategies (e.g., gated release of models, providing defenses in addition to attacks, mechanisms for monitoring misuse, mechanisms to monitor how a system learns from feedback over time, improving the efficiency and accessibility of ML).

11. Safeguards

Question: Does the paper describe safeguards that have been put in place for responsible release of data or models that have a high risk for misuse (e.g., pretrained language models, image generators, or scraped datasets)?

Answer: [NA]

Justification: This paper does not release new generative models or scraped datasets but rather derive from existing models and datasets. Therefore, we do not anticipate such risks.

Guidelines:

The answer NA means that the paper poses no such risks.

Released models that have a high risk for misuse or dual-use should be released with necessary safeguards to allow for controlled use of the model, for example by requiring that users adhere to usage guidelines or restrictions to access the model or implementing safety filters.

Datasets that have been scraped from the Internet could pose safety risks. The authors should describe how they avoided releasing unsafe images.

We recognize that providing effective safeguards is challenging, and many papers do not require this, but we encourage authors to take this into account and make a best faith effort.

12. Licenses for existing assets

Question: Are the creators or original owners of assets (e.g., code, data, models), used in the paper, properly credited and are the license and terms of use explicitly mentioned and properly respected?

Answer: [Yes]

Justification: We list the license and terms of use in Appendix A. Original creators/owners of all assets used in the paper are properly credited with citations.

Guidelines:

The answer NA means that the paper does not use existing assets.

The authors should cite the original paper that produced the code package or dataset.

The authors should state which version of the asset is used and, if possible, include a URL.

The name of the license (e.g., CC-BY 4.0) should be included for each asset.

For scraped data from a particular source (e.g., website), the copyright and terms of service of that source should be provided.

If assets are released, the license, copyright information, and terms of use in the package should be provided. For popular datasets, paperswithcode.com/datasets has curated licenses for some datasets. Their licensing guide can help determine the license of a dataset.

For existing datasets that are re-packaged, both the original license and the license of the derived asset (if it has changed) should be provided.

If this information is not available online, the authors are encouraged to reach out to the asset's creators.

13. New Assets

Question: Are new assets introduced in the paper well documented and is the documentation provided alongside the assets?

Answer: [NA]

Justification: The paper does not release new assets. We entirely use assets that have already been made available.

Guidelines:

The answer NA means that the paper does not release new assets.

Researchers should communicate the details of the dataset/code/model as part of their submissions via structured templates. This includes details about training, license, limitations, etc.

The paper should discuss whether and how consent was obtained from people whose asset is used.

At submission time, remember to anonymize your assets (if applicable). You can either create an anonymized URL or include an anonymized zip file.

14. Crowdsourcing and Research with Human Subjects

Question: For crowdsourcing experiments and research with human subjects, does the paper include the full text of instructions given to participants and screenshots, if applicable, as well as details about compensation (if any)?

Answer: [NA]

Justification: The paper does not involve crowdsourcing nor research with human subjects. Guidelines:

The answer NA means that the paper does not involve crowdsourcing nor research with human subjects.

Including this information in the supplemental material is fine, but if the main contribution of the paper involves human subjects, then as much detail as possible should be included in the main paper.

According to the NeurIPS Code of Ethics, workers involved in data collection, curation, or other labor should be paid at least the minimum wage in the country of the data collector.

15. Institutional Review Board (IRB) Approvals or Equivalent for Research with Human Subjects

Question: Does the paper describe potential risks incurred by study participants, whether such risks were disclosed to the subjects, and whether Institutional Review Board (IRB) approvals (or an equivalent approval/review based on the requirements of your country or institution) were obtained?

Answer: [NA]

Justification: The paper does not involve crowdsourcing nor research with human subjects.

Guidelines:

The answer NA means that the paper does not involve crowdsourcing nor research with human subjects.

Depending on the country in which research is conducted, IRB approval (or equivalent) may be required for any human subjects research. If you obtained IRB approval, you should clearly state this in the paper.

We recognize that the procedures for this may vary significantly between institutions and locations, and we expect authors to adhere to the NeurIPS Code of Ethics and the guidelines for their institution.

For initial submissions, do not include any information that would break anonymity (if applicable), such as the institution conducting the review.

Appendix

A Implementation Details

Input prompt templates. In this section, we outline the input prompt templates we used for all experiments for reproducibility. For seed instruction (i.e., No IO ), APE and OPRO, we adopt the following template for all datasets.

In the template above, QUERY_TEXT denotes the input text; INSTRUCTION denotes the instruction to be added, which is the principal optimizable component of IO methods; llm() denotes the location where LLM is prompted to generate an output. ANSWER_INSTRUCTION is a special, taskspecific sentence to ensure the LLM generates the final answer in a format that can be easily parsed. Specifically, for all multiple-choice questions-style tasks, it has the following content:

1 Show your final answer option bracketed between <answer > and

where the content of TASK_SPECIFIC_CONSTRAINT depends on the task:

boolean_expressions: (True or False only)

formal_fallacies: (valid or invalid only)

navigate, sports_understanding, causal_judgement, web_of_lies: (yes or no only)

word_sorting: (sorted words separated by spaces only)

all other tasks : None (empty string).

At test time, the final answer is extracted with the capturing pattern <answer>...<\answer> . When exemplars are added, we use the following template:

where DEMO_{i}_OUTPUT contains the entire response from the LLM to the corresponding input (not the final answer only). For ProTeGi and PromptAgent, we follow the templates used in the respective original papers with the format that puts the instruction and answer instruction in front of the test query:

Noting that instruction is stated once at the beginning only rather than repeated at each exemplar, in consistency to the original styles adopted by these papers.

Implementation details of IO methods. In this section, we describe the implementation details of the IO and EO methods adopted. For all methods, we use greedy decoding (temperature = 0) for the PaLM 2 ( text-bison-002 ) or Gemini target models. Whenever an optimizer model is used, we use temperature = 1.0, top_k = 40 and top_p = 0.8. For both PaLM 2 and Gemini models, we use the Google Cloud Vertex AI API available at https://cloud.google.com/vertex-ai .

APE : We adapt the official implementation available at https://github.com/keirp/ automatic_prompt_engineer . Instead of using 'instruction induction' [18] from exemplars which is the primary initialization method introduced in the original paper in §3.1, Forward Mode Generation or Reverse Mode Generation , we opt for the third option, Customized Prompts where we initialize APE at the seed prompt 'Let's think step by step' because 1) this ensures fairness in comparison with other methods and 2) we find initializing at the seed prompt actually leads to much better performance because it is well-known to induce step-by-step, CoT-style reasoning from the LLM. On the other hand, while the LLM induced initial instructions may describe the task better, the model often tends to utter the final answer without intermediate steps which we observe lead to much worse performance: on the BBH tasks selected for experimentation in this work, instruction induction using the meta-prompt provided by the APE paper only led to an average test accuracy of 56.7% on PaLM 2 ( text-bison-002 ), which is even worse than using the seed prompt with no additional optimization. For APE, we use a population size of 8 and allow for 4 generations.

OPRO : we adapt the official implementation available at https://github.com/ google-deepmind/opro . At each optimization step, we follow the authors by asking the optimizer model to generate 8 candidate prompts and we budget for 4 steps.

ProTeGi : We use the official implementation available at https://github.com/microsoft/ LMOps . We set the initial prompt to the seed prompt, and we always use the entire validation set to generate the 'gradients' (i.e., we use no mini-batching). We set the number of newly proposed instructions per optimization step to 8, where half of them come from 'gradients' (i.e., new instructions generated by using the optimizer model to critique past mistakes made by the target model) and the other half come from 'Monte Carlo samples' which are paraphrased/rewritten variants of the past prompts by the optimizer model. We again allow for 4 generations of mutations.

PromptAgent : We use the official implementation at https://github.com/XinyuanWangCS/ PromptAgent . We set the initial prompt to the seed prompt and use the default Monte Carlo Tree Search algorithm and set the number of maximum iterations to 32 to be consistent with the other methods described above.

Computational Resources. All experiments conducted in this work are accessible via public APIs where the underlying LLMs are hosted from the server side. There is no computational resource requirement on the client machine except that one can access Google Colab and run Python 3.10.

Datasets. As discussed briefly in §3.1, we rely on existing assets to perform experiments. Both the BIG-Bench Hard (BBH) dataset and the MMLU dataset we used are licensed under the MIT License (BBH: https://github.com/suzgunmirac/BIG-Bench-Hard/blob/main/LICENSE ; MMLU: https://github.com/hendrycks/test/blob/master/LICENSE ).

B Additional Experimental Results

B.1 Detailed Results on BBH

In Tables 5 to 10, we show the per-task performance breakdown using the PaLM 2 ( text-bison-002 ) target model whose aggregated results are presented in Table 1. In Table 11 and Table 12, we show the results using the Gemini 1.0 Pro and Gemini 1.5 Flash target models whose aggregated results are presented in Table 2 and Table 4, respectively. The aggregated and task-specific breakdown results on the GPT-3.5 ( gpt-3.5-turbo-0125 ) model are shown in Tables 13 and 14, respectively.

B.2 Detailed Per-task Results on MMLU

In Table 15, we show the per-task performance breakdown using the PaLM 2 ( text-bison-002 ) target model whose aggregated results are presented in Table 3. In Table 16, we show the MMLU results using the Gemini 1.0 Pro target model.

B.3 Additional Visualization of Comparison Across IO and EO

Complementary to Fig. 3 in the main text, in PaLM 2 model, we show the per-task change of test accuracy before and after applying EO for APE and OPRO in Fig. 9 (comparison between optimized exemplars vs. no exemplars ) and Fig. 10 (comparison between optimized exemplars vs random exemplars ). We also include a visualization on the effect of IO (comparing optimized instructions and initial instructions) in Fig. 11. We also visualize the Gemini 1.0 Pro results in Fig. 12.

25

50

75

100

Test accuracy (%)

0

5

10

15

20

25

Task index

+24%

+16%

+14%

+14%

+1%

+5%

-4%

+8%

-3%

+19%

+12%

+5%

+7%

+6%

+0%

+18%

+11%

+9%

+10%

+15%

+14%

-5%

+8%

+4%

+12%

63.0

73.1

No ES

Mut.

OPRO

25

50

75

100

Test accuracy (%)

0

5

10

15

20

25

Task index

+12%

+8%

+6%

-4%

+7%

+10%

+2%

+6%

+28%

-5%

+13%

+6%

+9%

+3%

+10%

+2%

+9%

+18%

+12%

+4%

+12%

+2%

-2%

+8%

+5%

+5%

65.7

72.8

No ES

Mut.

P.A.

25

50

75

100

Test accuracy (%)

0

5

10

15

20

25

Task index

+19%

+10%

+2%

+4%

+2%

+5%

+14%

0%

+28%

+8%

+8%

0%

+7%

-5%

-0%

+2%

+6%

+14%

+4%

+1%

+2%

+2%

+9%

+6%

+4%

+6%

66.9

72.9

Rand.

Mut.

No IO

25

50

75

100

Test accuracy (%)

0

5

10

15

20

25

Task index

+24%

+30%

+4%

+16%

-3%

+5%

+5%

+1%

+12%

+6%

+8%

+6%

+26%

+3%

+3%

+2%

+9%

+6%

+3%

+5%

+3%

+6%

+1%

+1%

-0%

+4%

69.1

76.2

Rand.

Mut.

APE

25

50

75

100

Test accuracy (%)

0

5

10

15

20

25

Task index

+19%

+12%

-2%

+8%

-6%

+5%

-7%

+1%

+26%

+5%

+8%

+6%

+7%

+1%

-2%

+0%

+8%

+7%

+2%

+1%

+7%

+14%

-8%

+2%

-0%

+2%

68.5

73.1

Rand.

Mut.

OPRO

25

50

75

100

Test accuracy (%)

0

5

10

15

20

25

Task index

+14%

+43%

+29%

+8%

-2%

+6%

+6%

+1%

+2%

+8%

+3%

+2%

+24%

+1%

-4%

+6%

-0%

+2%

+1%

+2%

+6%

+8%

-1%

-5%

+2%

+6%

70.8

77.3

Rand.

Mut.

ProT.

25

50

75

100

Test accuracy (%)

0

5

10

15

20

25

Task index

+27%

-2%

+3%

-1%

-0%

+18%

+2%

+1%

+50%

-1%

+1%

+14%

+8%

+6%

+4%

-1%

-3%

+2%

-0%

+3%

+2%

+6%

-2%

-8%

+2%

+3%

67.6

72.8

Rand.

Mut.

P.A.

25

50

75

100

Test accuracy (%)

0

5

10

15

20

25

Task index

+2%

+2%

+24%

+3%

+0%

+9%

+5%

-1%

+15%

+14%

+1%

+4%

-4%

0%

+4%

-4%

+2%

+14%

+4%

+3%

+7%

+1%

+11%

0%

+1%

+3%

60.3

65.0

No ES

APE

No IO

25

50

75

100

Test accuracy (%)

0

5

10

15

20

25

Task index

+12%

+1%

+24%

+4%

+1%

+2%

-18%

-2%

-2%

+23%

+11%

-0%

-1%

-3%

+2%

+2%

+4%

+12%

-4%

0%

+6%

-4%

+5%

+0%

-2%

-0%

60.3

63.0

No ES

OPRO

No IO

25

50

75

100

Test accuracy (%)

0

5

10

15

20

25

Task index

+14%

+16%

+5%

+8%

+5%

+6%

+18%

-5%

+26%

+29%

+15%

+9%

-8%

-1%

+2%

-2%

+10%

+21%

0%

-1%

+8%

+4%

+4%

+10%

0%

+12%

60.3

68.1

No ES

ProT.

No IO

25

50

75

100

Test accuracy (%)

0

5

10

15

20

25

Task index

+10%

-2%

+12%

+10%

+2%

+4%

+7%

-2%

+14%

+31%

+26%

+7%

-14%

+4%

-0%

+3%

+5%

+12%

-4%

+10%

+4%

+2%

+3%

-2%

-4%

+3%

60.3

65.7

No ES

P.A.

No IO

25

50

75

100

Test accuracy (%)

0

5

10

15

20

25

Task index

+22%

+20%

+12%

+9%

+1%

+12%

+12%

+7%

+41%

+10%

+2%

+12%

+9%

+10%

+14%

+14%

+18%

+12%

+4%

+6%

+10%

-5%

+18%

+8%

+10%

63.1

75.8

No ES

Mut.

No IO

25

50

75

100

Test accuracy (%)

0

5

10

15

20

25

Task index

0%

+26%

+2%

+4%

-3%

+20%

-0%

+5%

+2%

+6%

-8%

+4%

+38%

-1%

0%

+9%

-3%

+3%

+12%

+3%

-0%

-3%

-3%

+10%

+3%

-5%

71.1

75.8

Rand.

Mut.

No IO

25

50

75

100

Test accuracy (%)

0

5

10

15

20

25

Task index

+23%

+17%

+22%

+2%

0%

+4%

+13%

+12%

+24%

+16%

+10%

+19%

+50%

+7%

+9%

+7%

+15%

+16%

+14%

+7%

+10%

+18%

-7%

+17%

+5%

+11%

65.9

79.0

No ES

Mut.

ProT.

25

50

75

100

Test accuracy (%)

0

5

10

15

20

25

Task index

+2%

+28%

+2%

-0%

-5%

+8%

-3%

-0%

+1%

+24%

+10%

+3%

+31%

-3%

+4%

+6%

+4%

+7%

+4%

+4%

+4%

+21%

-4%

+10%

-4%

+10%

72.7

79.0

Rand.

Mut.

ProT.

Complementary to Fig. 5 in the main text, we visualize the additional BBH tasks on PaLM 2 in Fig. 13(a), Gemini 1.0 Pro in Fig. 13(b) and Gemini 1.5 Flash in Fig. 13. We also test key findings on a representative subset of 11 tasks in GPT-3.5 (Fig. 14. We visualize the MMLU results in Fig. 15.

Test acc.

Test acc.

























IO:







ES:

























































































































IO:





ES:











































































































































































90

100

boolean_expressions

60

70

causal_judgement

80

90

date_understanding

IO:

No IO

APE

ProTeGi

ES:

No ES

Random

Mutation

40

60

disambiguation_qa

60

70

formal_fallacies

50

75

geometric_shapes

80

90

Test acc.

hyperbaton

60

80

logical_deduction (5)

50

75

logical_deduction (7)

85

90

95

logical_deduction (3)

50

75

movie_recommend

80

100

multistep_arithmetic

60

70

Test acc.

navigate

90

100

object_counting

90

100

penguins_in_a_table

80

100

reasoning_about

colored_objects

70

80

90

ruin_names

60

70

salient_translation

error_detection

70

80

90

Test acc.

snarks

70

80

90

sports

understanding

95

100

105

temporal_sequences

90

100

tracking_shuffled

objects (5)

90

100

tracking_shuffled

objects (7)

95

100

tracking_shuffled

objects (3)

90

100

web_of_lies

50

60

70

word_sorting

'























IO:







ES:



















































































IO:





ES:

















































































































































































































































































































































 































































B.4 Varying Number of Shots

With the advent of LLMs that support longer context windows, an alternative approach to performing exemplar optimization is to scale the number of exemplars. Taking into the context of our setup, instead of selecting k exemplars from D c where k is a small value, we can scale k to a larger value or even use the entire D c as exemplars. In this section, we perform experiments on the relative merits of the two approaches.

We show the aggregated results in Table 17 and task-specific results for both models in Fig. 16 and Fig. 17. While we find increasing the number of exemplars generally leads to improved test performance up to k = 10 , further increase leads to performance deterioration. The increase in performance, however, does not diminish the importance of exemplar optimization as we show that using optimized exemplars outperform many more random exemplars, both at an aggregated level and on a task-specific level (the instances where optimized exemplars outperform random exemplars of any k , which are marked by magenta lines in Figs. 16 & 17, are more than 50% in both target models) - we argue that this suggests that exemplar optimization remains relevant even in modern LLMs with long contexts. In fact, we believe that novel exemplar search strategy in such many-shot setup can be an important next step, given that the number of possible combinations and permutations of exemplars explodes exponentially for a higher k , necessitating more advanced search strategies that would navigate this optimization landscape more effectively and efficiently. We defer a thorough investigation of this direction to a future work.





























































 











































































































































































































































































































































































































































B.5 Varying Validation Dataset Sizes

In the main experiments, we used 20%-80% validation-test split for all BBH tasks. In this section, we investigate the sensitivity of the various methods to changing validation data sizes, especially under very data-constrained setups such as very small validation data sizes. To do so, we vary the validation split to be {2, 4, 10, 20}% and we investigate on the combination of ProTeGi and Mutation, the bestperforming IO and EO methods, respectively. We summarize the average test accuracy against the size of validation data split (as a fraction of the total data available) in Fig. 18.

We find that optimization-based EO is remarkably robust towards the size of the validation size, with the smallest performance drop even with an extremely small validation set (i.e., 2% or 4%, which correspond to 5 or 10 validation samples). On the other hand,

0.02

0.10

0.20

Validation data (as fraction of all data)

60

65

70

75

Avg. test accuracy (%)

Seed instruction +

Mutated exemplars

ProTeGi +

No exemplars

ProTeGi +

Random exemplars

ProTeGi +

Mutated exemplars

Seed instruction +

No exemplars

ProTeGi dropped to a performance no better than un-optimized seed instruction under such an extremely data-constrained setup, and even the performance of ProTeGi + Mutation, which performs the best overall in the main text dropped significantly with a small validation data size.

B.6 EO with a Reduced Budget

In this section, we show the results of EO with search budget approximately halved (from m = 32 to m = 16 ) in Table 18 on top of the seed instructions without instruction optimization. We find that even after halving the budget (i.e., the EO strategies are now half as expensive), seed instruction and

optimized exemplars still outperform optimized instruction and random exemplars both in terms of average test accuracy and average rank.

B.7 Additional Results by Mixing-and-Matching IO and EO

Other choices of instruction optimizers and/or target models. We show results from mixingand-matching IO and EO in a similar manner to Fig. 7 but use ProTeGi (instead of APE in Fig. 7) on both PaLM 2 ( text-bison-002 ) and Gemini 1.0 Pro target models in Table 19, where we see that two key findings (that any mix-and-match outperforms IO and EO only and that optimal allocation completely bridges the gap compared to a more expensive routine) we made still hold, except on Gemini model, the optimal allocation occurs at 16/16 instead of 8/24. This confirms that the phenomenon we see in Fig. 7 in the main text is not specific to the choice of the target models and/or instruction optimizers but is instead generally applicable.

Mechanism of synergy between IO and EO. In this section, we give an example of why combined IO and EO outperform its constituting building blocks in Fig. 19: performing IO only (Fig. 19a) generates step-by-step text instructions, but without exemplars, the model is prone to making arithmetic errors. Performing EO only (Fig. 19b), on the other hand, subjects the model to the shortcomings of the un-optimized seed prompt, which, in this case, generates arithmetic operations-only outputs without textual reasoning. Combined IO and EO (Fig. 19c) addresses both issues, leading to vastly improved performance from both textual reasoning and exemplars.

B.8 Alternative Ways of Combining IO and EO

As discussed in §3.2, we conducted EO after IO in the main text: in our setup, the exemplars are bootstrapped from the model's own predictions, which in turn depend on the instructions given. As such, we performed instruction optimization first on the zero-shot setup and search for the optimal exemplar(s) amongst the predictions induced by the optimal instruction. While we believe this order is the most natural, it is also important to ensure that any findings derived in this paper are not biased by the particular order we chose. Thus, in this section, we also experiment with two alternatives:

Inverted order . We first optimize the exemplars based on the seed instruction ('Let's think step by step.' for most cases), and then freeze the exemplars and optimize the instruction on both Gemini 1.0 Pro and Gemini 1.5 Flash models. We used APE to optimize the instructions and random search to optimize exemplars wiht a total budget of m = 32 , which is identical to the setup presented in Fig. 7 in the main text.

Interleaved . Instead of framing the exemplar and instruction optimization as a two-staged approach, a better approach in principle is to optimize them jointly. Here, we propose an interleaved approach with the intention of conditioning instructions to the best exemplars discovered so far and vice versa given their inter-dependence. The pseudocode representation of the algorithm in shown in Algorithm 1.

Algorithm 1 Interleaved instructions and exemplars optimization. Exemplar optimization steps marked in blue.

1: Inputs : D val , target LLM, optimizer LLM, # generations T , instruction pop size |I| , exemplar pop size |E |, number of exemplar shots: k .

2: Initialize instructions I 0 with seed instruction or LLM-sampled instructions; initialize exemplars E 0 ←∅ , best instruction & exemplar so far I ∗ ≤ 0 , E ∗ ≤ 0 ← None

3: for all generation t ∈ { 1 , ..., T } do

4: # Optimize instructions conditional on best exemplars

5: Evaluate prompt metrics on D val : ˜ s ← E ( x,y ) ∼D val [ g ( f LLM ( x, I ( i ) t , E ∗ ≤ t , ) , y )] ∀ I ( i ) t ∈ I t .

7: Update I ∗ ≤ t ← arg max I t ∈I t ˜ s ( I t | E ∗ ≤ t )

6: Retain ˆ I t , the top instructions in I t in terms of ˜ s and use the optimizer LLM to rewrite without changing the semantic meaning I t +1 ← resample( ˆ I t ) .

8: # Optimize exemplars conditional on best instructions

9: if E t = ∅ then

10: Fill E t +1 with input-output (inc. intermediate outputs) pairs randomly sampled from D c ( I ∗ ≤ t ) , the subset of D val where the target model answered correctly under the best instruction seen so far : ∀ E j ∈ E t +1 , E j ←{ e ( m ) } k , e j ∼ D c ( I ∗ )

j m =1 ≤ t

11: else 12: Fill E t +1 with |E| mutated copies of E ∗ ≤ t : randomly replace an input-output pair in E ∗ ≤ t with another pair from D c ( I ∗ ≤ t ) .

13: end if

16:

14: Evaluate prompt metrics on D val : ˜ s ← E ( x,y ) ∼D val [ g ( f LLM ( x, I ∗ ≤ t , E ( j ) t , ) , y )] ∀ E ( j ) t ∈ E t 15: Update E ∗ ≤ t ← arg max E t ∈E t ˜ s ( E t | I ∗ ≤ t )

end for

17: Return instruction-exemplar combination with the highest score.

We present the comparison between the original and inverted orders in Table 20, where we found that the findings (in particular, Insights 2 and 3 ) are not affected by the optimization order. For the interleaved approach, we find it to achieve a test accuracy of 74.6% on 26 BBH tasks on PaLM 2, which is broadly on par with the two-staged approach with m IO = 16 , m EO = 16 - we thus primarily focused on the two-staged approach for its simplicity, and defer a detailed exploration on better ways to exploit the interdependence of exemplars and instructions to future work.

B.9 Case Study on How LLMs Respond to Exemplars and Instructions

Complementary to Insight 2 and Fig. 8, in this section, we give some examples on the LLM response when prompted with the optimized instructions and/or exemplars in representative tasks.

Method

(Test acc.)

Base

(38%)

Best IO

(52%)

Best EO

(85%)

Prompt

Let's follow a step-by-step process.

Let's carefully analyze the question and dissect our reasoning process. In order to do this, we will need to track multiple pieces of information over time. We will also need to pay attention to the details of each swap that occurs.

Q: Alice, Bob, Claire, Dave, Eve, Fred, and Gertrude are dancers at a square dance. At the start of a song, they each have a partner: Alice is dancing with Ophelia, Bob is dancing with Melissa, Claire is dancing with Jamie, Dave is

dancing with Sam, Eve is dancing with Patrick, Fred is dancing with Rodrigo, and Gertrude is dancing with Karl.

Throughout the song, the dancers often trade partners. First, Dave and Claire switch partners. Then, Alice and Eve switch partners. ... Finally, Dave and Alice switch partners. At the end of the dance, Fred is dancing with

Options: (A) Ophelia ... (G) Karl

Show your final answer option bracketed between <answer> and </answer> at the end.

A: Let's follow a step-by-step process. Let's follow the steps one by one:

1. Dave and Claire switch partners.

Dave was dancing with Sam, and Claire was dancing with Jamie. So now, Dave is dancing with Jamie, and Claire is dancing with Sam. ...

7. Dave and Alice switch partners.

Dave was dancing with Karl, and Alice was dancing with Patrick. So now, Dave is dancing with Patrick, and Alice is dancing with Karl.

At the end of the dance, Fred is dancing with Melissa.

<answer>(B)</answer>

Method

(Test acc.)

Base

(55%)

Best IO

(82%)

Best EO

(97%)

Prompt

Let's think step by step.

Let's think step by step. In each step, we will look at the statement made by each person and determine whether they are telling the truth or not. If a person is telling the truth, we will continue to the next step. If a person is lying,

we will stop and conclude that the person who said the previous statement is also lying.

Question: Maybelle tells the truth. Antwan says Maybelle tells the truth. Audrie says Antwan tells the truth. Ryan says Audrie lies. Delfina says Ryan lies. Does Delfina tell the truth? Show your final answer (Yes or No only)

bracketed between <answer> and </answer>.

A: Let's think step by step. Let's think step by step.

Maybelle tells the truth.

Antwan says Maybelle tells the truth. So Antwan tells the truth.

Audrie says Antwan tells the truth. So Audrie tells the truth.

Ryan says Audrie lies. So Ryan lies.

Delfina says Ryan lies. So Delfina tells the truth.

<answer>Yes</answer>

B.10 Comparison Against PromptBreeder

In this section, we compare the two-stage EO-IO combined search mentioned in §3.2 against PromptBreeder [14], the SoTA method that has the option of optimizing both instructions and exemplars. In contrast to our method which is a simple combination of APE (for instruction optimization) and mutation (for exemplar optimization), PromptBreeder features a much expansive search space introduced by its different mutation operators on the instruction and the meta-prompts (referred to as hyper-mutation in Fernando et al. [14]). As such, the algorithm requires many more iterations before convergence - for our experiment, we use a population size of 40 and allow for 10 generations. Since PromptBreeder uses binary tournament evolution for each generation which will result in half of its population being replaced with new prompts requiring evaluations D val , there are total 40 + 10 × (40 / 2) = 240 traversals on the validation set D val , which is approximately an order-of-magnitude larger than the budget used in this paper and the discovered prompts can be

deemed as an approximation of the performance 'upper bound' when a SoTA optimizing agent is left to freely explore an expansive search space with ample search budget. As a comparison, we run our two-staged EO and IO algorithm for 100 iterations, with first 25 iterations used for IO and the remaining 75 for EO. Due to the much larger computational costs, we run the comparison on four selected tasks: movie_recommendation , multistep_arithmetic_two , object_counting and ruin_names .

0

100

200

25

50

75

Validation acc. (%)

movie_recommendation

0

100

200

Number of evaluations on

val

0

25

50

75

multistep_arithmetic_two

0

100

200

50

100

object_counting

0

100

200

25

50

75

ruin_names

PromptBreeder

Ours

We summarize the results in Table 23 and Fig. 20. It is worth noting that our simple two-stage search performs on par or better than PromptBreeder in 3/4 tasks in terms of both validation and test metrics while using less than half of the budget and using a much simpler instruction optimization heuristic. Given that the chief difference between our algorithm and PromptBreeder is the explicit iterative optimization for exemplars (noting that the 'context shuffling' routine used in PromptBreeder, which permutes the exemplars in the prompt, relies purely on stochasticity), this again suggests that the importance of exemplar optimization may actually outweigh complicated instruction optimization as we highlighted in Insight 2 in §3.2 - in fact, even the the PromptBreeder authors themselves remarked in App. J that ' ... We find that in the few-shot evolution case, the contexts dominate, and often the task-prompts drift into nonsense. They are less critically determining of fitness than the evolved contexts. '.

B.11 Additional Examples of How SoTA IO Uses Exemplars

Complementary to Insight 4 in §3.2 of the main text, in this section, we show additional examples illustrating how SoTA IO techniques may spontaneously and opportunistically use quasi-exemplars. For selected examples, we also perform ablation experiments similar to the experiment in Fig. 8 where we either retain or remove the identified quasi-exemplars to study the effect of their presence on the final performance and we show the results in Table 24 - it is further worth noting that in multiple cases, the instruction optimizers discovered instructions that consist of quasi-exemplars only (e.g., multi_arithmetic_two and movie_recommendation in Table 24), which further corroborates our point made in Insight 4 .

C Best Prompts Discovered

In this section, we present the best optimized prompts discovered by any method on each of the BBH tasks studied.

Listing 4: disambiguation_qa : No IO + Mutation. Test accuracy = 71.5%

For each given argument , classify it as deductively valid or invalid based on the explicitly stated premises. A deductively valid argument is one where the conclusion logically follows from the premises. If an argument is deductively invalid , even when the premises are true , the conclusion can still be false.

Be cautious , some arguments may have premises that are nonsensical or contradictory. In such cases , simply focus on determining whether or not the conclusion is supported by the premises , regardless of their content.

Read each argument and provided premises carefully and attentively. If the argument can be demonstrated to be invalid based on the premises , respond with "invalid ," otherwise , answer "valid."

"Is Fred a cousin of Robert? Is Joe related to Bob? In large families , it is sometimes difficult to keep track of all one's relatives. The following argument seeks to clarify some such relations: First of all, every great -grandfather of Chad is an ancestor of Douglas or a cousin of Henry. Next , every schoolmate of Trevor is neither a cousin of Henry nor an ancestor of Douglas. Hence , no great -grandfather of Chad is a schoolmate of Trevor."

Is the argument , given the explicitly stated premises , deductively valid or invalid? Options:

-valid

-invalid

Show your final answer (valid or invalid only) bracketed between <answer > and </answer >. <answer >valid </answer >

==

Be cautious , some arguments may have premises that are nonsensical or contradictory. In such cases , simply focus on determining whether or not the conclusion is supported by the premises , regardless of their content.

Read each argument and provided premises carefully and attentively. If the argument can be demonstrated to be invalid based on the premises , respond with "invalid ," otherwise , answer "valid."

"Here comes a perfectly valid argument: First of all, every loyal buyer of Tocca soap is an occasional purchaser of Bentley Organic soap. Next , being a loyal buyer of Tocca soap is sufficient for being a frequent consumer of L'Oreal shampoo. Plus ,some regular user of Lever soap is not an occasional purchaser of Bentley Organic soap or not a frequent consumer of L'Oreal shampoo. We may conclude that not every regular user of Lever soap is a loyal buyer of Tocca soap."

Is the argument , given the explicitly stated premises , deductively valid or invalid? Options:

-valid

-invalid

Show your final answer (valid or invalid only) bracketed between <answer > and </answer >. <answer >valid </answer >

==

Be cautious , some arguments may have premises that are nonsensical or contradictory. In such cases , simply focus on determining whether or not the conclusion is supported by the premises , regardless of their content.

Read each argument and provided premises carefully and attentively. If the argument can be demonstrated to be invalid based on the premises , respond with "invalid ," otherwise , answer "valid."

(J) triangle

Listing 6: geometric_shapes : APE + Random Search. Test accuracy = 88.5%

Listing 7: hyperbaton : PromptAgent + Random Search. Test accuracy = 90.5%

Listing 8: logical_deduction_five_objects : OPRO + Random Search. Test accuracy = 62.5%

Listing 9: logical_deduction_seven_objects : OPRO + Mutation. Test accuracy = 62.5%

Listing 10: logical_deduction_three_objects : No IO + Random Search. Test accuracy = 84.5%

Listing 11: movie_recommendation : PromptAgent + Mutation. Test accuracy = 95.0%

Listing 12: multistep_arithmetic_two : ProTeGi + Mutation. Test accuracy = 82.5%

Listing 13: navigate : ProTeGi + Random Search. Test accuracy = 68.5%

Listing 16: reasoning_about_colored_objects : APE + Diversity. Test accuracy = 80.5%

Q: The following translations from German to English contain a particular error. That error will be one of the following types: Named Entities: An entity (names , places , locations , etc.) is changed to a different entity. Numerical Values: Numerical values (ordinals or cardinals), dates , and/or units are changed. Modifiers or Adjectives: The modifiers and adjectives pertaining to a noun are changed. Negation or Antonyms: Introduce or remove a negation or change comparatives to their antonyms . Facts: Trivial factual errors not pertaining to the above classes are introduced in the translations. Dropped Content: A significant clause in the translation is removed. Please identify that error. Source: Boyd Kevin Rutherford ist ein US-

amerikanischer Politiker. Translation: Boyd is an American politician. The translation contains an error pertaining to Options: (A) Modifiers or Adjectives (B) Numerical Values (C) Negation or Antonyms (D) Named Entities (E) Dropped Content (F) Facts Show your final answer option bracketed between <answer > and </answer > at the end. A: Let's think step by step. The error in the translation is that the name "Boyd Kevin Rutherford" is reduced to just "Boyd". The correct translation should be: Boyd Kevin Rutherford is an American politician. Therefore , the error pertains to (D) Named Entities. The final answer is: <answer >(D)</answer > == Q: The following translations from German to English contain a particular error. That error will be one of the following types: Named Entities: An entity (names , places , locations , etc.) is changed to a different entity. Numerical Values: Numerical values (ordinals or cardinals), dates , and/or units are changed. Modifiers or Adjectives: The modifiers and adjectives pertaining to a noun are changed. Negation or Antonyms: Introduce or remove a negation or change comparatives to their antonyms . Facts: Trivial factual errors not pertaining to the above classes are introduced in the translations. Dropped Content: A significant clause in the translation is removed. Please identify that error. Source: Dunau , laut Ortstafel Dunau -Lathwehren und auf Landkarten auch als Lathwehren -Dunau bezeichnet , ist eine Ortslage des Stadtteils Lathwehren der Stadt Seelze in der Region Hannover in Niedersachsen. Translation: Dunau , according to the local plaque Dunau -Lathwehren , is a locality of the district of Lathwehren in the city of Seelze in the Hanover region of Lower Saxony. The translation contains an error pertaining to Options: (A) Modifiers or Adjectives (B) Numerical Values (C) Negation or Antonyms (D) Named Entities (E) Dropped Content (F) Facts Show your final answer option bracketed between <answer > and </answer > at the end. A: Let's think step by step. The original German sentence mentions "laut Ortstafel Dunau Lathwehren und auf Landkarten auch als Lathwehren -Dunau bezeichnet". This part is translated as "according to the local plaque Dunau -Lathwehren". The error here is that the translation does not mention "auf Landkarten auch als Lathwehren -Dunau bezeichnet", which means "also referred to as Lathwehren -Dunau on maps". The correct translation should be: Dunau , according to the local plaque Dunau -Lathwehren and also referred to as Lathwehren Dunau on maps , is a locality of the district of Lathwehren in the city of Seelze in the Hanover region of Lower Saxony.

Listing 18: salient_translation_error_detection : APE + Random Search. Test accuracy = 62.5%

Let's approach this task in a systematic manner. First , let's identify all the different objects that are being swapped in the context. Then , let's trace the swaps and keep track of the objects as they change hands. Finally , let's use our understanding of the swaps and the initial positions of the objects to answer the question. We also need to make sure that all the information necessary to answer the question is

contained in the context and check that the number of players is the same as the number of objects.

Alice , Bob, Claire , Dave , and Eve are holding a white elephant gift exchange. At the start of the event , they are each holding a present of a different color: Alice has a purple present , Bob has a blue present , Claire has a brown present , Dave has a yellow present , and Eve has a black ball.

As the event progresses , pairs of people swap gifts. First , Alice and Bob swap their gifts. Then , Bob and Dave swap their gifts. Then , Claire and Bob swap their gifts. Then , Dave and Alice swap their gifts. Finally , Dave and Eve swap their gifts. At the end of the event , Dave has the

Options:

(A) purple present

(B) blue present

(C) brown present

(D) yellow present

(E) black ball

Show your final answer option bracketed between <answer > and </answer > at the end. Let's trace the swaps one by one:

Alice and Bob swap their gifts: Alice gives her purple present to Bob, and Bob gives his blue present to Alice. So now Alice has the blue present , and Bob has the purple present.

Bob and Dave swap their gifts: Bob gives his purple present to Dave , and Dave gives his yellow present to Bob. So now Bob has the yellow present , and Dave has the purple present.

Claire and Bob swap their gifts: Claire gives her brown present to Bob, and Bob gives his yellow present to Claire. So now Bob has the brown present , and Claire has the yellow present.

Dave and Alice swap their gifts: Dave gives his purple present to Alice , and Alice gives her blue present to Dave. So now Alice has the purple present again , and Dave has the blue present again.

Dave and Eve swap their gifts: Dave gives his blue present to Eve, and Eve gives her black ball to Dave. So now Dave has the black ball , and Eve has the blue present.

At the end of the swaps , Dave has the black ball.

Therefore , the answer is: <answer >(E)</answer > ==

Alice , Bob, Claire , Dave , and Eve are dancers at a square dance. At the start of a song , they each have a partner: Alice is dancing with Rodrigo , Bob is dancing with Sam, Claire is dancing with Izzi , Dave is dancing with Jamie , and Eve is dancing with Ophelia.

Throughout the song , the dancers often trade partners. First , Bob and Eve switch partners . Then , Alice and Bob switch partners. Then , Eve and Bob switch partners. Then , Dave and Alice switch partners. Finally , Eve and Claire switch partners. At the end of the dance , Alice is dancing with

Options:

(A) Rodrigo

(B) Sam

(C) Izzi

(D) Jamie

(E) Ophelia

Show your final answer option bracketed between <answer > and </answer > at the end. Let's identify the dancers and their initial partners:

-Alice is dancing with Rodrigo.

-Bob is dancing with Sam.

-Claire is dancing with Izzi.

-Dave is dancing with Jamie.

-Eve is dancing with Ophelia.

Now, let's trace the swaps:

-Bob and Eve switch partners: Bob is now dancing with Ophelia , and Eve is dancing with Sam.

-Alice and Bob switch partners: Alice is now dancing with Sam, and Bob is dancing with Rodrigo.

-Eve and Bob switch partners: Eve is now dancing with Rodrigo , and Bob is dancing with Sam again.

-Dave and Alice switch partners: Dave is now dancing with Alice , and Alice is dancing with Jamie.

-Eve and Claire switch partners: Eve is now dancing with Izzi , and Claire is dancing with Rodrigo.

At the end of the dance , Alice is dancing with Jamie.

Listing 22: tracking_shuffled_objects_five_objects : ProTeGi + Random Search. Test accuracy = 70.0%

Dave was dancing with Karl , and Alice was dancing with Patrick. So now, Dave is dancing with Patrick , and Alice is dancing with Karl.

Listing 23: tracking_shuffled_objects_seven_objects : APE + Mutation. Test accuracy = 86.0%

We will methodically address this by breaking it down into manageable portions. We will monitor the final status of every entity (e.g., Alice , Bob, and Claire) after the

transactions noted in the problem. We will also monitor the flow of the transactions . Alice , Bob, and Claire are dancers at a square dance. At the start of a song , they each have a partner: Alice is dancing with Helga , Bob is dancing with Patrick , and Claire is dancing with Sam. Throughout the song , the dancers often trade partners. First , Claire and Alice switch partners. Then , Claire and Bob switch partners. Finally , Alice and Bob switch partners. At the end of the dance , Bob is dancing with Options: (A) Helga (B) Patrick (C) Sam Show your final answer option bracketed between <answer > and </answer > at the end. Let's track the partners of Alice , Bob, and Claire throughout the dance: -Initially , Alice is dancing with Helga , Bob is dancing with Patrick , and Claire is dancing with Sam. -After the first switch , Claire and Alice switch partners , so Alice is now dancing with Sam, and Claire is dancing with Helga. -After the second switch , Claire and Bob switch partners , so Claire is now dancing with Bob, and Bob is dancing with Helga. -After the third switch , Alice and Bob switch partners , so Alice is now dancing with Bob , and Bob is dancing with Sam. Therefore , at the end of the dance , Bob is dancing with Sam. <answer >(C)</answer > == Alice , Bob, and Claire are dancers at a square dance. At the start of a song , they each have a partner: Alice is dancing with Lola , Bob is dancing with Patrick , and Claire is dancing with Melissa. Throughout the song , the dancers often trade partners. First , Alice and Claire switch partners. Then , Bob and Claire switch partners. Finally , Claire and Alice switch partners. At the end of the dance , Alice is dancing with Options: (A) Lola (B) Patrick (C) Melissa Show your final answer option bracketed between <answer > and </answer > at the end. Let's track the partners of Alice , Bob, and Claire throughout the dance: -Initially , Alice is dancing with Lola , Bob is dancing with Patrick , and Claire is dancing with Melissa. -After the first switch , Alice and Claire switch partners , so Alice is now dancing with Melissa , and Claire is dancing with Lola. -After the second switch , Bob and Claire switch partners , so Bob is now dancing with Melissa , and Claire is dancing with Patrick. -After the third switch , Claire and Alice switch partners again , so Claire is now dancing with Melissa , and Alice is dancing with Patrick. At the end of the dance , Alice is dancing with Patrick. is:

Listing 25: web_of_lies : APE + Random Search. Test accuracy = 99.5%

Peer reviews:
{"IS_META_REVIEW": false, "comments": "SUMMARY:\nThis paper aims to study the performance impact of instruction optimization (IO) and exemplar selection (ES) both isolation and combination, as the automatic prompt optimization literature mostly only focuses on either IO or ES. Its empirical results on a wide range of tasks (e.g., BBH and MMLU) show that optimizing ES may bring more performance gain than IO, and highlight the importance of studying ES and its combination with IO.\n\nSTRENGTHS:\n- Compared with previous works on APO, this work provides relatively comprehensive experiments to show combining IO with ES can further boost the APO performance.\n- This work examines quite a number of combinations of different IO and ES approaches.\n\nWEAKNESSES:\n1. The paper only considers PaLM 2 and Gemini Pro (which are from the same source) as the target models. It is meaningful to see if the conclusion (i.e., ES outweighs IO) still holds for different target models (e.g., ChatGPT/GPT-4).\n\n2. Some important settings are not explained. For example, in line 199, why ES is only performed from the exemplars self-generated by the target model, instead of choosing from $\\mathcal{D}_{val}$. For $\\mathcal{D}_c(I^*)$, only selecting the correctly predicted samples may not bring the highest performance, as some works show that wrong exemplars in ICL may be more valuable. In many IO works, they typically use more than 100 query evaluation budgets, instead of choosing $m=32$ naively in this paper.\n\n3. The authors have the following objective in their paper:\n>We aim to meet this by evaluating ES and IO in the context of APO by answering the following ... 2) How do we make the **optimal use** of the limited data and computational budget under the current APO framework?\n\nBecause of the synergy between ES and IO, the optimal use of ES and IO should exist at the *joint optimization* of ES and IO. However, the authors have only shown the performances of combining independently optimized instruction and optimized exemplars conditioned on that instruction. The optimality should be found when the exemplars and instructions are optimized together and the corresponding combinatorial optimization problem should cater to the space $\\mathcal{I}\\times\\mathcal{E}^k$ rigorously. Intuitively, the optimized instruction should also depend on the given exemplars at each iteration. So, if the authors want to completely study how the effect of combining IO and ES, they should consider this joint optimization problem. Unfortunately, to the best extent, this paper only empirically shows (through brute force) that there may exist an optimal allocation of budgets $m_{IO}$ and $m_{ES}$ used for running the 'mixing-and-matching' strategy of ES and IO, which is far from the so-called **optimal use**.\n\nQUESTIONS:\n1. How did you choose the seed instruction? Do different seed instructions have different outcomes?\n\n2. In Table 3, why ProTeGi is worse than (NO IO) if its optimization starts from the same seed instruction?\n\n3. If you revert the process, that means you start ES first and run IO conditioned on the best ES you found, will this make the $\\Delta IO$ greater than  $\\Delta ES$?\n\nLIMITATIONS:\nYes", "TITLE": "Official Review", "RECOMMENDATION": 5, "REVIEWER_CONFIDENCE": 4, "SOUNDNESS_CORRECTNESS": 2, "DATE": "2024-07-09"}

{"IS_META_REVIEW": false, "comments": "SUMMARY:\nThe paper studies the instruction optimization and exemplar selection in auto prompt optimization. It has the following contributions:\n1) Showing that optimizing ES improves significantly over IO and the importance of ES\n2) Showing the synergy effect between ES and IO\n3) Extensive experiments are conducted to validate the conclusions\n\nSTRENGTHS:\n1. This paper is well-written with good presentation\n2. The formulation of ES and IO as optimization is clear\n3. The experiments on ES and IO and a mix-matching of these two are clear and extensive\n4. The idea of comparing ES and IO is novel to me\n\nWEAKNESSES:\n1. The authors have made a significant effort in summarizing the existing works, however, there are still some of the SOTA IO methods not mentioned:\n- Connecting Large Language Models with Evolutionary Algorithms Yields Powerful Prompt Optimizers: https://arxiv.org/abs/2309.08532\n- Use Your INSTINCT: INSTruction optimization for LLMs usIng Neural bandits Coupled with Transformers: https://arxiv.org/abs/2310.02905\n2. Some of the baselines are mentioned in related work, however, the author did not compare with them (e.g., InstructZero). This can be important since these methods are SOTA methods of IO and may potentially produce much better results than the methods compared in this paper. It is true that the authors use simple methods for ES, however, do note that exemplars are natural data points (good data) while instructions are usually generated differently by different methods and hence a more careful design is needed. \n3. The no IO is based on a very good instruction “let’s think step by step” while no ES is just no examplar at all. Therefore for fair comparison /drawing insights from the experiments, the improvement on ES needs to be compared to random (or at least nearest) to say which optimization is more important.\n4. As the author mentioned in mix-matching section, the improvement of the R.S., Mutation in Table 1, 2 can be because of the extra computation/queires to LLM used in the search phase. How much does the delta in these tables are from this factor? And how would this affect the conclusion of comparing the importance of ES and IO? These are unclear from the current experiments.\n\nQUESTIONS:\n1. The experiments are all conducted on PaLM and Gemini which are from the same company and hence some similar underlying designs. Can the results generalize to other major LLMs, e.g., GPT-4, Claude, Llama.\n2. How does the author implement Nearest? Is it finding the nearest for each test input? or for a whole validation dataset?\n3. ES after IO is used to run the result in Table 1 and 2, have the authors tried IO after ES, will the result change?\n\nLIMITATIONS:\nThe limitations are stated in the weakness. It mainly lies in the mismatch between experimental settings and the insights.", "TITLE": "Official Review", "RECOMMENDATION": 6, "REVIEWER_CONFIDENCE": 5, "SOUNDNESS_CORRECTNESS": 2, "DATE": "2024-07-11"}

{"IS_META_REVIEW": false, "comments": "SUMMARY:\nThis paper provides a comprehensive investigation of instruction optimization and exemplar selection.\n\nSTRENGTHS:\n- The paper is very well motivated, and studies a very important problem, which is the relative importance of exemplar selection and instruction optimization. Therefore, the paper is likely to inspire ideas for future research in these ares.\n- The paper is well written and well organized. I particularly like the way some results are presented, such as Table 1 and Figure 6. These results are very nicely organized and clearly presented, effectively conveying the intended insights.\n- A particularly nice feature of the proposed algorithm is that it only uses the single validation set for both IO and ES.\n\nWEAKNESSES:\n- The performances achieved by the different combinations of algorithms are indeed good, however, in terms of the relative comparison of the importance of IO and ES, I think it is possible that an unfair advantage is given to ES. First of all, in all methods studied in this paper, IO is performed first followed by ES, this allows ES to adapt to an already optimized IO, while IO is not able to adapt to exemplars since it is performed first before any ES. In addition, only those exemplars on which the optimized instruction predicts correctly are used for ES, this also gives an advantage to ES since it uses information from an already optimized instruction, whereas IO is not able to exploit information from already optimized exemplars since IO is performed first. Therefore, it puts into question the reliability of a major conclusion drawn from the paper, i.e., ES is much more important than IO.\n- As mentioned in the last paragraph of Section 2, to the best of my knowledge, there is indeed a general lack of works on joint IO and ES. I'd like to point the authors to two concurrent works along this line: https://arxiv.org/abs/2405.18369, https://arxiv.org/abs/2405.16122. The second work may be of more interest, because it is exactly using something similar to the combinatorial BO approach which is alluded to on lines 295-296.\n\nQUESTIONS:\n- The total number of prompt evaluations is selected as $m=32$. In my impression, this is in general a small number compared to the other related works on IO or ES. For example, OPRO uses more than 100 evaluations in most of its experiments. I wonder will the conclusions change if a larger $m$ is used?\n- It has been widely reported that the ordering of the exemplars matters a lot. I wonder how do you determine the ordering?\n\nLIMITATIONS:\nA limitation is discussed in the Conclusion section.", "TITLE": "Official Review", "RECOMMENDATION": 5, "REVIEWER_CONFIDENCE": 3, "SOUNDNESS_CORRECTNESS": 2, "DATE": "2024-07-13"}

{"IS_META_REVIEW": false, "comments": "SUMMARY:\nThis paper profoundly discusses how the deisgn of Automatic Prompt Optimization (APO) methods influence the performance of current instruction-following pre-trained large language models (LLMs). Concretely, the authors first systematically summarize and compartmentalize current APO methods into two lines: instruction optimziation (IO) and exempler sleection (ES), with their mathematical definitions. Then a series thorough and incremental experiments are carried out, of which primary discoveries includes: 1) reusing model-generated data (obtained from validation phase) as exempler shows performance improvement against standalone IO methods, with better efficiency. 2) combining IO and ES together would bring \"1+1>2\" performance improvement. Under these experimental results, the authors claim that studying ES methods should be put as top priority of APO studies.\n\nSTRENGTHS:\n1. This paper is very well-writen. The related works are appropriately summarized. The mathematical definition of APO, IO and ES are crystal clear and hence helps identify the difference between IO and ES. The experimental results are carefully arranged and presented.   \n2. The findings, though obtained from limited tasks and models, are still well supported by the current experimental results. It is very meaningful for the APO community, and deserves more attention and further investigation and development.\n\nWEAKNESSES:\nAfter reading this paper from top to the bottom, I found my concerns revolve around limited evaluation: I would appreciate the authors if they could at least conduct an additional experiment on GPT-3.5 Turbo to reinforce the convincingness of all findings, since GPT models hold quite some users. To summarize, although I fully understand that ideally comprehensive evaluation experiment is never easy, the above aspects deserve to be investigated, at least. Besides, the contribution of this paper mainly involves benchmarking and concluding meaningful findings, which in my view, should be more suitable for Benchmark Track of NeurIPS.\n\nQUESTIONS:\nsee Weaknesses.\n\nLIMITATIONS:\nThe authors have summarized the limitations of this work in Conclusion of the paper, which are objective. I strongly suggest the authors include more datasets (e.g., multimodal tasks) and models (e.g., GPT series) in their future works as they promised.", "TITLE": "Official Review", "RECOMMENDATION": 5, "REVIEWER_CONFIDENCE": 3, "SOUNDNESS_CORRECTNESS": 3, "DATE": "2024-07-18"}

{"IS_META_REVIEW": false, "comments": "SUMMARY:\nThis work focus on evalution and comparison study of automatic prompt optimization (APO) methods. These methods are broadly categorized into instruction optimization (IO) and exemplar selection (ES). This paper seeks to bridge the gap between these two methods by comprehensively comparing the performance of representative IO and ES techniques both isolation and combination. This work found that how to select examplars can outweigh how to optimize instructions.\n\nSTRENGTHS:\n1. The presentation of this work is good. From the textual part, the authors clearly present the definition and representative methods of IO and ES. From those graphical presentation, they clearly present the comparison results of different methods.\n2. The setting of experimental study is convincing. The sequence from 'initial instruction + No exemplars' -> 'Optimized instruction + No examplars' -> Optimized instruction + Random exemplars' -> 'Initial instruction + Optimized exemplars' -> 'Optimized instruction + Optimized exemplars' make the comparison results clear and convicing.\n3. The finding of this work is insightful, this work found that ES is more important than IO which a bit do not fit in with the current direction of the study. So, this work may help to provide a reference for future research.\n\nWEAKNESSES:\n1. In the experimental study part, this work only use two kinds of LLMs, PaLM 2 and Gemini, which is not quite adequate.\n2. The evaluation criteria are a bit simple, only considering the acc rate. For engneering requirements, how many efforts (e.g. number of tokens; expense of the APO process; how much level of ) will be required during the prompt optimization is also important. Other reasonable metrics maybe also of help.\n\nQUESTIONS:\n1. What's the difference between methods from IO and ES in terms of the cost. Are there any differences in statistical characteristics between these two categories?\n\nLIMITATIONS:\nSee weakness and question part.", "TITLE": "Official Review", "RECOMMENDATION": 7, "REVIEWER_CONFIDENCE": 3, "SOUNDNESS_CORRECTNESS": 4, "DATE": "2024-07-22"}

{"IS_META_REVIEW": false, "comments": "SUMMARY:\nThis work studies automatic prompt optimization, seeking to compare and to connect the literatures on instruction optimization and on example selection. They find that existing approaches for using LMs to self-generate examples and selecting between them can outperform the best existing instruction optimizers, but that combining the two is effective. They use this to recommend larger emphasis on few-shot examples in future prompt optimization work.\n\nSTRENGTHS:\n1. The work asks a timely question that is formulated well, i.e. about the \"relative importance and performance impact of ES and IO, both in isolation and when combined together\". This challenges overemphasis on instruction optimization (IO) in plenty of recent work.\n\n2. In the experimental evaluation, the authors consider a wide and representative set of four Instruction Optimization methods from APE to OPRO, and take substantial steps to equalize their budgets for comparison.\n\n3. The authors do an excellent job at communicating their findings from a large number of comparisons in the form of several concrete and overall well-argued/supported Insights.\n\nWEAKNESSES:\n1. The authors evaluate on BIG-Bench and MMLU tasks, which to my knowledge are extremely limited in scope. Most are narrow reasoning puzzles or manipulation tasks, e.g. multiple choice, and may not reflect the type or complexity of typical prompts people use in the increasingly elaborate open-ended LM systems out there. This has an effect on the extent of \"self-generated\" examples, which in this work are reduced to just chain of thought reasoning if I understand correctly, where all \"inputs\" and \"labels\" (in this case final answers) are known in advance, rather than trajectories of actual input/output labels for prompts per example. How would such open-endedness interact with the findings?\n\n2. The authors already find very compelling results with ES, but the scope of existing example-based optimizers in the cited work like DSPy seem wider in terms of the search/optimization method and in the generation approach. For the latter, the authors employ a large LM to optimize the instructions for smaller LMs, but related work permits similarly a large LM (or the same LM but in already example-optimized form) to generate the examples for small LMs. As a result of this asymmetry, the effect of the source of instruction or example optimization is not explored or isolated. This seems unlikely to affect the results direction, but it may affect their magnitude substantially, perhaps making IO+ES less attractive than just ES.\n\nQUESTIONS:\n1. This is not a weakness of the present paper, but in the manner described here APE and OPRO are presented rather similarly in Sec 3.1. Overall, this raises the question of how the authors ensured their implementations are faithful to the original algorithms while being fair to all methods within a new test environment.\n\n2. I appreciate the note in lines 199-201 about self-generation. The way this is presented suggests the authors thing it's important, but it's rather buried now and hard to concretely understand without an example or further elaboration, especially because the tasks considered in this work are rather \"simple\", e.g. not clearly multi-step. In addition, the term Example Selection (ES), which is not particularly universal in the literature, is perhaps setting this up for confusion, since selection directly assumes the presence of a set from which to select.\n\nLIMITATIONS:\nn/a", "TITLE": "Official Review", "RECOMMENDATION": 7, "REVIEWER_CONFIDENCE": 4, "SOUNDNESS_CORRECTNESS": 3, "DATE": "2024-07-30"}