Evoke: Evoking Critical Thinking Abilities in LLMs via Reviewer-Author Prompt Editing
Authors: Xinyu Hu
Pengfei Tang
Simiao Zuo
Zihan Wang
Bowen Song
Qiang Lou
Jian Jiao
Denis X Charles
Venue: ICLR 2024 poster (2024)
Keywords: large language models
automatic prompt engineering
in-context learning
Primary area: generative models
TL;DR: We propose Evoke, an automatic prompt refinement framework to improve the effectiveness of large language models.

Abstract: Large language models (LLMs) have made impressive progress in natural language processing. These models rely on proper human instructions (or prompts) to generate suitable responses. However, the potential of LLMs are not fully harnessed by commonly-used prompting methods: many human-in-the-loop algorithms employ ad-hoc procedures for prompt selection; while auto prompt generation approaches are essentially searching all possible prompts randomly and inefficiently. We propose Evoke, an automatic prompt refinement framework. In Evoke, there are two instances of a same LLM: one as a reviewer (LLM-Reviewer), it scores the current prompt; the other as an author (LLM-Author), it edits the prompt by considering the edit history and the reviewer's feedback. Such an author-reviewer feedback loop ensures that the prompt is refined in each iteration. We further aggregate a data selection approach to Evoke, where only the hard samples are exposed to the LLM. The hard samples are more important because the LLM can develop deeper understanding of the tasks out of them, while the model may already know how to solve the easier cases. Experimental results show that Evoke significantly outperforms existing methods. For instance, in the challenging task of logical fallacy detection, Evoke scores above 80, while all other baseline methods struggle to reach 20.

Published as a conference paper at ICLR 2024
EVOKE: EVOKING CRITICAL THINKING ABILITIES IN LLMs VIA REVIEWER-AUTHOR PROMPT EDITING
Xinyu Hu
Pengfei Tang
Simiao Zuo
Zihan Wang*
Bowen Song Qiang Lou
Jian Jiao Denis Charles
Microsoft University of Washington
University of Michigan
ABSTRACT
Large language models (LLMs) have made impressive progress in natural lan- guage processing. These models rely on proper human instructions (or prompts) generate suitable responses. However, the potential of LLMs are not fully har- nessed by commonly-used prompting methods: many buman-in-the-loop algo- rithms employ ad-hoc procedures for prompt selection; while auto prompt gen- eration approaches are essentially searching all possible prompts randomly and inefficiently. We propose Evoke, automatic prompt refinement framework. In Evoke, there are two instances of same LLM: one as reviewer (LLM- Reviewer), it scores the current prompt; the other as an author (LLM-Author), it edits the prompt by considering the edit history and the reviewer's feedback. Such an author-reviewer feedback loop ensures that the prompt is refined in each iteration. We further aggregate a data selection approach to Evoke, where only the hard samples are exposed to the LLM. The hard samples are more important be- cause the LLM can develop deeper understanding of the tasks out of them, while the model may already know how to solve the casier cases. Experimental results show that Evoke significantly outperforms existing methods. For instance, in the challenging task of logical fallacy detection, Evoke scores above 80, while all other baseline methods struggle to reach 20.
INTRODUCTION
Consider an intriguing trio that first glance seems unrelated: bumble bees, cell phones, and excit- ing news. At a superficial level, their commonality might note their plural forms; however, a more profound analysis reveals shared essence: they all "create buzz." This comparison sheds light on the depth and intricacy of human cognitive processes. At the heart of such processes is critical thinking, the ability to conceptualize, analyze, question, and evaluate ideas and beliefs. As we tran- sition to the domain of artificial intelligence, it is observed that large language models (LLMs) have remarkably evolved as general problem solvers, urging us to ponder:
Can LLMs think on their own?
In practice, we observe that existing prompting methods are inadequate in evoking the critical think- ing abilities of LLMs.. For example, in Figure T we show two prompts for solving common concept task. From Figure (left), we see that for the input trio "bumble bees, cell phones, and ex- citing news". the LLM outputs a superficial common concept "plural form" using the hand-crafted prompt. On the other hand, with the prompt generated by the proposed method, the LLM demon- strates much deeper understanding about the task i.e., generates the correct answer "can cause a buzz" (see Figure [ right). These results indicate that the quality of prompts are directly related to the performance of LLMs. In this work, we focus on prompting methods that enables LLMs to think their own.
The current prompting methodologies exhibit significant drawbacks. Many prompting methods are ad hoc because of their human-in-the loop development paradigm. In such process, given target task, we first draft an initial prompt. Then, we refine the prompt using techniques such as chain- of of-thought, shw- demonstrations, and coding-style problem descriptions (Wei et al. 2022c
"Corresponding author; xinyuhu microsooft.cmm

---

Published as a conference paper at ICLR 2024
Task: Common Concept
Sample imput: Tbubble beas, 'cell phones; 'cxeiting nows]
Candidates
Edred lustr.
commonality between the inputs. commonalit should be a meaningful characteristic, proporty,
Stepa:
Edited Edited Instr. N
Score Score
Supt: Score promptx
Get took
applice all imputs, just superficial or coincidental feature.
common characteristic For the given list objects.
For example, con used for repoiry a valid communnlity for l'sewing', wweechess 'sunger'), but Joois shills fotning broad vague, and contaln the letter e too trivial and inreevant.
Candidatos
Stepl:
Candidates Edited Enstr.
Instr:
Score
Taak Ace
Edit
Output: Plural form
Inst: Seoce
Aaok Acc
Edited lnstr N
Output: can cunse huzz:
LL.M-Auther (with Memoryy
L.MM-Rerlewer (with Memorg)
Figure I: Comparison between hand-crafted and Evoke prompts.
Figure 2: Simplified workflow of Evoke.
Gao et al. 2023) based the model's performance the target task. We note that in practice, hand-crafted prompt optimized for one task rarely translates to satisfactory performance another task (Zhang al. 2023). Therefore, each task becomes new expedition, with its set of trials, errors, and validations. Such an ad hoc human-in-the-loop development procedure introduces extensive human labor requirements, which significantly hinder the applicability of LLMs in real- world applications.
Existing works develop algorithms automatically generate prompts instead of relying on ad hoc human optimization (Shin et al.) [2200] Honovich et al.) [022] Zhou et al. 2022).. However, these methods often lack feedback loops, such that the refinement procedure essentially performs ran- dom search. For example, in each refinement iteration, Zhou et al. (022) simply rephrases the prompt into multiple candidates, and then select the candidate that yields the best performance as the refined prompt. Note that such procedure fails to learn from past successes and failures, such that refined prompt does not enrich the original prompt with additional context.
We propose Evoke, which addresses the aforementioned drawbacks by leveraging an author-reviewer paradigm. In this paradigm, there are two distinct purposes an LLM can serve: one instance as an author (LLM-Author) tasked with editing prompts, and another instance reviewer (LLM- Reviewer) tasked with evaluating the quality of the prompts generated by the LLM-Author. Each role is played independently by separate instances of the same LLM.
Critical thinking is not something you do once with an issue and then drop it. requires that we update knowledge new information comes in.
Daniel Levitin
The essence of this quote resonates with the feedback loop in the workflow of Evoke, as depicted in Figure The workflow comprises three steps: First, the LLM-Author edits prompts from previous iterations, taking into account the past edits and the feedback from the LLM-Reviewer. Second, the I.LM-Reviewer scores the revised prompts from the L.MM-Author, and the top-n candidates with the highest scores are selected for subsequent procedures. The LLM-Reviewer employs a memory module that stores history edits, prompts and task accuracy of history prompts. Finally, the task accuracy for each instruction is computed.
To further enhance the efficacy of Evoke, we propose data selection strategy. In this strategy, only the hard samples selected by selector are exposed to the LLM. The intuition is that the LLM can develop deeper understanding of the tasks out of the hard samples. while it already knows how to solve the easier cases. Through extensive experiments (see Figure TO in the experiments), we see that retaining the hard samples indeed improves efficacy of Evoke.
We conduct extensive experiments to demonstrate the effectiveness of Evoke. Specifically, eight tasks from the Instruction Induction (Honovich et al. 2022) dataset and the Big Bench Instruction Induction (Zhou et al.) 2022) dataset, we show that Evoke significantly outperforms existing auto- matic prompt engineering approaches. For example, on the challenging logical fallacy detection task, Evoke achieves score of over 80, while all the baseline methods struggle reach 20. We also show that Evoke can improve LLMs' robustness against adversarial attacks, and can also handle fine-grained named entity recognition tasks with exceptional performance. As example, Evoke
2

---

Published as a conference paper at ICLR 2024
achieves significant performance gain adversarially constructed dataset, indicating that the proposed method can improve robustness of LLMs. Additionally, we provide detailed analysis on the effectiveness of each component of Evoke.
RELATED WORK
Large Language Models Recently, LLMs have shown emergent abilities- capaailittes to perform tasks they weren't explicitly trained for (Wei et al.
2022a b: Bubeck al., 2023). This includes common sense question answering. code generation, and cross-domain problem solving, enriching their utility across unforeseen domains (Chen et al. 202 Sarsa et al. 2022 Thirunavukarasu al., 2023 Huang& & Chang [022] Du et al. 2023). Subsequently, adapting LLMs to specific problems has drawn attention, and several methods have been proposed: Reinforcement Learning from Hu- man Feedback (RLHF (uyang et al. 2022), efficient fine-tuning (Hu et al., 2022 Dettmers et al.) 2023). and prompt engineering [White et al. 2023), among others. Each method has its pros and cons. For instance, RLHF can significantly improve performance but may require extensive human annotations. Efficient fine-tuning, on the other hand, can be less resource-intensive but might fall short in achieving the desired level of task-specific optimization. Prompt engineering, while inno- vative, may require a well-crafted prompt to effectively guide the model towards accurate outputs.
In- mn-Context Learning and Prompt Engineering In In-Context Learning (ICL) refers to the ability of LLMS to learn a new task from a small set of examples presented within the context (the prompt) at inference time, without updating any parameters (Wei et al.
2022a). This paradigm has signif- icantly improved the capabilities of LLMs across various tasks. Many studies have explored the reasons behind such improvements, examining aspects like Bayesian optimization and the difficulty
of demonstrations (Xie et al. 2022, Min et al. 2022 Liu et al. 2022 Yoo et al. 2022).
Prompt engineering plays pivotal role in facilitating ICL. It entails the design of prompts that arm the LLM with the essential information needed to learn and adeptly perform the new task. Each prompt essentially sets the stage for the LLM, enclosing the task's requirements and guiding the model towards producing the desired output. By carefully crafting prompts, it is possible to leverage the inherent capability of LLMs, enabling them tackle wide range of tasks even with limited or no prior explicit training on those tasks. Recently, methods such as Chain-of- -hoought (CoT), Zero-CoT, Self-Consisteney, Program-Aided, and Few-Shot Prompting have been demonstrated to be effective (Wei et al. 2022c Kojima et al. 2022 Wang et al. 2022 Gao et al. 2023 McDonell 2021).
Reynolds &
Automatic Prompt Engineering The existing methodologies for automating discrete prompt op- timization have their roots in instruction induction, as discussed by [Honvvich et al. 2022 was discovered that LLMs can generate natural language instructions based on small number of input- output pair examples. Building on this, Zhou al. (2022) proposed a new algorithm for the auto- matic generation and selection of instructions for LLMs. The algorithm, named Automatic Prompt Engineer (APE), is capable generating prompts that achieve human-level performance across a diverse range of NLP tasks. Work has also been done automating prompt generation for specific domains like code generation, as discussed in Shrivastava et al. 2023
3 ITERATIVE REVIEWER- PROMPT EDITING
3.1 OVERVIEW
In Ævoke, the same L.M plays two different roles: an author (LLMM-Auhhr) that is in charge of editing and refine prompts, and reviewer (evieereee that is in charge of scoring the refined prompts. We use two different prompts for the author's and the reviewer's task.
LLt-ouuhor edits and generates new prompts based feedback from LLM-Reviewer. The prompt for LLthor consists of several components:
a Input for editing: Current task instruction to be refined and training data;
Instruction for editing: "We've ve provided pairs consisting of inputs, the teacher's 's correct answers, and the students' responses. Please review the incorrect responses from the slu-
3

---

Published as a conference paper at ICLR 2024
Algorithm 1: Evoke
Require: Training set: Initial prompt for the target task (i.e., the want to refine).
!/ Initian Pritiaalin aalizaiion
LLM-Sclector Initialize data scoring instruction.
LLM--uthor: Initialize prompt editing instruction.
LLM-Reviews Initialize prompt reviewing instruction. while S do
LLMM-Slecctor
Assign difficulty scores for each data point in the training set.
Select training subset based on the difficulty level.
// GLM-Author
LLM-Author generates multiple prompts based training data and own memory.
// MeM--Reieewrr
scores the quality of cach generated prompt from LLM-Author based on its own memory.
Select top-n prompts based on the generated scores from LLM- -eMeeeeieeer.
task accuracy all prompts.
// Memory update
Memory of LLM-Author appends (edits, scores).
Memory of
appends (edits, prompts, task accuracy).
Return:
The prompt with the highest task accuracy.
dents and summarize key points that could be adjusted in the instruction to enhance student accuracy. Highlight major edits and present the updated task instruction."
Memory: prior history (edits, scores).
LLM-Author refines the instructions (prompts for the given task) by utilizing the training data and a memory component. We note that the memory consists of all prior (edit, score) pairs, where the score comes from -ev-eweee This memory component enables LLM-Author execute increasingly effective edits, drawing upon feedback from previous edits.
LLM-Reviewer scores the quality of prompts generated by LLM-Author. The input prompt for LLM-Reviewer consists of several components:
a Input for scoring: problem description and current instruction from LLM-Author;
b Instruction for scoring: "Please rate the following instruction scale of to 10, where 10 represents the highest level of clarity in problem description, execution steps, and comprehensive explanation of the problem.".
Memory: prior (edits, instructions, task accuracy).
The instructions generated by Lut--uuthr are forwarded L.M--eeviwwer for evaluation. Based on the scores generated by LLM-Reeiewer, only subset of high-scoring candidates is selected to move on to the subsequent iteration. Through this iterative editing process between LLM-Author and Leveeweee LLM-Author can refine instructions in each iteration. Details of the algorithm can be found in Algorithm I
To illustrate the effectiveness of Evoke, first three edits from the Movic Recommendation task in Big Bench are presented in Figure To start with, the prompt contains the basic task instruction. Next, it extracts key factors considered in movie recommendation, such the genre of cach movic, the distance between the given movies and the movies the user has watched before, and the popularity of the movies. In the final step, well-explained example is presented with detailed explanation following aforementioned factors. In summary, Evoke successfully concludes the key components of movie recommendation, and curates a demonstration with detailed explanation.
3.2 DATA SELECTION VIA LEM-SELECTOR
In practice, we find that not all samples are equally important to model performance (see Figure I0). In particular, find that even without prompt refinement, the LIM already knows how solve
4

---

Published as a conference paper at ICLR 2024
Step
Step {Prompt from Step 1]
Step
(Prompi from Step 2]
To help you make good recommendation, consider the following factors:
Example: If the user watched The Godflther, The Godlather Part and Goodfellas, the options are The Departed, Scarface, The Irishman, and Casino, possible answer is:
Given user's interest in movies he watched previously: [watched] Now given four different movies A D. please recemmmend one that might be the most interest of the user. Options: {list)".
genre, theme,
movies.
The similarity difference
between the movies and the ones the
<Answer> AnswerrAAAAsweer -Answer>
user watched before.
The populurity, ralings, und reviews of the movies.
[detailed explonations)
Figure 3: Illustration of Prompt Editing for the first three steps in the Task of Movie Recommenda- tion within Big Bench.
some "easier" cases. Therefore, only use "hard" samples in each refinement iteration. Specif- ically, we assign a third role besides an author a reviewer to the LLM: a data selector. The LLM- Selector evaluates the difficulty level (on scale of 10) of each data point by assessing, based on the current task instruction, how challenging it to derive the correct answer from the input. The input prompt for LLM-Selector consists of several components:
a Input for evaluating difficulty level: current instruction and input-output pair;
Instruction for evaluating difficulty level: "As an experienced teacher with insight into the various levels of difficulty of exam questions, please rate the following question a scale of I to 10, considering factors such as conceptual understanding. application of knowledge, problem- skills, time required, clarity language, and accessibility, where denotes extremely easy and 10 denotes extremely difficult.".
Empirically, we can further improve effectiveness of Evoke by using such data selection strategy.
EXPERIMENTS
We conduct extensive experiments to demonstrate the effectiveness of Evoke. We show that for any given task, the prompts generated by Evoke include clear definitions and well-structured task execu- tion steps. Moreover, these prompts feature demonstrations accompanied by detailed explanations. In all experiments, we utilize the Azure OpenAl API service (GPT-4) for the involved L.Mss.
4.1
MAIN RESULTS
Datasets. We perform a comprehensive evaluation on eight tasks from Instruction Induction (Hon- ovich et al. 2022) and Big Bench Instruction Induction (BBII) (Zhou et al. 2022), including
orthography starts with: Extract the words starting with given letter from the input sentence.
common concept: Find common characteristic for the given objects. rhymes: Write word that rhymes with the input word.
movie recommendation: Recommend movies similar to the given list of movies.
logical fallacy detection: Detect informal and formal logical fallacies.
presuppositions as nli: Determine whether the first sentence entails or contradicts the second,
winowhy: Evaluate the reasoning in answering Winograd Schema Challenge questions.
epistemic reasoning: Determine whether one sentence entails the next.
These tasks covers wide range of natural language understanding, reasoning and inference tasks. For each task, we divide the dataset randomly into two sets, 60% of the data allocated for training (prompt refinement) and the remaining 40% is for testing (prompt evaluation).
5

---

Published as a conference paper at ICLR 2024
Baselines. We compare methods against two baselines: human curated prompts (Human) from Honovich et al. (2022); Suzgun et al. (2022) and automatic prompt engineer (APE) proposed in (Zhou et al. 2022p. APE first deduces imitial prompt from input-output pairs, and subsequently employs LLMs to refine and generate new prompt candidates. However, prompts are simply para- phrased during the refinement process of APE, which largely resembles random searching in the space of all possible prompts.
Main Results. Figure 4 demonstrates experimental results. We observe that Evoke outperforms all the baselines in all eight tasks. For example, on the challenging logical fallacy detection task from BBII, performance of Evoke more than 80, while performance of both APE and Human are below 20. This is because Evoke is adept at conceptualizing the core definition of a task, decomposing complex task into smaller subtasks, and curating relevant demonstrations accompanied by detailed explanations. To demonstrate the power of Evoke, we show the generated prompt for logical fallacy detection Table [ We see that the prompt begins with a clear task introduction and objective, followed by fine-grained definition of logical fallacy. It then articulates the criteria for evaluation and the task steps to follow. Lastly, it provides a list of common logical fallacies, each accompanied by detailed description. Additionally, well-structured prompt for epistemic reasoning is presented T
Instruction Induction
Big Bench Instruction Induction
Human
30
APE
80
Evoki (Durs)
70
(%g
50
40
40
30
90
20
10
artha starts
com concept
mynees
movie.c rec
logical fal
winosiy
reasoning
Figure 4: Results eight tasks from the Instruction Induction and the Big Bench Instruction Induc- tion datasets. report the average accuracy over three runs.
4.2 TOWARDS ADVERSARIAL ROBUSTNESS
Despite their superior performance, LLMs not ro- bust to adversarial attacks (Wang et al. 2023). For ex- ample, when asking GPT-4 whether "pretty" is pos- itive word, the model can output the correct answer. However, if ask whether "prettye"", clear typo of "pretty", is a positive word, the LLM outputs an oppo- site answer. We show that Evoke can generate prompts which alert the LLM in paying attention to potential typos, and thus can improve model robustness.
Human
APE
RocEnc
Evoke
100
90
Accuracy
80
70
We adopt two datasets: SST-2 (Socher et al. 2013) is sentiment classification task, where we need to decide whether movie review is positive or neg- ative; and QQP (Wang al., 2019) is a task where need to determine whether two sentences are para- phrases of each other.
Datasets.
SS12
Adversarial SST2
OOP Adversanial DQP
Results on clean and adver- sarially attacked SST2 and QQP datasets. We report the average accuracy over three runs. We note that RobEne is only applied to the attacked data.
Figure 5:
To evaluate whether Evoke can improve LLMs' robust- ness, we add typos to the datasets. Specifically, we per- form character-level adversarial attacks for each sam- ple. In the attack, we change at most one character each word, and we change at most 4 words in each sentence (Jones al. 2020 In this way, the constructed adversarial texts human-

---

Published as a conference paper at ICLR 2024
Table 1: Prompt generated by Evoke for logical fallacy detection.
Prompt from Evoke
Breakdown Introduction and objective
In this task, you are required to identify both informal and formal logical falla- cies in the provided input statements. Your response should be a binary value: return if the query is logically valid (i.e., free from any logical fallacies), and return the query is logically invalid (i.e., contains least one logical fallacy).
Definition of logical fallacy
A logical fallacy refers to an error in reasoning. Informal fallacies are often content-dependent, such as appealing to irrelevant authority or making hasty generalizations. Formal fallacies, on the other hand, are structural errors in
reasoning that occur regardless of the content.
is crucial to consider the structure and the substance of the argument, evalu- ating whether the conclusions follow logically from the premises, and whether the premises and assumptions are factual and valid. Be cautious not to let per- sonal beliefs interfere with your analysis.
Evaluation criteria
For each given pair, compare the input statement against the principles of logical reasoning, to determine whether it contains a logical fallacy or not. Ensure your answer reflects the presence absence of logical fallacies, thus determining the logical validity invalidity of the statement.
Task steps
Common examples of logical fallacy
Here are some common examples of logical fallacies:
Ad Hominem: {details}
Appeal to Nature: (details)
Hasty Generalization: {details}
Post Hoc: {details]
False Cause: {details)
Table 2: Prompt generated by Evoke for epistemic reasoning.
Prompt from Evoke
Breakdown Introduction and objective
In this task, your goal is to determine whether the statement in the "Hypoth- esis" logically follows from the statement in the "Premise." This is known entailment. If the "Hypothesis" statement is logical consequence of the "Premise" statement, then an entailment. If not, then it non- entailment.
-Make sure to carefully consider the relations and assumptions mentioned in both the "Premise" and the "Hypothesiss statements.
Guidelines
-The entailment does not depend on the truth of the statements, but rather whether the logic in the "Hypothesis" follows from the "Premise".
-Pay close attention to the wording and structure of the sentences to analyze whether one entails the other.
Examples:
Examples
Entailment
Premise: The sun rises in the cast.
Hypothesis: The sun rises.
Explanation: The Hypothesis is a simplified version of the Premise and does not introduce any new information contradictions, hence it's an entailment. Non-entailment
Premise: Sarah believes that all cats are black.
Hypothesis: All cats are black. Explanation: Even though the Hypothesis is expressed in the Premise, it's tied to Sarah's 's belief and not presented as a fact, hence it's non-entailment.
Now, review provided pairs of statements. Determine if the Hypothesis logically follows from the Premise and respond with either entailment or non- entailment.
Task Execution
interpretable and simulate real typos. As example, sample from SST-2 is "that's pure pr hype". and its corresponding adversarial (corrupted) sample after the attack is "tha' 'cs pure pr hyp". We evaluate performance of different prompting methods on the corrupted samples.
Baselines. Besides APE and Evoke, we evaluate another model: RobEnc (Jones et al. 2020). which is widely-used rule-based defense approach. RobEnc works clustering denoiser cluster and

---

Published as a conference paper at ICLR 2024
APE
Evoke
Now given a user query, please do sentiment analysis where represents negative sentiment and represents positive sentiment. Please note that the imput may contain typos. These are not intentional and do not reflect the user's actual sentiment. You should intended meaning from the context before assigning sentiment score. example, merely badv trather than puinfzully awfl' could be corrected to merely rather than painfully awful' scored as Similarly, 'I comelling mobion' could be corrected compelling motion' and scored as 1.
Provided with user query, please conduct a sentiment analysis. In this analysis. score of O represents negative sentiment, while score of represents positive senliment. Ensure to carefully check for any typos before providing a response.
correct the typos or infer the
Figure 6: Prompts from APE and Evoke on adversarial attacked SST-2 task
denoise potentially corrupted inputs into an encoding, then the denoised encoding is fed to the subsequent model (e.g.. GPT-4) for inference. RobEnc learns rule-based word cluster for denoising: for example, if the word "hallo" is clustered around the word "hello", then all the "hallo" in the input will be converted to "hello".
Figure summarizes experimental results. We observe that Evoke significantly outper- forms all the baselines in all the tasks. The performance gain is more significant for adversarially constructed datasets, e.g., Adversarial-SST" and Adversarial-QU -QP... To understand this, we show the prompts generated by APE and Evoke in Figure [ We that although the prompt from APE pro- vides a clear instruction regarding the given task and acknowledges the existence of typos, it does provide clear guidelines how to address the typo. On the other hand, the prompt from Evoke provides detailed explanations and actionable suggestions about defending against typos.
Results.
4.3 TOWARDS FINE-GRAING TASKS: NAMED ENTITY RECOGNITION
Tasks in Figure and Figure 5 are all sentence-level classification tasks, e.g., deciding whether sentence is of positive negative sentiment. In this section, we investigate whether Evoke can handle more fine- -e-igaained tasks, such as token-level named entity recognition (Schneider et al. 2020
Zuo al. 2023)
Pvoke
User search queries may contain various entity types such
100
Brand, Product, Product Models, Services. Specification Attribute. Location, Media
Assign labels each entily follows: (Brand: Brand, Other Entily Other, Entity]. Refer to below example for guidance.
90
80
Example Q: year meafee
security
70
A: <Result- Thinking> From common knowledgs, McAfee is brand and interact socurity is a product model. The query also specification
60
includes year"
</Thinking" <Label> 1:Spee_ AitribAt, ypur:Spe Attribule, meafee:Brand, Product:Product Modol, security: Models Labbll>> <Reeult>
50
40
English
[Example 2-43
Figure/ 7: Results of APE and Evoke on an in- house multi-lingual NER dataset.
Figure 8: Prompt from Evoke the NER task.
We collect multi-lingual in-house query data from a search engine, and for each token the query, goal is to assign the token to pre-defined class (e.g., brand, location).
We illustrate results of Evoke on the fine ome-grained NER task in Figure 7 We see that Evoke sig- nificantly outperforms APE on all the languages. We further show the prompt generated by Evoke in Figure g From the prompt, we see that Evoke is able to automatically generate examples and explanations about the task.
8

---

Published as a conference paper at ICLR 2024
Comman Concept
Comman Concept
0.175
Accuracy
0:50
Euy
0.15
Random
0.20
0.10
0.075
0.030
0.025
Score
Stee
Adversarial
1.0
Accuracy
Accuracy
0.0
8.
0.4
8.
Score
Figure 9: Correlation between scores gener- ated by LLM-Reviewer and task accuracy.
Figure 10: Task accuracy over the number of iteration steps.
4.4 ANALYSIS
LLM-Reviewer can janjudge the quality of prompts. Recall that in Evoke, LLM-Reviewer scores the prompts generated by LLM-Author. We empirically show that the scores reflect the quality of the generated prompts. To examine the effectiveness of these scores, we illustrate the relationship between the scores and the task accuracy in Figure ? In the experiments,
consider Adversarial-SST2 and Common-Concept. From the results, we see that the scores can indeed reflect the final task accuracy. For example, for Common- Conccp,,,, 5% when the prompt score is 6, and the task accuracy increases to about 17% when the prompt score increases A similar trend also revealed on the Adversarial-SSTT task. We see that when the score is 7.5, the final task accuracy barely reaches 75%. And when the score increases to 8, the task accuracy increases 85%.
tasks:
see that the task accuracy is about
LLM-Author iteratively improves prompt generation. In Evoke, because LLM-Authon takes the feedback from A...-Reviewwr into consideration, can iteratively improve the generated prompt. We demonstrate this in Figure [0] (the left-most orange bars). From the results, we see that indeed the final task accuracy continues to increases when we increase the number of iteration steps. For example, Adversarial-SST2, with one refinement iteration, the final task accuracy is about 75%. When we increase the number of refinement iterations to 3, we see that task accuracy significantly increases to above 90%.
Effectiveness of LLM-Selector. Recall that Evoke, we only consider the "hard" samples in each iteration. We demonstrate the effectiveness of such strategy in Figure TO We consider three settings: Hard is the strategy that we adopt Evoke; Random when we randomly select samples instead of selecting based on score; and Easy is when we select the easy samples instead of the hard ones. From the results, we see that on both Common-Concept and Adversarial SST-2, Easy yields the worst performance, indicating that the hard samples are more helpful than the easy ones. Moreover, we observe that performance of Random is worse than Hard (i.e., Evoke), further implying the effectiveness the proposed data selection strategy.
CONCLUSION
We propose Evoke, an author-reviewer framework for automatic prompt engineering. In Evoke, the same LLM serves two roles: as reviewer it scores the quality of the prompt; and author refines the prompt, taking the feedback of the reviewer into account. We further propose data selection strategy, where we only expose the hard samples to the model. Extensive experiments show that Evoke outperforms existing automatic prompt engincering approaches.

---

Published as a conference paper at ICLR 2024
REFERENCES
Sébastien Bubeck, Varun Chandrasekaran, Ronen Eldan, Johannes Gehrke, Eric Horvitz, Ece Ka- mar, Peter Lee, Yin Tat Lee, Yuanzhi Li, Scott Lundberg, et al. Sparks of artificial general intelligence: Early experiments with gpt-4. ArXiv preprint, abs/2303.12712 2023.
Mark Chen, Jerry Tworek, Heewoo Jun, Qiming Yuan, Henrique Ponde de Oliveira Pinto, Jared Kaplan, Harri Edwards, Yuri Burda, Nicholas Joseph, Greg Brockman, al. Evaluating large language models trained on code. ArXiv preprint, abs/2107.03374 2021.
Tim Dettmers, Artidoro Pagnoni, Ari Holtzman, and Luke Zettlemoyer. Qlora: Efficient finetuning of quantized llms. ArXiv preprint, hts/2305514314, 2023.
Yuging Du, Olivia Watkins, Zihan Wang, Cédric Colas, Trevor Darrell, Pieter Abbeel, Abhishek Gupta, and Jacob Andreas. Guiding pretraining in reinforcement learning with large language models, 2023.
Luyu Gao, Aman Madaan, Shuyan Zhou, Uri Alon, Pengfei Liu, Yiming Yang, Jamie Callan, and Graham Neubig. Pal: Program-aided language models. In International Conference Machine Learning, pp. 10764-10799. PMLR, 2023.
Or Honovich, Uri Shaham, Samuel Bowman, and Omer Levy. Instruction induction: From few examples to natural language task descriptions. ArXiv preprint, abs/2205. 10782, 2022.
Edward J. Hu, Yelong Shen, Phillip Wallis, Zeyuan Allen-Zhu Yuanzhi Li, Shean Wang. Lu Wang. and Weizhu Chen. Lora: Low-rank adaptation of large language models. In The Tenth Inter- national Conference on Learning Representations, ICLR 2022, Virtual Event, April 25-29, 2022. OpenReview.net, 2022.
Jie Huang and Kevin Chen-Chual Chang. Towards reasoning in large language models: A survey. ArXiv preprint, abs/2212. 10403, 2022.
Erik Jones, Robin Jia, Aditi Raghunathan, and Percy Liang. Robust encodings: A framework for combating adversarial typos. In Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics, pp. 2752-2765 Online, 2020. Association for Computational Lin- guistics. doi: 10. 12.28633/1/0020aaammammm.
Takeshi Kojima, Shixiang Shane Gu, Machel Reid, Yutaka Matsuo, and Yusuke Iwasawa. Large language models are zero-shot reasoners. Advances in neural information processing systems, 35:22199 22213. 2022.
Jaahhang Liu, Dinghan Shen, Yizhe Zhang, Bill Dolan, Lawrence Carin, and Weizhu Chen. What makes good in-context examples for GPT-3? In Proceedings of Deep Learning Inside Out (Dee- LIO 2022): The 3rd Workshop on Knowledge Extraction and Integration for Deep Learning Architectures, pp. 100-114, Dublin, Ireland and Online, 2022. Association for Computational Linguisties. doi: 10.18063/111 18855/v1/2022..66 deeiioo-.del
Sewon Min, Xinxi Lyu, Ari Holtzman, Mikel Artetxe, Mike Lewis, Hannaneh Hajishirzi, and Luke Zettlemoyer. Rethinking the role of demonstrations: What makes in-context learning work? In Proceedings of the 2022 Conference Empirical Methods in Natural Language Processing, pp- 11048-1 11044, Abu Dhabi, United Arab Emirates, 2022. Association for Computational Linguis- tics.
Long Ouyang, Jeffrey Wu, Xu Jiang, Diogo Almeida, Carroll Wainwright, Pamela Mishkin, Chong Zhang, Sandhini Agarwal, Katarina Slama, Alex Ray, al. Training language models to follow instructions with human feedback. Advances in Neural Information Processing Systems, 35: 27730-27744, 2022.
Laria Reynolds and Kyle McDonell. Prompt programming for large language models: Beyond the few-shot paradigm. Extended Abstracts of the 2021 CHI Conference on Human Factors in Computing Systems, pp. 1-7, 2021.
10

---

Published as a conference paper at ICLR 2024
Sami Sarsa, Paul Denny, Arto Hellas, and Juho Leinonen. Automatic generation of programming exercises and code explanations using large language models. In Proceedings of the 2022 ACM Conference on International Computing Education Research- Volume 1, pp. 27-43, 2022.
Elisa Terumi Rubel Schneider, João Vitor Andrioli de Souza, Julien Knafou, Lucas Emanuel Silva e Oliveira, Jenny Copara, Yohan Bonescki Gumiel, Lucas Ferro Antunes de Oliveira, Emer- son Cabrera Paraiso, Douglas Teodoro, and Cláudia Maria Cabral Moro Barra. BioBERTpt- Portuguese neural language model for clinical named entity recognition. Pruceedings of the 3rd Climical Natural Language Processing Workshop, pp. 65-72, Online, 2020, Association for Computational Linguistics. doi: 10.18653/v1/2020 1tt.6653511222200mmmcaaallll/l/
Taylor Shin, Yasaman Razeghi, Robert L. Logan IV, Eric Wallace, and Sameer Singh. Auto- Prompt: Eliciting Knowledge from Language Models with Automatically Generated Prompts. In Proceedings of the 2020 Conference on Empirical Methods Natural Language Process- ing (EMNLP), pp. 4222-4235, Online, 2020. Association for Computational Linguistics. doi: 10.18653/v wttp://ee0000002.mmm.ee main.346.
Disha Shrivastava, Hugo Larochelle, and Daniel Tarlow. Repository-Ievel prompt generation for large language models of code. In International Conference on Machine Learning, pp. 31693- 31715. PMLR, 2023.
Richard Socher, Alex Perelygin, Jean Wu, Jason Chuang, Christopher D. Manning, Andrew Ng, and Christopher Potts. Recursive deep models for semantic compositionality over a sentiment treebank. In Proceedings of the 2013 Conference on Empirical Methods in Natural Language Processing, pp. 1631-1642, Seattle, Washington, USA, 2013. Association for Computational Lin- guistics.
Mirac Suzgun, Nathan Scales, Nathanael Schärli, Sebastian Gehrmann, Yi Tay, Hyung Won Chung, Aakanksha Chowdhery, Quoe Le, Ed H Chi, Denny Zhou, et al. Challenging big-bench tasks and whether chain-of-thought can solve them. ArXiv preprint, abs/2210.092611 2022.
Arun James Thirunavukarasu, Darren Shu Jeng Ting, Kabilan Elangovan, Laura Gutierrez, Ting Fang Tan, and Daniel Shu Wei Ting. Large language models medicine. Nature medicine,
pp. 1-11, 2023.
Alex Wang. Amanpreet Singh, Julian Michael, Felix Hill, Omer Levy, and Samuel R. Bowman. GLUE: A multi-task benchmark and analysis platform for natural language understanding. In 7th International Conference Learning Representations, ICLR 2019, New Orleans, LA, USA, May 6-9, 2019. OpenReview.net, 2019.
Jindong Wang, Xixu Hu, Wenxin Hou, Hao Chen, Runkai Zheng, Yidong Wang, Linyi Yang. Haojun Huang, Wei Ye, Xiubo Geng, et al. On the robustness of chatgpt: An adversarial and out-of- distribution perspective. ArXiv preprint, abs/2302. 12095, 2023.
Xuezhi Wang, Jason Wei, Dale Schuurmans, Quoe Le, Ed Chi, Sharan Narang, Aakanksha Chowdh- ery, and Denny Zhou, Self-consistency improves chain of thought reasoning in language models. ArXiv preprini, abs/2203.11171, 2022.
Jason Wei, Maarten Bosma, Vincent Y. Zhao, Kelvin Guu, Adams Wei Yu, Brian Lester, Nan Du, Andrew M. Dai, and Quoc V. Le. Finetuned language models are zero-shot learners. In The Tenth International Conference on Learning Representations, ICLR 2022, Virtual Event, April 25-29, 2022. OpenReview.net, 2022a.
Jason Wei, Tay, Rishi Bommasani, Colin Raffel, Barret Zoph, Sebastian Borgeaud, Dani Yo- gatama, Maarten Bosma, Denny Zhou, Donald Metzler, et al. Emergent abilities of large language models. ArXiv preprint, abs/2206.07682. 2022b.
Jason Wei, Xuezhi Wang, Dale Schuurmans, Maarten Bosma, Fei Xia, Ed Chi, Quoc V Le, Denny Zhou, et al. Chain-of-thought prompting elicits reasoning in large language models. Advances in Neural Information Processing Systems, 35:24824-24837, 2022c.
Jules White, Quchen Fu, Sam Hays, Michael Sandborn, Carlos Olea, Henry Gilbert, Ashraf El- nashar, Jesse Spencer-Smith, and Douglas Schmidt. A prompt pattern catalog to enhance prompt engineering with chatgpt. ArXiv preprint, abs/2302. 1382, 2023.
11

---

Published as a conference paper at ICLR 2024
Sang Michael Xie, Aditi Raghunathan, Percy Liang, and Tengyu Ma. An explanation of in-context learning as implicit bayesian inference. The Tenth International Conference on Learning Rep- resentations, ICLR 2022, Virtual Event, April 25-29, 2022. OpenReview.net, 2022.
Kang Min Yoo, Junyeob Kim, Hyuhng Joon Kim, Hyunsoo Cho, Hwiyeol Jo, Sang-Woo Woo Lee, Sang- goo Lee, and Taeuk Kim. Ground-truth labels matter: A deeper look into input-label demon- strations. In Proceedings of the 2022 Conference on Empirical Methods in Natural Language Processing, pp. 2437, Abu Dhabi, United Arab Emirates, 2022. Association for Computa- tional Linguistics.
Biao Zhang, Barry Haddow, and Alexandra Birch. Prompting large language model for machine translation: A case study. ArXiv preprint, abs/2301.07069, 2023.
Yongchao Zhou, Andrei Ioan Muresanu, Ziwen Han, Keiran Paster, Silviu Pitis, Harris Chan, and Jimmy Ba. Large language models are human-level prompt engineers. ArXiv preprint, abs/2211.019100 2022.
Simiao Zuo, Pengfei Tang. Xinyu Hu, Qiang Lou, Jian Jiao, and Denis Charles. Deeptagger: Knowledge enhanced named entity recognition for web-based ads queries. ArXiv preprint, abs/2306. 17413, 2023.
12

---

Published as a conference paper at ICLR 2024
A INSTRUCTION INDUCTION
In Evoke, we use the author-reviewer framework to modify a task-speeific prompt. In the experi- ments, use off-the-shelf algorithm to generate the initial task-specific prompt. Table strates examples of using instruction induction (Honovich et al. 2022) for prompt initialization.
Table 3: Three examples of instruction inferred from input-output pairs
Input
Output
Inferred Instruction
Departure
Arrival
Get antonym
am Mike
Ich ben Mike
Translate to German
Build
Built
Get passive voice of the given verb
B PROMPTS OF LLM ROLES IN EVOKE
B.I LLM-REVIEWER
Prompt for As an experienced teacher, you are well-versed in discerning effective instruction that guides stu- dents toward correct answers. Please the following instruction scale of to 10, where 10 represents the highest level of clarity in problem description, execution steps, and comprehensive explanation the problem.
The task at hand is titled: {description}
History that may help you: (memory) The instruction to be rated is follows: {instruction)
Kindly provide your rating below.
B.2 LLM-AUTHOR
Prompt for LLt--uthor
Task Instruction: {instruction)
We've provided pairs consisting of inputs, the teacher's correct answers, and the students' responses. Please review the incorrect responses from the students and summarize key points that could be adjusted in the instruction to enhance student accuracy.
Pairs: {pairs}
History that may help you: {memory]
To improve the outcome, please revise the task instruction. Highlight major edits and present the updated task instruction.
13

---

Published as a conference paper at ICLR 2024
B.3 LLM-SELECTOR
Prompt for LLM-Selector
As experienced teacher with insight into the various levels of difficulty of exam questions, please rate the following question on scale to 10, considering factors such as conceptual under- standing, application of knowledge, problem-solving skills, time required, clarity of language, and accessibility, where denotes extremely easy and 10 denotes extremely difficult. Task instruction: (instruction] Input: }input}
Correct answer: (answer)
14

---

Published as a conference paper at ICLR 2024
C GENERATED INSTRUCTIONS
We include generated instructions from tasks below.
C.I ORTHOGRAPHY STARTS WITH
Prompt from Evoke
Given an imput sentence and specified letter, identify the word or words starting with the given letter. If there are two or more words in a sequence starting with the specified letter, include all of them as single answer. Ensure to present the word or group of words.
Here are the steps follow:
-Read the provided input sentence carefully.
-Idettffy word or words that start with the specified letter.
If there consecutive words starting with the specified letter, group them together entity. -oor example, if the input is prefer eating apples." and the specified letter is [e], your answer should be eating.
C.2 COMMON CONCEPT
Prompt from Evoke
Given a list, find the commonality between the inputs. The commonality should be a meaningful characteristic, property, relation that applies all the inputs, just superficial coincidental feature.
For example, can be used for repairs is a valid commonality for ['sewing', 'wrenches', 'glue', 'surgery'], ]. but tools or skills for joining is too broad and vague, and contain the letter e is too trivial and irrelevant.
C.3 RHYMES
Prompt from Evoke
For this task, you are required to find word that rhymes with the given word. The word you provide should not be the same as the given word, and should be a real, correctly spelled word from the English language. A rhyming word is defined as a word that has the last syllable sounding identical to the last syllable of the given word. For example, the given word is "hat", word that rhymes with it is "cat".
Here are the steps to complete this task: Read the given word carefully.
Think of word that has the same ending sound as the given word.
Ensure that the word you thought of is real word, is spelled correctly, and is not the same as the given word.
Write down the rhyming word next to the given word.
Now, please proceed with finding a word that rhymes with each of the following words.
15

---

Published as a conference paper at ICLR 2024
C.4 MOVIE RECOMMENDATION
Prompt from Evoke
Given user's 's interest in movies he watched previously: 'watched'. Now given four different movies from A to D, please recommend one that might be the most interest of the user.
To help you make a good recommendation, consider the following factors:
- Thee genre, theme, and tone of the movies. For example, if the user likes comedy, action, or drama.
The similarity or difference between the movies and the ones the user watched before. For example, if the movies are part of a series, a remake, or a spin-off.
The popularity, ratings, and reviews of the movies. For example, if the movies are critically acclaimed, award-
or have a large fan base.
Use these factors to compare and contrast the movies and explain why you think one of them the best choice for the user. Do not just pick a movie based on your personal preference or guesswork. Example: If the user watched The Godfather, The Godfather Part II, and Goodfellas, and the options A) The Departed, B) Scarface, C) The Irishman, and D) Casino, a possible answer is: A
The Departed is a crime thriller that has a similar genre, theme, and tone to the movies the user watched before. It is also remake of Hong Kong film called Infernal Affairs, which adds twist to the familiar story of undercover agents and mobsters. The Departed is a highly popular and acclaimed movie that won four Oscars, including Best Picture and Best Director. It has a star- studded cast that includes Leonardo DiCaprio, Matt Damon, Jack Nicholson, and Mark Wahlberg. The user might enjoy the suspense, the plot twists, and the performances of the actors in this movie. Therefore, recommend The Departed as the best option for the user.
C.5 LOGICAL FALLACY DETECTION
Prompt from Evoke
In this task, you are required to identify both informal and formal logical fallacies in the provided input statements. Your response should be binary value: return 1 if the query is logically valid (i.e., free from any logical fallacies), and return 0 if the query is logically invalid (i.e., contains at least one logical fallacy). A logical fallacy refers to an error in reasoning. Informal fallacies are often content-dependent, such as appealing to irrelevant authority or making hasty generaliza- tions. Formal fallacies, on the other hand, are structural errors in reasoning that occur regardless of the content. It is crucial to consider the structure and the substance of the argument, evalu- ating whether the conclusions follow logically from the premises, and whether the premises and assumptions are factual and valid. Be cautious not to let personal beliefs interfere with your anal- ysis. For each given pair, compare the input statement against the principles of logical reasoning. to determine whether it contains a logical fallacy or not. Ensure your answer reflects the presence or absence of logical fallacies, thus determining the logical validity or invalidity of the statement. Here are some common examples of logical fallacies:
Ad Hominem: Attacking the character of a person making an argument rather than the argument itself.
AAppell to Nature:
Claiming something is good because it's natural, or bad because it's unnatural. Hasty Generalization: Making a broad claim based on small or unrepresentative sample size.
- -Post Hoc: Assuming that because one event followed another, the first event caused the second event.
False Cause: Assuming a false or misleading cause-and and-anfectt relationship.
16

---

Published as a conference paper at ICLR 2024
C.6 PRESUPPOSITIONS AS NLI
Prompt from Evoke
Determine whether the first sentence entails, contradicts, is neutral to the second sentence. The term "entailment" means that the information in the first sentence logically supports or leads to the conclusion presented in the second sentence. The term "contradiction" means that the information in the first sentence logically opposes or disproves the information in the second sentence. The term "neutral" implies that the information the first sentence neither supports nor opposes the information in the second sentence; they are unrelated or the relation between them is ambiguous. It's important to focus on the factual information provided rather than assumptions or external knowledge. Make sure to carefully read both sentences and analyze their logical relation based only on the given text.
Entailment:
The information in the first sentence supports the conclusion in the second sentence. Contradiction: The information the first sentence opposes or disproves the information in the second sentence.
Neutral: The information in the first sentence neither supports nor opposes the information in the second sentence, the relation between them is ambiguous.
For each pair, please provide the correct judgment between entailment, contradiction, and neutral, based only the provided text. Please avoid assumptions and focus solely the text provided.
C.7 WINOWHY
Prompt from Evoke
In the given text, you are required to evaluate the reasoning provided concerning the identification of the antecedent of a pronoun sentence. The antecedent is the noun that the pronoun is referring to. Carefully examine the reasoning to determine if it accurately identifies the antecedent based solely on the information presented within the sentence itself. Here are the steps you should follow: Read the sentence and the reasoning provided thoroughly.
Assess whether the reasoning accurately identifies the antecedent of the pronoun based solely on the provided text. Avoid making assumptions or using external knowledge.
Iff the reasoning correctly identifies the antecedent the pronoun, based on the information given in the sentence.
IIf the reasoning fails to accurately identify the antecedent of the pronoun or relies on assumptions external information.
Remember,
Your evaluation should strictly be based on the information provided in the text. Your goal is to assess the accuracy of the reasoning in identifying the antecedent of the pronoun.
17

---

Published as a conference paper at ICLR 2024
C.8 EPISTEMIC REASONING
Prompt from Evoke
In this task, your goal is determine whether the statement in the "Hypothesis" logically follows from the statement in the "Premise." This is known as entailment. If the "Hypothesis" statement is logical consequence of the "Premise" statement, then it is entailment. If is not, then it is non-entailment.
-Make sure to carefully consider the relations and assumptions mentioned in both the "Premise" and the "Hypothesis" statements.
-The entailment does not depend on the truth of the statements, but rather whether the logic in the "Hypothesis" follows from the "Premise".
Pay close attention the wording and structure of the sentences to analyze whether one entails the other.
Examples:
Entailment
Premise:
The sun rises in the cast. Hypothesis: The sun rises. Explanation: The Hypothesis is simplified version of the Premise and does not introduce any information or contradictions, hence it's an entailment.
Non-entailment
Premise: Sarah believes that all cats are black.
Hypothesis: All cats black. Explanation: Even though the Hypothesis is expressed in the Premise, it's tied to Sarah's 's belief and not presented as a fact, hence it's non-entailment. Now, review the provided pairs of statements. Determine if the Hypothesis logically follows from the Premise and respond with either entailment or non-entailment.
18

---

Published as a conference paper at ICLR 2024
C.9 ADVERSARIAL SST2
Prompt from Evoke
Now given user query, please do sentiment analysis where 0 represents negative sentiment and represents positive sentiment. Please note that the input may contain typos. These are not intentional and do not reflect the user's actual sentiment. You should try to correct the typos or infer the intended meaning from the context before assigning a sentiment score. For example, 'to merely badv trather than painfzully awfl' could be corrected to 'to merely bad rather than painfully awful' and scored as O. Similarly, 'I comelling mobion' could be corrected to 'a compelling motion and scored as 1. Sentiment analysis.
C.10 ADVERSARIAL QQP
Prompt from Evoke
You will be given a pair of questions and asked to determine whether they are paraphrases of each other. Paraphrases are questions that have the same meaning ask about the same information, even if they use different words or structures. Please answer with a binary value if the questions paraphrases, or O if they not. Please pay close attention to typos, spelling, grammar, and punctuation before answering, as they may affect the meaning of the questions. you are not sure whether the questions paraphrases not, you use strategies to help you decide, such as:
Compare the keywords and topics of the questions. Do they match relate each other? Rewrite one question in different way and see if it still conveys the same message as the other question. Think about the context and purpose of the questions. Are they asking for the same type of information or response?
For example, the questions What is the capital of France? and Which city is the seat of the French government? are paraphrases, because they both ask about the same fact and can be answered with the same word (Paris). However, the questions How do you play the guitar? and What are some guitar chords? are not paraphrases, because they ask for different kinds of information and have different levels of specificity.
19

---

Published as a conference paper at ICLR 2024
D ADDITIONAL EXPERIMENTS
D.1 INITIAL PROMPT
The influence initial prompt quality on the efficacy of Evoke was an important aspect of our in- vestigation. To thoroughly examine this, we conducted additional experiments using two alternative initial prompts: task description and an empty string. As illustrated in Table below, Evoke's per- formance showed relative insensitivity to the quality of the initial prompt. These results, further underscore the robustness of our approach.
Performance Deferrnce
-1.0
rntial Prompt Comparisen
Empty Instruction Induction
Task Description Instruction Induction
ortic stars
concopt
Shymes
logical Aal
Weosly
esi reasuning
3572
alc Nr.072
emglich
movie
adx
Figure 11: Comparison of different prompt initialization methods: The blue bar indicates that, average, using task description as the initialization prompt is more effective, while the orange bar suggests that an empty string performs on par with instruction induction on average.
20

---

Published as a conference paper at ICLR 2024
D.2 OPEN SOURCE MODELS
We expanded our experiment scope to include open source models Mistral-7B-Openorca and Llama2-70B-Instruue Our findings reveal notable performance enhancements with these models, affirming the versatility and robustness of our Evoke framework beyond GPT-4.
180
Matral 70
Matrral 70 Exite
wage 2013
COP
Figure 12: Comparison of test accuracy on APE and Evoke for Mistral-7B- -Be-Oooorocaa and Llama2- 70B-Instrune the Evoke prompt outperforms the baseline for both open source models.
21


Reviews:
[{"review_id": "rtRkBslfa5", "reviewer": "Reviewer_hcVA", "summary": "This work proposed an author-reviewer framework, Evoke, for automatic prompt engineering. In this framework, the same LLM serves two roles: as a reviewer it scores the quality of the prompt; and as an author it refines the prompt, taking the feedback of the reviewer into account. On top of this, the authors further propose a data selection strategy, where they only expose the hard samples to the model. This work also conduct extensive experiments to demonstrate that Evoke outperforms existing automatic prompt engineering approaches.\n\n====After authors' discussion===\nI have read through the authors' response, and I think they have addressed all my concerns. This is a nice work, and many thanks for the efforts!", "questions": "- Are all the prompts generated by Evoke concatenated together during the evaluation phase?\n\n- How do the quality of initial prompts affect the effectiveness of the Evoke method throughout iterations?", "limitations": "", "rating": 8, "confidence": 4, "soundness": 4, "presentation": 4, "contribution": 4, "strengths": "[+] The Evoke framework is novel in its approach to improving LLM prompt refinement using an author-reviewer framework (with a data selector).\n\n[+] The performance improvements seems impressive especially on some very difficult tasks.", "weaknesses": "[-] Iteration-Dependent: The efficacy of the approach may heavily depend on the number of iterations and the quality of feedback loops, which could vary significantly based on the complexity of the task and the initial prompt quality.\n\n[-] Complexity and Overhead: The Evoke model might introduce additional complexity and computational overhead since we need to do the iterative selection process T times.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "913Bee3MIs", "reviewer": "Reviewer_PdAc", "summary": "This article introduces an innovative approach to optimizing the prompts for large language models. It presents the \"Evoke\" method, which combines three instances of the language model to generate, verify, and select prompts for each example. After several iterations, the generated prompts guided by the performance on the training set will generalize well to the test set. The experimental results demonstrate that Evoke outperforms several baseline methods on challenging datasets.", "questions": "1. A key concern revolves around the distinctions between Evoke and \"Large Language Models as Optimizers\" by DeepMind. Clarifying these differences is pivotal for a comprehensive understanding of both approaches.\n\n2. Could you provide more information regarding the number of steps involved in generating prompts for each task? Additionally, is the prompt length substantial for each task?\n\n3. Efficiency may be a concern when dealing with very large training datasets, as it necessitates running the large language model multiple times on the entire training dataset to obtain evaluation scores. A discussion on potential optimizations or trade-offs for such scenarios would be beneficial.\n\n4. Section 4.4 highlights the effectiveness of the LLM-Selector. However, it remains unclear why, if the generated prompt is superior, there isn't consistently better or at least equivalent performance on easier examples. This phenomenon is briefly mentioned without a deeper exploration or analysis. Further elaboration would enhance the paper's clarity.", "limitations": "", "rating": 5, "confidence": 4, "soundness": 3, "presentation": 2, "contribution": 2, "strengths": "1. The problem of automatic prompt generation addressed in this paper holds significant importance in the age of Large Language Models (LLMs). While LLMs can generalize across various tasks, the quality of the generated prompts plays a pivotal role in overall task performance. Therefore, I believe that researching this problem is both important and intriguing.\n\n2. The proposed Evoke method offers a straightforward yet highly effective solution. It seamlessly combines different types of LLMs to create a versatile pipeline for task resolution, all without the need for additional training phases while maintaining exceptional performance.\n\n3. The analysis of Evoke's robustness and suitability for fine-grained tasks underscores its versatility and applicability in a wide range of scenarios.", "weaknesses": "1. The Evoke method shares significant similarities with \"Large Language Models as Optimizers\" from DeepMind. While there are variations in the detailed processes, the core contributions and methodologies closely resemble each other. Since the DeepMind paper was released before the ICLR deadline, it is imperative to include a discussion of this relationship in the paper.\n\n2. Some of the detailed tables and figures could be relocated to the appendix to allow ample space for a more comprehensive description of the experimental settings and the Evoke method itself. A more detailed explanation of the Evoke process would enhance the paper's clarity.\n\n3. The rationale behind selecting the Instruction Induction and Big Bench Instruction Induction datasets is not clearly articulated. Some of these tasks may be practical, while others could be considered more akin to toy tasks. Additionally, the use of the LLM as merely GPT-4, a black-box large language model, raises the question of whether the Evoke method is equally effective with other large language models, such as Llama 2 or ChatGPT.\n\n4. Comparing Evoke to other baseline methods may be perceived as somewhat unfair, given that Evoke leverages supervised information from the training dataset. This distinction should be acknowledged when discussing the comparative results.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "KhH9ehLs8m", "reviewer": "Reviewer_7Mrx", "summary": "This paper describes a method for refining prompts using GPT4. It uses GPT4 in three settings: as an author (to refine prompts) as a Reviewer (to assess quality of prompts and give feedback) as a selector (to select good examples to include in prompts). \n\nThe author instance takes as input the current prompt, and the historical feedback to create a new prompt. The reviewer takes the previous prompts, end-task accuracy when using the prompts to assess quality of the new prompt. Lastly, the selector identifies instances that would be difficult for the model to answer given the current prompt and these instances are used by the author and the reviewer.\n\n Experiments have been presented with qualitative examples from the Instruction Induction datasets. Comparisons against human-generated prompts, and a baseline method for automated prompt engineering ( a method based on paraphrasing existing prompt candidates) reveal significant improvements on multiple tasks including logical fallacy detection, movie recommendation, common concept identification. Experiments on perturbed prompts reveal that the method also results in improved robustness towards typographical errors as it ends up generating instructions that explain how those should be handled by the model. The paper also presents an analysis indicating a correlation between the Reviewer scores and the end-task accuracy. It also shows that as the cycle repeats, the author generates better prompts (as determined improved end-task accuracy) and that the use of the selector for identifying difficult instances help improve performance.", "questions": "1.  The choice of using a reviewer to estimate scores instead of directly relying on end-task scores is an interesting one? Is it for computational reasons/costs to query GPT4? An elaboration would help (I may have misunderstood this as aspect). I had to re-read the sections describing the author, reviewer, selector multiple times along with the algo block to be sure of whats going on. This section could benefit from some more details in an image and/or writing. \n2. While the paper only uses GPT4, I was wondering if the authors could experiment with open-access models such as Falcon 180B and other smaller-scale models. I am sure that the authors would agree \"LLMs\" cannot do whatever the paper describes in general -- if someone does not use the GPT4 OpenAI model, is there a takeaway from this paper? A study of how this behavior changes with scale (of model sizes) could be a useful addition.", "limitations": "", "rating": 6, "confidence": 3, "soundness": 2, "presentation": 2, "contribution": 3, "strengths": "- Simple method to improve the performance of prompts\n- Experiments on instruction induction tasks show a significant improvement", "weaknesses": "- Writing needs to be improved in some parts (more details in questions)\n- Studies only GPT4 - unclear if there are any takeaways from the paper if one isn't using GPT4.  (see question)", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "Hy07H1rJI9", "reviewer": "Reviewer_QMNd", "summary": "The paper introduces Evoke, an automatic prompt refinement framework designed to enhance the performance of large language models (LLMs). Traditional prompting methods often underutilize the potential of LLMs due to ad-hoc prompt selection or inefficient random search. Evoke employs an author-reviewer feedback loop with two instances of an LLM: one acting as a reviewer and scoring the current prompt, and the other as an author, which refines the prompt based on feedback and edit history. Additionally, Evoke incorporates a data selection approach that exposes the LLM to more challenging samples, allowing for deeper task understanding.", "questions": "What are the loss functions for each module? \n\nWhat are the targets for example for first difficulty ranking module?", "limitations": "", "rating": 6, "confidence": 3, "soundness": 2, "presentation": 3, "contribution": 2, "strengths": "The proposed method outperforms state-of-the-art approaches.\n\nThe paper is well-written and easy to follow.\n\nThe approach is not human labor intensive.\n\nThe self improving adversarial setting is interesting.", "weaknesses": "Not enough details are provided about loss functions of each module.\n\nNot enough explanation of the modules; making it difficult to reproduce for other researchers.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}]

Author rebuttal:

Decision: Accept (poster)