Explaining Patterns in Data  with  Language Models via Interpretable Autoprompting
Authors: Chandan Singh
John Xavier Morris
Jyoti Aneja
Alexander M Rush
Jianfeng Gao
Venue: Submitted to ICLR 2023 (2023)
Keywords: Interpretability
explainability
XAI
AI for science

Abstract: Large language models (LLMs) have displayed an impressive ability to harness natural language to perform complex tasks. In this work, we explore whether we can leverage this learned ability to find and explain patterns in data. Specifically, given a pre-trained LLM and data examples, we introduce interpretable autoprompting (iPrompt), an algorithm that generates a natural-language string explaining the data. iPrompt iteratively alternates between generating explanations with an LLM and reranking them based on their performance when used as a prompt. Experiments on a wide range of datasets, from synthetic mathematics to natural-language understanding, show that iPrompt can yield meaningful insights by accurately finding groundtruth dataset descriptions. Moreover, the prompts produced by iPrompt are simultaneously human-interpretable and highly effective for generalization: on real-world sentiment classification datasets, iPrompt produces prompts that match or even improve upon human-written prompts for GPT-3. Finally, experiments with an fMRI dataset show the potential for iPrompt to aid in scientific discovery.

Under review as a conference paper at ICLR 2023
EXPLAINING PATTERNS IN DATA WITH LANGUAGE MODELS VIA INTERPRETABLE AUTOPROMPTING
Anonymous authors Paper under double-blind review
ABSTRACT
Large language models (LLMs) have displayed an impressive ability to harness natural language to perform complex tasks. In this work, we explore whether we can leverage this learned ability to find and explain patterns in data. Specif- ically, given a pre-trained LLM and data examples, we introduce interpretable autoprompting (iPrompt), an algorithm that generates a natural-language string explaining the data. iPrompt iteratively alternates between generating explana- tions with an LLM and reranking them based on their performance when used as a prompt. Experiments on a wide range of datasets, from synthetic mathematics to natural-language understanding, show that iPrompt can yield meaningful insights by accurately finding groundtruth dataset descriptions. Moreover, the prompts produced by iPrompt are simultaneously numan-interrrettan tive for generalization: on real-world sentiment classification datasets, iPrompt produces prompts that match even improve upon human- written prompts for GPT-3. Finally, experiments with an fMRI dataset show the potential for iPrompt aid in scientific discovery.
and highly effec-
INTRODUCTION
Large language models (LLMs) have attained an extraordinary ability to hamnss natural language for solving diverse natural-language problems (Devlin et al. 2018), often without the need for fine- tuning (Brown et al. 2020 Sanh et al., (021). Moreover, LLMs have demonstrated the capacity to
excel at world problems, such as mathematics (Lewkowycz et al. 2022) and scientific question
answering (Sadat & Caragea 2022
In this work, we probe whether we can leverage the Iearned skills an LLM to find and explain patterns in dataset. To do so, invert the typical problem of fitting LLM to data and instead ask whether we can use a fixed LLM to produce a latural-language string explaining dataset patterns. Our approach this problem centers around prompting. Prompting has emerged an effective method for adapting LLMs to perform new tasks (Liu et al. (021a). A prompt string is combined with each example dataset before querying LLM for answer.
While prompts were initially constructed manually, recent work has shown success in antoprompt-- ing, i.c. automatically finding prompt via optimization (Shin et al., 2020 & Liang 2021 However, previous work on learning natural language prompts Shin et al. (020 does not produce prompts that are meaningful to humans.
Our approach, interpretable autoprompting (iPrompt), extends autoprompting to generate seman- tically meaningful natural- languugg prompt that explains key characteristic of the data (see Fig. B For example, given a dataset of examples of addition, e.g. 2. 5 = 7 to yield the natural-language description Add the inputs. iPrompt is an iterative algorithm that al- ternates between (i) proposing candidate explanations with an LLM, (ii) reranking the candidates based on their performance when used as a prompt, and (iii) exploring new candidates.
+ 4, we use an L.M
To evaluate iPrompt, curate diverse collection of datasets written in natural language (Table where our goal is to accurately infer a ground-trum pattern. The dataset includes a number of syn- thetic math datasets, well as language tasks from the Natural Instructions V2 dataset (Wang al.) 2022). We find that iPrompt outperforms baseline autoprompting methods in successfully finding correct description across these datasets. Moreover, the generated descriptions interpretable,

---

Under review as a conference paper at ICLR 2023
Zero-shot prediction
iPrompt (Dataset explanation)
Add the inputs
prompt
Input: 31 Output: 4
Input: 4 Output: 11
data
Input:
data
Input: Output: 14
LLM
LLM
Add the inputs prompt
Figure I: Interpretable autoprompting (iPrompt) inverts the standard prediction problem to instead find natural-language explanation of the data using fixed, pre-trained large language model (LLM).
Table 1: Dataset Explanation Task. For full details on each dataset, see Appendix A.1
Collection
#
Description
Dataset names
Inverse synthetic
10
Simple mathematical functions
Add two, Subtract two, Multiply two, Divide two, Max two, First number, Square, Exponentiate, Double, Fibonacci Country capital, Antonyms, Check edibility,
math
Inverse Allen NLI (Wang et al. 2022
10
Diverse language tasks
Rhyme generation, Country currency, Check prime, Check vegetarian, Find typo, Gender classification, SQL query generation
Sentiment
4
Sentiment classification
SST-2, RottenTomatoes, IMDB, Financial Phrasebank
Natural- lanuual-anguage IMRI (Huth et al. 2016
20
Find underlying category from list of words that excite an fMRI voxel
Extracting a pattern from a set of words, each corresponding to a different voxel
allowing human auditing and enabling strong generalization performance when used prompt in a new setting (i.e. when used for a different LLM). On real-world sentiment classification datasets, iPrompt even produces prompts that match or improve upon human- human-written written prompts for GPT-3. Fi- nally, we qualitatively explore iPrompt in a neuroscience task, in which we seek to understand the mapping of semantic concepts in the brain from fMRI imaging (data from Huth et al. (206)).
2 DATASET EXPLANATION TASK
Task definition Given a dataset comprised of input-output string pairs {(x y ).... (x.N Ny )}, the goal is produce a "semantically meaningful" natural-language string that explains the relation- ship between x and y. We require that a string consists of human-understandaar sequence of incongruous tokens. For example, in the task shown in Fig. B the task is recover text synonymous to Add the inputs given samples of data performing addition.
text rather than a
Datasets Table M shows the four collections of datasets we study: (1) Inverse Synthetic Math with datasets that require inferring an underlying mathematical function of one or two numbers; (2) In- verse Allen NLI (ANLI), a selection of crowdsourced language tasks (Wang et al.. 2022) with easily verifiable descriptions (e.g. Find a country's 's capital); (3) Sentiment, consisting
four real-world
sentiment classification tasks and (4) fMRI, a dataset involving brain responses to natural language, motivated by the goal of recovering unknown explanations. In addition to data examples, the first two collections contain ground-truth description and simple rules to test whether extracted de- scription matches the ground-truth one. For example, when adding two numbers (Fig. 3), the rule checks whether deseription contains any of the keywords add, sum, +.
The examples in each task do not directly contain the task description. For example, when inferring the Add two numbers task, the examples do not contain a plus sign or any synonyms of the word add such as combine. For classification tasks such as Check edibility or Check prime, the label provided in the example text is simply yesino rather than the given labels, e.g. ediblelnon-edible.
2

---

Under review as a conference paper at ICLR 2023
GPT-2 58)
GPT-Neo (2.78)
GPT-) (6B)
GPT-3 (1758)
Add
Add
Reddictoon atourscy
Subtract-
Subtract-
Subcract
Subtract
Max
Max
Nax
Fist
First
Fist
Dfffrence
Maxmum
Fiiz
Diffeence
Maxmum
fist
Difference
Moxmum
Rist
Sum
Noximum
0.00
Prompt keyword
Figure 2: Model accuracy depends on having an accurate prompt for large models (GPT-J 6B and GPT-3). The model given the prompt Return the of the impuls., where the shown prompt keyword before querying the output given two inputs numbers in string. Darker indicates higher accuracy, and high accuracy along the diagonal indicates that the correct prompt induces the highest accuracy.
filled with
Evaluation We evaluate dataset explanation on two criteria: closeness to the ground prompt and ability generalize
prompt for other models. To evaluate similarity to the ground truth, we score a ranked list of prompts based on mean reciprocal rank (MRR). Given a set of datasets D (Di,..., } we compute: MRR Doi Σ 2i=1 rank," where rank; the one-indexed rank of the first correct explanation. We evaluate correctness based on whether the generated explanation contains one of set of problem-specife keywords. To measure generalization. we use the top- ranked string as a zero-shot prompt for different language model, and evaluate whether that model is able to solve the task.
AUTOPROMPTING METHODS
In this section, we detail approaches for tackling the general problem autoprompting before in- troducing our method for interpretable autoprompting (iPrompt) in Sec. 3.2
We specify autoprompting
discrete search problem. Given dataset of n input-output pairs {(x' y²), ..., (x", y²)} and pre-trained LLM f that returns the log bog-robbbiility of given string, goal of autoprompting is to find natural-language explanation 8 maximizing:
argmax Σ
(render(s, 2 2"))))
The render function a mroblem-specifee function that renders natural language string from the prompt and each example in the dataset (x', y²)))). We use S to indicate the set of fluent strings, under some notion of syntactic fluency. Solving this search problem exactly is intractable.
A assumption of this objective is that semantically accurate prompts lead model to assign higher probability to the data. To check this assumption, we analyze four datasets from the inverse synthetic math collection that share common structure for the inputs and prompts: each dataset admits a prompt of the form Return the _ of the inputs., then given two input numbers and queried for the output.
Fig. 2 shows the accuracy of different models at performing these tasks when given different input prompts For small models, the prompts are unsuccessful, but for large models (e.g. GPT-J 6B and GPT-3), the model is accurate if and only if given the correct prompt This result provides evidence that, at least for large models, the search for prompt that maximizes performance correlates well with the underlying task.
'The accuracy is normalized for each task using softmax in order to visualize the effect of differing prompts. For details each model, see Tahle A3

---

Under review as a conference paper at ICLR 2023
Proposal
(ii) Reranking
(iil) Exploration
the numbers
numbers
Input: 3 Output: 4
Round
Input: 7 Output:
oulput
Sum in order
Sum
input: Ouiput
Prompt:
Compute the output
Compute output
Combine the numbers
Output
Sumn the numbers
Sum the numbers
Sum the numbers
Inout: 5 Outout: 10
Round
Input: 9 Output: 12
Sum all inputs
Sum all Inputs
Output:
Prompt:
Combine the numbers
Combine arguments
Combine the arguments
Combine the numbers
Figure 3: Overview of iPrompt.
3.1 BASELINE METHODS
AutoPrompt AutoPrompt (Shin et al. 2020) targets the objective posed in Eq. using gradient- based local search. AutoPrompt searches for following the gradients of the objective Eq. ① with respect to individual tokens in š. By iteratively computing these gradients, it can discretely change individual words in & and then check whether or not the newly updated & improves the objective score. The of gradients allows AutoPrompt to find effective prompt but makes difficult to find answers that satisfy the fluency constraint S.
Average-output suffix decoding LLMs themselves can be directly used predict prompt strings. We can give the model a prompt that includes examples such as the following context string: In: 2 5 Out: 7, To compute the oulput from the imput, and sample the output for the blank
template
cover a prompt 3. Sampling directly from f helps ensure that the generated explanation is fluent and semantically meaningful. We decode the output using beam search to find the highest-probability outputs for multi-token prompts | model's context, and then average the model's output logits across all the examples in the dataset before decoding the output, an approach we refer to as average-suffix decoding. However, this technique is insufficient to find high-scoring prompts.
To improve on this approach, we place several examples into the
3.2 PROPOSED METHOD: IPROMPT
Our method, iPrompt shown in Fig- S iterative local search algorithm that alternates between three steps: (i) proposing candidate prompts. (ii) reranking candidate prompts, (iii) exploration:
(i) Proposal:
Candidate prompts are generated by extending the zero-shot LLM generation. Given data instance
prefix, sample number of candidate prompts I The maximum length of each candidate is pre-specified and fixed. For example, in the add-trss task (Fig. 3) we may generate four candidates: (Combine the numbers, Return the output, Sum in order, Compute the outpur).
Herr prefer beam search here over alternatives such nucleus sampling (Holtzman interested in finding an accurate prompt description with as few samples possible.
2019
'One could use either average suffix decoding or suffix decoding with single sample. For computational efficiency, we use suffix decoding with only single sample. We also add randomly decode the output rather than using beam-search, our iterative pracedure recover from initially finding inaccurate candidates.
4

---

Under review as a conference paper at ICLR 2023
(ii) Reranking: Given candidates, the objective Eq. [ is evaluated for each candidate prompt The top few candidates which maximize the objective are kept, e.g. narrowing down the candidates to (Combine the numbers, Sum in order).
(iii) Exploration: Each of the top candidates from reranking is truncated at a random position. These truncated candidates are used as prefix string when generating candidate prompts via suffix decoding. For example, we may randomly select the start the previous candidates and fill in the CComiine the numbers, Combine both arguments, Sum endings: {Combine the Sum -
the numbers, Sum all inputs]
The algorithm is repeated until identifying suitably strong 8, e-g. selecting Sum the numbers. Step (i) and (iii) ensure that prompts remain fluent, while step (ii) improves the score of the prompts on the objective. Computationally, iPrompt only requires running inference on the pre-trained LLM, yielding significantly lower memory requirement than methods such as AutoPrompt, which require access the LLM's gradients.
RESULTS
Accuracy prompts Table compares prompting methods based on the set of candidate descrip- tions they generate using GPT-J (a 6-billion parameter model) as the LLM (Wang & Komatsuzaki, 2021 The MRR rows show that iPrompt considerably increases the mean reciprocal rank (MRR) (Sec.
the baselines, implying that iPrompt more effectively generate descriptions that accurately reflect the underlying data pattern. The "top-prompt correctness" rows show the percent- age of datasets for which the top-ranked candidate prompt produced by each method is labeled as accurate by manual inspection (see all prompts in Appendix A.2). On the ANLI datasets, iPrompt again outperforms the baselines, although all methods perform poor an absolute sense (S 30%). The zero-shot results show the accuracy of GPT-J when using the top prompt found by each model; for the math datasets the iPrompt prompt elicits an improvement over the baselines, but for the ANLI datasets all prompts induce poor performance
Table 2:
Accuracy for dataset explanation measured via (i) MRR, (ii) top-prompt correctness, and (iii) zero-shot accuracy on unseen examples. All experiments are on GPT-J 6B. For all metrics, higher is better.
iPrompt
AutoPrompt
Average suffix
MRR
0.71
0.30
0.07
Math
Top- poppprompt correctness
80%
30%
20%
Zero-shot acc.
51.5%
41.6%
10.0%
MRR
0.30
0.17
0.01
ANLI
Top prompr correctness
30 %
0%
10%
Zero-shot acc.
4.7%
1.9%
5.1%
Qualitative assessment of top prompts Table 4 shows the top-ranked prompt generated by each method for selected datasets. iPrompt often finds a prompt that is somewhat indicative of the un- derlying relationship, even if the phrasing is not perfect. For example, for the add two numbers dataset, it finds Write a function returns the classes of the output (e.g. yes no?) rather than capturing the underlying relationship. The prompts found by iPrompt also read as coherent strings compared to AutoPrompt, which re- turns incoherent set of tokens. See all found prompts, including for average-suffix decoding in Appendix A.2
add(. For difficult datasets, the iPrompt string sometimes simply
shows the generalization accuracy Generalization of generated prompts to new models. Table when using the prompts generated using GPT-J (Table and testing them on different L.MMs The
$ Here, we restrict generated prompts tokens (sce full discussion of experimental details in Ap- pendix A.3
Accuracy is computed following Brown al. 2020
Raffel et al. 2020 using exact matching with beam
search, beam width of 4, and length penalty of
0.6
5

---

Under review as a conference paper at ICLR 2023
Table 3: Generalization accuracy (zero-shot) when testing the prompts generated with GPT-J the LLM across different models. iPrompt yields strong performance, usually improving over Auto- Prompt despite maintaining interpretability, and sometimes performing close the human-written prompt. Numbers within 2% of the top accuracy (excluding human- prompts) for each model are shown bold.
Human-writter
iPrompt
AutoPrompt
Average Sufix
No prompt
OPT 6.7B
12.7
19.3
18.9
4.5
8.4
Math
GPT 20B
76.1
54.4
23.2
21.3
8.5
GPT-3 175B
76.0
62.1
40.8
16.9
28.4
OPT 6.7B
10.7
4.7
8.5
7.9
ANLI
GPT 20B
31.0
14.2
5.6
13.7
4.0
GPT-3 175B
37.6
11.7
2.7
13.4
7.7
Table 4: Examples of generated prompts.
Human written prompt
iPrompt
AutoPrompt
Generate SQL statement from
Write SQL to produce output
ributed grandfathorExceptionaa- propri intent Lara
question asking for certain data.
You are given country name and you need to return the curreney the given country.
Select currency code for new
renciesthethe Dmitrythe mortg
ANLI
Return whether the input food dish is vegetarian (yes no).
yes or no? This
Novtherhethethethe
In this task, you are given ad- jective, and your job generate its word
What is the opposite of
prevailingthethe weakestthe wins
antonym. antonym word opposite meaning to it.
Return the sum of the inputs.
Write function int add( When you multiply two If m then subtract
addedthe +the use worked multiplythethe the Multiple opposably exactly subtractFor YEAR
Return the product of the inputs.
Return difference of the inputs.
Which number has a bigger value Return the maximum of the inputs.
Numberthcchhhhssticcaaaaaaau
Math
Return the first of the inputs.
The first digit of both values
greater name sorting indiscrim to numbers
Square the input to get the output.
Write function that calculates square
multiplythe hypot Norttheirl
Given input x, return x*
write a function called double that
ADDthc introducedpareat con- traceptives
Rotten Tomatoes
Answer Yes if the input is positive and No if the input negative.
fast, funny, highly enjoyable film. Answer: Yes 3.1/
suke Medals; does CFR Sab"|=> NormalConstructed Umbunit sat- isfy Good-ram
SST-2
> Answer the input is positive and No if the input is negative.
life Answer: (because M.LauuneeCCEddneeehhhhheeee about life) This
Holy Kubrick incorporatedher- ent#S = SPECIAL Pyth
same prompts effectively improve accuracy across different models compared to having prompt. The gap between iPrompt and AutoPrompt is larger for larger models (i.e. GPT 20B and GPT-3), suggesting that by generating fluent prompts iPrompt better captures generalizable description of the task. Human-writte written prompts still outperform the autoprompting methods on this task.
Investigating iPrompt in sentiment classification Finally, we study the more difficult task of prompting for sentiment classification, using four popular datasets (Socher et al. 2013 Malo al. 2014 Pang & Lee 2005). The aim is to find a -aeasst- prompt that can describe particular sentiment classification setting. To accommodate for complex input-output relationship, allow

---

Under review as a conference paper at ICLR 2023
Table 5: Zero-shot accuracy on sentiment classification datasets using prompts generated with the GPT-J 6B models. Evaluation is performed both on the original GPT-J 6B parameter model and testing generalization to GPT-3. The model needs to produce the correct answer (Yes, No, Maybe) out of the entire vocabulary (without rank-eval). Values are averaged over three random seeds for prompt-generation errors are standard errors of the mean.
Human written
iPrompt
AutoPrompt
No prompt
Financial phraschank
24.3
62.4 + 0.1
6.8 + 2.9
0.0
Rotten Tomatoes
44.4
70.5 + 1.4
57.1 + 3,4
0.0
GPT-J 6.7B
SST-2
53.6
82.8 + 1.9
40.0 + 7.9
0.0
IMDB
32.5
21.3 + 9.3
12.1 + 0.9
3.5
Financial phrasebank
54.1
65.0
2.7
0.4
Rotten tomatoes
58.6
52.5
37.5
0.9
GPT-3 175B
SST-2
60.4
83.6
5.2
0.6
IMDB
79.0
1.3
0.9
1.1
each method generate up to 16 tokens (our manually-writter sentiment classification prompts range from 13-16 tokens). We use Yes and No as positive and negative labels, and require the LLM generate the proper output, opposed simply ranking the two options.
Table 5 shows the zero-shot performance of the prompts elicited by different methods. Prompts are generated using GPT-J 6B and evaluated using both GPT-J 6B and GPT-3. iPrompt outperforms only AutoPrompt, but also the manually-written prompt on three of the four datasets. The exception the IMDB dataset, which has extremely long examples and may not be well suited for the zero shot setting. Accuracy is measured on the test set when available; otherwise, it is measured on held-out 25% of the train set
Table shows an example of comparing the prompts from the Rotten Tomatoes dataset, for which iPrompt and AutoPrompt induce similar zero-shot accuracy. Here and in other cases, iPrompt some- times discovers a prompt that is paraphrase of an example one would find in the training set or prototypical example for a class.
SCIENTIFIC INVESTIGATION INTO FMRI NATURAL-LANGUAGE DATASET
We now explore using iPrompt in a (very simplified) neuroscience experimental setup (Sec. 5 central challenge in neuroscience is understanding how and where semantic concepts are represented in the brain. A recent seminal study (Huth et all 2016) explores this question by investigating where different natural- natural-language categories are represented in the human neocortex. Specifically, the authors collect functional MRI (IMRI) responses as human subjects listen to hours of narrative stories. They then build predictive model of these responses for each voxel (i.e. small region in space) in the brain, which takes as input the words contained in the stories (and other features). To interpret these individual voxel models, they cluster the words in the narrative stories into 12 groups and manually annotate them, resulting in 12 categories, such as tactile, visual, and professional. Finally, they view the spatial mapping of these 12 concepts (projected onto low dimensions) across the brain using their individual voxel models.
We revisit small piece of this study's analysis through the lens of iPrompt. Specifically, we ask whether iPrompt could generate plausible categories that are well-represented across the brain but differ from the manually identified 12. We begin by fitting a predictive model for each voxel. follow- ing the pipeline of the original study (details in Appendix A.5)) We then use the resulting models to identify a list of the top- 15 words which most excite each voxel. For example, the top- 15 words that excite the best-predicted voxel are: sheet, edges, diameter, strips, cardboard, copper, steel, colored, coloured, leaf. wire, cap, paper, shaped, tin. To identify plausible semantic category, we construct template string follows: The following list of words all belong to the same semantic category: n' sheet, edges, ..., shaped, tin. We then use iPrompt (again with a GPT-6B parameter model)
Diffferent from the other experiments in this paper, we initialize AutoPrompt with random tokens instead of all the, as we find AutoPrompt fails to find an effective solution for longer prefix lengths when all tokens initialized to the.

---

Under review as a conference paper at ICLR 2023
DVD
Color
Material
Figure 4: Representations of the iPrompt-elicited concepts material (blue) and color (red) across the surface of the neocortex are spatially clustered and smooth. Only the top 10,000 pret-predicted voxels are shown, remaining voxels are shown in black. Plotted with pycortex (Gao et al. 2015).
to generate category by filling in the blank (restricted single token). To make iPrompt more effective, for each voxel we use iPrompt set of examples consisting of 15 permutations of the top-15 words, allowing the model to find patterns that are not overly sensitive to the word-ordering.
Given the top categories for each voxel, we analyze the mapping of recurring categories across the neocortex. We aggregate the top-15 inferred categorie over the top-15 best-predicted voxels and find that the most frequently inferred categories are: material, color, surface, text, fabric. Interestingly, these are sensible quantities that different voxels could reasonably be selective for. We spatially map each of these identified categories (e.g. material) across the 10,000 best best- presiipedd voxels by using the LLM in a second way. For each voxel, we condition the LLM the top-15 words list, and evaluate the predicted probability for each category, iee. The following list of words all belong to the same semantic category: sheet, edges, shaped, tin The semantic category they all belong to, in one word, is The higher this predicted probability, the more selective we infer that a voxel is for the category. Fig. shows these predicted probabilities for the top-two inferred categories (material and color) across the cortex of a human subject.
(again GPT--B)
While there is no groundtruth for this semantic map, one noteworthy feature of the resulting map that it is spatially smooth (quantitatively, Fig. A2 shows that the variance of the map among neighboring pixels is significantly lower than we would expect by shuffling the map's values). This non-trivial, as nowhere in the modeling process was spatial information incorporated: each voxel was modeled independently and the displayed prediction the underlying map to be smooth, both due to local connectivity in brain regions and also because the BOLD signal measured by fMRI does not have perfect spatial resolution. Thus, the fact that our inferred map is smooth suggests that (i) something about these categories is genuinely captured by the representation in the human brain, and (ii) that the iPrompt approach was able to reflect at least some of it. Beyond the two categories shown, the five categories generated by iPrompt exhibit spatial smoothness across the neocortex (Fig.
queried independently. We expect
6 RELATED WORK
Prompting and autoprompting. With the advent of large-scale models, prompting (i.c. finding the right prompt use to query an LLM for given task) has exploded as an area of inquiry, often yielding impressive improvements in performance (Brown et al.) 2020; Petroni et al. 2019
Liu
al.
202 and spurring a line of work aiming make prompting easier (Strobelt et 2022
We apply stemming and remove stopwords before choosing the best categories.
8

---

Under review as a conference paper at ICLR 2023
et al. 2022 Bach al. 2022; Logan IV et al. 2022). Recently, autoprompting (i.e. automatically searching for prompt or prompt-embedding via optimization) has emerged, with methods such prefix-tuning (i & Liang 202Tp, P-tuning (Liu et al. 2021b), prompt-tuning with rules (Han et al., (02T), knowledgeable prompt tuning (Hu et al. 2021) and many more (Liu et al. 2021a). These strategies use gradient descent to find set of "adapter" parameters that maximize model performance, but do not require that the new parameters map back to tokens in discrete space, rendering them uninterpretable.
A few methods tackle the more difficult problem of searching for prompts that can be expressed in natural language tokens. The closest related work AutoPrompt (Shin et al. 2020). which performs autoprompting via input gradients (see Sec. B Similarly, adversarial triggers (Wallace et al. 2019) use autoprompting to identify adversarial inputs which can be used to change a model's prediction. These methods effectively alter model's predictions, but do not constrain the discovered prompts to be semantically meaningful, resulting prompts that are difficult to interpret (Webson & Pavlick
1021 Another related work directly finetunes an LLM to describe the difference between two datasets (hong et al. 2022
Alternative methods for neural-network interpretation A popular method for interpreting neu- ral networks is inspect an LLM's individual predictions via feature importances (Lundberg al.
2019 Ribeiro et al. 2016), fcature-interaction importances (Singh et al. 2019 Tsang el al. 2017).
extractive rationales (Zaidan & Eisner 2008; [hha al.. [021)) ) or natural-language explanations for individual predictions (Hendricks et al. [016; Camburu et al. 2018). These works can provide meaningful insights for individual predictions, but it difficult parse them into an understanding of an entire dataset. Alternatively, one can investigate whether an L.M'' learned representations
2019) or by directly analyzing model's internal via probing (Conneau et al., 2018 Liu Avei
weights and activations (Wang al. 2021: Olah et al. 2018: Meng al. 2022). However, these
approaches are limited in their ability to generate previously unknown descriptions of data. A dif-
ferent approach involves distilling information into transparent model, e.g. Tan et al. (2018) Ha
et al. 2021b: Singh & Gao (2022).
Problems related to interpretable autoprompting The problem statement presented in this work closely resembles the widely studied problems of symbolic regression (Augusto & Barbosa 2000
Schmidt & Lipson 2009).. program synthesis (Gulwani et al. 2017, Manna & Waldinger 1980).
text/table summarization (Kryšcinski et al., 2019: Liu et al. 2018). and pattern discovery
data-
mining (Hnd (007). In these cases, data examples are given with the goal of inferring a symbolic
expression, program, or text summary that is consistent with the data. iPrompt can be viewed as an
algorithm for symbolic regression, in which the set of allowable symbols consists of semantically meaningful natural latural- strings and their optimization is guided by a pre-trained LLM.
7 CONCLUSION AND DISCUSSION
iPrompt makes meaningful step towards finding natural-language prompts that both (i) mantically meaningful and (ii) yield strong generalization performance. Nevertheless, the search algorithms used in this work are computationally intensive and fail to recover descriptions of com- plex datasets. Future work could explore algorithmic variants that make interpretable autoprompting more efficient and accurate.
Besides algorithmic improvements, future work could explore using iPrompt in different ways. One such direction could elicit targeted information from data via the use of a template. For example, one may use iPrompt extract feature importance by prepending the learned prompt with the string "To get the answer from the inputs, the most important inputs are " As another example, in scientific study such the IMRI study in Sec. S a scientist interested in particular topic (c.g. fear) may investigate that particular topic by making a more specific template (e.g. How are these words related the concept of "?ear"?).
While we focus on text, iPrompt could be applied more generally in any setting where an LLM performs well and takes input in sumaa-understandabbe form. For example, in computer vision, an interpretable autoprompt may look like a mask of an image, and vision-language models, an interpretable prompt may be description of vision task e.g. find the largest shape this image.

---

Under review as a conference paper at ICLR 2023
REFERENCES
Douglas Adriano Augusto and Helio JC Barbosa. Symbolic regression via genetic programming. In Proceed- ings. Vol. I. Sixth Brazilian Symposium on Neural Nenworks, pp. 173-178. IEEE, 2000.
Stephen H Bach, Victor Sanh, Zheng-Xin Xin Yong. Albert Webson, Colin Raffel, Nihal Nayak, Abheesht Sharma, Taewoon Kim, M Saiful Bari, Thibault Fevry, et al. Promptsource: An integrated development environment and repository for natural language prompts. arXiv preprint arXiv:2202.01279 2022.
Sid Black, Gao Leo, Phil Wang. Connor Leahy, and Stella Biderman. GPT-Neo: Large Scale Autoregressive Language Modeling with Mesh-Tensortun Tensorflow. March 2021. doi:
http///oooo.99/7//// URL https: If you use this software, please cite it using these metadata.
//doi. .rgg/orrg//// 10. 5281/ wwwo//eeodoo.oo 5297715
Sid Black, Stella Biderman, Eric Hallahan, Quentin Authony, Leo Gao. Laurence Golding, Horace He. Connor Leahy. Kyle McDonell, Jason Phang. et al. Gpt-neox-20b: An open-source autoregressive language model. arXiv preprint Aiv:v:2204.0000 06745, 2022.
Tom Brown, Benjamin Mann, Nick Ryder, Melanie Subbiah, Jared D Kaplan, Pratulla Dhariwal, Arvind Nee- lakantan, Pranav Shyam, Girish Sastry, Amanda Askell, et al. Language models are few-shot learners. Advances in neural information processing systems, 33:18777 1901, 2020.
Oana-Maria Camburu, Tim Rocktäschel, Thomas Lukasiewicz, and Blunsom, e-snli: Natural language inference with natural language explanations. Advances in Neural Information Processing Systems, 31, 2018.
Alexis Conneau, German Kruszewski, Guillaume Lample, Loie Barrault, and Marco Baroni. What you can cram into single vector: Probing sentence embeddings for linguistic properties. arXiv preprint arXiv:1805.0100 2018.
Jacob Devlin, Ming-Wir transformers for language understanding. arXiv preprint arX:v:1810.04805, 2018.
Ming-Wei Wei Chang, Kenton Lee, and Kristina Toutanova. Bert: Pre-training of deep bidirectional
James S Gao, Alexander G Huth, Mark D Lescroart, and Jack L Gallant. Pycortex: interactive surface visualizer for fmri. Frontiers in neuroinformatics, pp. 23, 2015.
Sumit Gulwani, Oleksandr Polozov, Rishabh Singh, et al. Program synthesis. Foundations and Trends@ in Programming Languages, 4(1-2)11-19, 2017.
Wooscok Ha, Chandan Singh, Francois Lanusse, Srigokul Upadhyayula, and Bin Yu. Adaptive wavelet distil- lation from neural networks through interpretations. Advances in Neural Information Processing Systems, 34, 2021.
Xu Han. Weilin Zhao, Ning Ding, Zhiyuan Liu, and Maosong Sun. Ptr: Prompt tuning with rules for text classification, arXiv preprint arXiv:2105.11255 2021.
David Hand. Principles of data mining. Drug safety, 30(7):621-622) 2007.
Lisa Anne Hendricks, Zeynep Akata, Mareus Rohrbach, Jeff Donahue, Bernt Schiele, and Trevor Darrell. Generating visual explanations. European conference on computer vision, pp. 3-19. Springer, 2016.
Holtzman, Jan Buys, Li Du, Maxwell Forbes, and Yejin Choi. The curious case of neural text degeneration. arXiv preprint arXiv: 190400951 09751, 2019.
Shengding Hu, Ning Ding, Huadong Wang, Zhiyuan Liu, Juanzi Li, and Maosong Sun. Knowledgeable prompt-tumin tuning: Incorporating knowledge into prompt verbalizer for text classification. arXiv preprint arXiv:2108. 02035, 2021.
Alexander G Huth, Wendy A De Heer, Thomas L Griffiths, Frédéric E Theunissen, and Jack L Gallant. Natural speech reveals the semantic maps that tile human cerebral cortex. Nature, 532(7600):453 458, 2016.
Wajciech Kryšciński, Nitish Shirish Keskar, Bryan McCann, Caiming Xiong, and Richard Socher. Neural text summarization: A critical evaluation. arXiv preprint arXiv: (xXv::908.00860 2019.
Aitor Lewkowyez, Anders Andreassen, David Dohan, Ethan Dyer, Henryk Michalewski, Vinay Ramasesh, Ambrose Slone, Cem Anil, Imanol Schlag, Theo Gutman-Solf Solo, et al. Solving quantitative reasoning prob- lems with language models. arXiv preprint arXiv:2206. 14858, 2022.
Xiang Lisa Ii and Percy Liang. Prefix-tuning: Optimizing continuous prompts for generation. arXiv preprint 101. 10000, 2021.
10

---

Under review as a conference paper at ICLR 2023
Frederick Liu and Besim Avei. Incorporating priors with feature attribution text classification. arXiv preprint arXiv:/906.0 08286, 2019.
Pengfei Liu, Weizhe Yuan, Jinlan Fu, Zhengbao Jiang, Hiroaki Hayashi, and Graham Neubig. Pre-train, prompt. and predict: systematic survey of prompting methods in natural language processing. arXiv preprint arXiv:2107.13360 13586, 2021a.
Tianyu Liu, Kexiang Wang, Lei Sha, Baobao Chang, and Zhifang Sui. Table-to-text generation by structure- aware seq2seq learning. Thirty-Second AAAI Conference on Artificial Intelligence, 2018.
Xiao Liu, Yanan Zheng, Zhengxiao Du, Ming Ding, Yujie Qian, Zhilin Yang, and Jie Tang. Gpt understands, too. arXiv preprint arXiv: 10388, 2021b.
Robert Logan IV, Ivana Balazevic, Eric Wallace, Fabio Petroni, Sameer Singh, and Sebastian Riedel. Cut- ting down on prompts and parameters: Simple few-shot learning with language models. Findings of the Association for Computational Linguistics: ACL 2022, pp. 2824-2835, Dublin, Ireland, May 2022. Association for Computational Linguistics. doi: 10.18653/v1/22 1111666/112222..nmmlmll f2n002..inin URL https:
//acl meclanthcllgg..o. orq/2022.2222.. findings-aci. 222
Yao Lu, Max Bartolo, Alastair Moore, Sebastian Riedel, and Pontus Stenetorp. Fantastically ordered prompts and where to find them: Overcoming few-shot prompt order sensitivity. In Proceedings of the 60th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers). pp. 8086-8096 Dublin, Ireland. May 2022. Association for Computational Linguistics. doi: 10.18653//1/2022..a0 hcwp://2222..clooooooaa 1ong.555... URL
20222 acl- -ooll--oong..5. 556
Scott M Lundberg. Gabriel Erion. Hugh Chen, Alex DeGrave, Jordan M Prutkin, Bala Nair, Ronit Katz, Jonathan Himmelfarb, Nisha Bansal, and Su-In Lee. Explainable ai for trees: From local explanations to global understanding. arXiv preprint arXiv: NXXKvv/995.04666 1905004610, 2019.
P. Malo, A. Sinha, P. Korhonen, J. Wallenius, and P. Takala. Good debt or bad debt: Detecting semantic orientations in economic texts. Journal of the Association for Information Science and Technology, 65, 2014.
Zohar Manna and Richard Waldinger. A deductive approach to program synthesis. ACM Transuctions on Programming Languages and Syxiems (TOPLAS), 2(1):90-121, 1980.
Kevin Meng. David Bau, Alex Andonian, and Yonatan Belinkov. Locating and editing factual knowledge in gpt. arXiv preprint arXiv:2202.0522 05262,, 2022.
Chris Olah, Arvind Satyanarayan, lan Johnson, Shan Carter, Ludwig Schubert, Katherine Ye, and Alexander Mordvintsev. The building blocks of interpretabillity. Distill, 3(3) 2018.
Bo Pang and Lillian Lec. Sceing stars: Exploiting class relationships for sentiment categorization with respect to rating scales. In Proceedings of the ACI, 2005.
Fabio Petroni, Tim Rocktäschel, Patrick Lewis, Anton Bakhtin, Yuxiang Wu, Alexander Miller, and Sebastian Riedel. Language models as knowledge bases? arXiv preprint 10X.:09909.0106, 2019.
Alec Radford, Jeffrey Wu, Rewon Child, David Luan, Dario Amodei, Ilya Sutskever, et al. Language models are unsupervised multitask learners. OpenAl blog, 1(8):9, 2019.
Alec Radford, Jong Wook Kim, Chris Hallacy, Aditya Ramesh, Gabriel Goh, Sandhini Agarwal, Girish Sas- try, Amanda Askell, Pamela Mishkin, Jack Clark, et al. Learning transferable visual models from natural language supervision. In International Conference on Machine Learning, pp. 8748-8763. PMLR, 2021.
Colin Raffel, Noam Shazeer, Adam Roberts, Katherine Lee, Sharan Narang, Michael Matena, Yanqi Zhou, Wei Li, Peter J Liu, et al. Exploring the limits of transfer learning with a unified text-to-text transformer. J. Mach. Learn. Res., 21(140):1-67, 2020.
Marco Tulio Ribeiro, Sameer Singh, and Carlos Guestrin. Why should trust you?: Explaining the predic- tions of any classifier. In Proceedings of the 22nd ACM SIGKDD International Conference on Knowledge Discovery and Data Mining, pp. 1135-1144, ACM, 2016.
Mobashir Sadat and Coreela Caragea. Scinli: A corpus for natural language inference on scientific text. arXiv preprint 2xXiv:203.066723 2022.
Victor Sanh, Albert Webson, Colin Raffel, Stephen H Bach, Lintang Sutawika, Zaid Alyafeai, Antoine Chaf- fin, Amaud Stiegler, Teven Le Scao, Arun Raja, et al. Multitask prompted training enables zero-shot task generalization. arXiv preprint arXiv:2110.08207. 2021.
11

---

Under review as a conference paper at ICLR 2023
Michael Schmidt and Hod Lipson. Distilling free-form natural laws from experimental data. science, 324 (5923):81-88 2009.
Lei Sha, Oana-Mare Maria Camburu, and Thomas Lukasiewicz. Learning from the best: Rationalizing predictions by adversarial information calibration. In AAAI, pp. 13771-13779, 2021.
Taylor Shin, Yasaman Razeghi, Robert L Logan IV, Eric Wallace, and Sameer Singh. Autoprompt: Eliciting knowledge from language models with automatically generated prompts. arXiv preprint arXiv:2010. 2020.
Chandan Singh and Jianfeng Gao. Emb-gam: an interpretable and efficient predictor using pre-trained lan- guage models. arXiv preprint arXiv:2209.1/72 11799, 2022. doi: 10.485550/mxiv.2209.1777 URL https: //arxiv.orgy wwtp:/rrgrbbbbb...r.r.g
Chandan Singh, James Murdoch, and Bin Yu. Hierarchical interpretations for neural network predictions. International Conference on Learning Representations, pp. 26, 2019. URL. httt::://openreet net
Richard Socher, Alex Perelygin, Jean Wu, Jason Chuang, Christopher D Manning, Andrew Ng, and Christopher Potts. Recursive deep models for semantic compositionality over a sentiment treebank. In Proceedings of the 2013 conference on empirical methods in natural language processing, pp. 1631-1642, 2013.
Hendrík Strobelt, Albert Webson, Victor Sanh, Benjamin Hoover. Johanna Beyer, Hanspeter Pfister, and Alexander M Rush. Interactive and visual prompt engineering for ad-hoc task adaptation with large lan- guage models. arXiv preprint 220iv:208.0078800 2022.
Sarah Tan, Rich Caruana, Giles Hooker, and Yin Lou. Distill-ans and compare: Auditing black-box models using transparent model distillation. In Proceedings the pp. 303-310, 2018.
the 2018 AAAI/ACM Conference on AI, Ethics, and Society,
Michael Tsang, Dehua Cheng, and Yan Liu. Detecting statistical interactions from neural network weights. arXiv preprint arXivv 779.77777.777..... 2017.
Eric Wallace, Shi Feng, Nikhil Kandpal, Matt Gardner, and Sameer Singh. Universal adversarial triggers for attacking and analyzing nlp. arXiv preprint arXiv: /008.00900000.2 2019.
Ben Wang and Aran Komatsuzaki. GPT-J-6B: Billion Parameter Autoregressive Language Model. https: //github. pegtthub.ccommkingooooo///ssseee http:///gpp.oilmm/meeeeeaaaaaaaa
jax May 2021.
Xingqiao Wang, Xiaowei Xu, Weida Tong, Ruth Roberts, and Zhichao Liu. Inferbert: a transformer-based causal inference framework for enhancing pharmacovigilance. Frontiers in Artificial Intelligence, 4:659622, 2021.
Yizhong Wang. Swaroop Mishra, Pegah Alipoormolabashi, Yeganeh Kordi, et al. Benchmarking generalization via in-context instructions on 1,600+ language tasks. arXiv, 2022.
Albert Webson and Ellie Pavlick. Do prampt-based models really understand the meaning of their prompts? arXiv preprint arXiv:2109.0 01247, 2021.
Omar Zaidan and Jason Eisner. Modeling annotators: A generative approach to learning from annotator ra- tionales. In Proceedings of the 2008 conference Empirical methods in natural language processing, pp. 31-40, 2008.
Susan Zhang, Stephen Roller, Naman Goyal, Mikel Artetxe, Moya Chen, Shuohui Chen, Christopher Dewan, Mona Diab, Xian Li, Xi Victoria Lin, et al. Opt: Open pre-trained transformer language models. arXiv preprint arXiv:2205.00 200.0000000 2022.
Ruiqi Zhong, Charlic Snell, Dan Klein, and Jacob Steinhardt. Describing differences between text distributions with natural language. In International Conference on Machine Learning, pp. 27099-277 -1199-2711666 PMLR, 2022.
12


Reviews:
[{"review_id": "t1W3hAz6cp", "reviewer": "Reviewer_R7d2", "summary": "", "questions": "", "limitations": "", "rating": 5, "confidence": 4, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "", "summary_of_the_paper": "This paper proposes a method named **iPrompt**  to automatically generate high-quality prompts using LLM. iPrompt iteratively mutates top prompts by (1) generating explanations with an LLM (2) reranking them based on their performance and (3) exploring new candidates. The authors verify the effectiveness of the proposed method on Math, Sentiment, NLU, and fMRI datasets.", "summary_of_the_review": "Overall, this paper investigates a practical and interesting direction that generates readable and high-quality prompts by PLM itself. The designed algorithm is well-motivated. However, the experiment part is relatively weak and cannot prove the empirical contribution of this paper.", "strength_and_weaknesses": "Strength:\n1. The paper is good writing and easy to follow. \n2. The proposed method can search readable prompts without access to LLM parameters and gradients.\n\nWeakness:\n1. The proxy task(evaluation task) is important for the prompt search, however, the authors did not investigate more on the selection of the proxy task. The current proxy task is to measure the closeness and similarity to the \"ground-truth prompt\". However, what is the definition of a \"ground-truth prompt\"? The author did not elaborate more. If it is some existing description of the task label, it may not a good prompt to serve as a \"ground-truth\" prompt. Did the author explore other evaluation methods to select good prompts?\n2. Some results(e.g. Table5) are not convincing. As the PLM zero-shot performance is sensitive to the prompt, the authors should report the average accuracy and std across several prompts when compared with other baselines. The Human-written performance of Rotten Tomatoes/SST-2/IMDB seems too low, as I have simply tried several manual prompts on GPT2-xl(1.3B)  and achieve much higher accuracy. (i.e., the IMDB accuracy of prompt *' The IMDb movie review in negative/positive sentiment is: \"<S1>\"'*  is 83.4)\n3. The compared baseline is weak. More recent baselines should be considered, e.g., Making pre-trained language models better few-shot learners(https://arxiv.org/pdf/2012.15723.pdf)\n\n\n\n", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {"correctness": "3: Some of the paper’s claims have minor issues. A few statements are not well-supported, or require small changes to be made correct.", "technical_novelty_and_significance": "3: The contributions are significant and somewhat new. Aspects of the contributions exist in prior work.", "empirical_novelty_and_significance": "2: The contributions are only marginally significant or novel."}, "extra_text": {}}, {"review_id": "qbCcD8spF1", "reviewer": "Reviewer_a8A9", "summary": "", "questions": "", "limitations": "", "rating": 3, "confidence": 4, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "", "summary_of_the_paper": "This paper proposes a method called iPrompt to achieve interpretable autoprompting, that iteratively generates possible explanations of the input data, rerank them, and finally generate new explorations via truncating generated candidates and using them as prefixes to generate new candidates.\n\nThe authors show experiments over Synthetic Math and ANLI where the proposed approach outperforms AutoPrompt and Average Suffix decoding. On sentiment classification datasets the authors show quite competitive performance of iPrompt compared to human-written prompts.", "summary_of_the_review": "Overall the problem studies is interesting and the method is somewhat novel, but the experimental results are fairly weak and the interpretability of the generated prompts are not well supported. Thus I recommend rejection.", "strength_and_weaknesses": "Strengths:\n- The problem of auto generating prompts/explanations given a dataset is important to the research community on prompting/language models.\n- The idea of generating prompt candidates and then reranking/refining them as optimization is somewhat novel.\n\nWeakness:\n- The experiment results are relatively weak.\n1) the main results are only shown for two synthetic tasks, math and ANLI, and are only compared to two relatively weak baselines, AutoPrompt and suffix decoding. Why there is no human-written baselines? In addition, there are more recent methods that try to find better discrete prompts like:\n\na) Gao et al. Making pre-trained language models better few-shot learners. ACL 2021.\n\nb) Deng et al. RLprompt: Optimizing discrete text prompts with reinforcement learning. EMNLP 2022. \n\nc) Schick et al. Automatically identifying words that can serve as labels for few-shot text classification. COLING 2020.\n\nwhich should be cited and compared with if possible.\n\n2) Table 5, on the more realistic task sentiment classification, iPrompt was compared with human-written prompts which shows a relatively better performance. How are the human-written prompts determined? Existing work has shown that instructions/prompts have a large effect on the final task performance, hence for a fair comparison the authors should show best/worst/avg results w.r.t. multiple possible human-written prompts, e.g., from PromptSource (https://github.com/bigscience-workshop/promptsource) where multiple prompts are crowd-sourced for various tasks including sentiment classification.\n\n3) In Table 3, for ANLI, iPrompt is on par with average suffix decoding. Is there any analysis to explain why it can't achieve a better performance on this task?\n\n- All the experiment results are shown on relatively easy tasks (where the prompt can be easily defined/written). Can the authors show results on more complex NLP tasks? E.g., question-answering, natural language inference (the actual task instead of the synthetic one used in the paper). This could better demonstrate how the proposed method is able to generate more complex prompts and understand \"what each task is actually doing\" rather than just providing label choices.\n\n- Better interpretability of the generated prompts is a major claimed contribution of this paper, but it's not well supported and there is no associated study on this. Can the authors provide a human eval to show the generated prompts indeed have better interpretability (and how close it is to human-written prompts)? Based on Table 4, it's really hard to tell how interpretable the iPrompt-generated prompts are, since many of them do not make much sense to humans, e.g., \"Select currency code for a new\" (truncated randomly), \"What is the opposite of 1\" (what is \"1\"?). Especially for the more realistic task sentiment classification, the prompts generated by iPrompt do not seem meaningful at all.\n\n- Some experimental details are not clear. What is $n$ used in experiments for each task? Is there an ablation study on $n$ to show how many samples are needed for iPrompt to perform well?", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {"correctness": "2: Several of the paper’s claims are incorrect or not well-supported.", "technical_novelty_and_significance": "3: The contributions are significant and somewhat new. Aspects of the contributions exist in prior work.", "empirical_novelty_and_significance": "3: The contributions are significant and somewhat new. Aspects of the contributions exist in prior work."}, "extra_text": {}}, {"review_id": "QCgcuOjRvUy", "reviewer": "Reviewer_5C3P", "summary": "", "questions": "", "limitations": "", "rating": 3, "confidence": 4, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "", "summary_of_the_paper": "This paper attempts to find interpretable prompts for the few-shot tasks. An iterative local search algorithm is proposed. The algorithm exploits the ability of LLMs to provide interpretable prompt. The algorithm is evaluated on some simple datasets.\n", "summary_of_the_review": "Overall I feel that the problem is not clearly defined. This further led to confusion in the experimental design and inappropriate selection of baselines.", "strength_and_weaknesses": "Strength:\n1. According to Table 4, the prompts found by iPrompt are interpretable.\n2. According to Table 3, the prompts found by iPrompt have higher generalization ability than baselines.\n\nWeak.\n1. the problem is not well-defined. In particular, how to assess the interpretability of prompts? The metrics proposed in this paper only consider \"whether the generated explanation contains one of a set of problem-specific keywords\". I would expect a more reasonable metric.\n2. Whether the algorithm is reasonable is not explained. (1) The iteration does not always converge. (2) Why does the algorithm need multiple iterations? I do not understand the necessity of this iteration. (2) How to ensure that the model generates explanations for the relationship between input and output in the proposal step, e.g. in Fig 3, I don't understand why the LLMs generate \"combine the numbers\".\n3. I have doubts about some experiments. I think some baselines are designed incorrectly. For example, (1) why is the accuracy of GPT-3 only 28.4 for the simple Inverse synthetic math task and 0.6% for the SST-2 task? I think the authors deliberately chose the setting that favors their method. (2) AutoPrompt apparently does not have the ability of generalization across models, so it is not suitable as a baseline for Table 3.\n(4) The task involved in the experiment is very simple. I think this means that the method proposed in this paper is not suitable for solving complex tasks.", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {"correctness": "2: Several of the paper’s claims are incorrect or not well-supported.", "technical_novelty_and_significance": "2: The contributions are only marginally significant or novel.", "empirical_novelty_and_significance": "2: The contributions are only marginally significant or novel."}, "extra_text": {}}]

Author rebuttal:

Decision: Reject