ReadPrompt: A Readable Prompting Method for Reliable Knowledge Probing
Authors: Zezhong WANG
Luyao YE
Hongru WANG
Wai-Chung Kwan
David Ho
Kam-Fai Wong
Venue: EMNLP 2023 Findings (2023)
Keywords: Prompt
Pre-trained Language Model
Readability
Knowledge Probing
Fact Retrieval
LAMA Dataset.
TL;DR: We propose a method for automatically searching for word combinations to form sentences as readable prompts, aiming to probe the knowledge embedded in PLMs. Compared to existing methods, our approach offers higher accuracy and more reliable results.

Abstract: Knowledge probing is a task to assess the knowledge encoded within pre-trained language models (PLMs) by having the PLM complete prompts such as "Italy is located in \_\_,". The model's prediction precision serves as a lower bound for the amount of knowledge it contains. Subsequent works explore training a series of vectors as prompts to guide PLMs towards more accurate predictions. However, these methods compromise the readability of the prompts. We cannot directly understand these prompts from their literal meaning, making it difficult to verify whether they are correct. Consequently, the credibility of probing results derived from these prompts is diminished. To address the issue, we propose a novel method called ReadPrompt, which aims to identify meaningful sentences to serve as prompts. Experiments show that ReadPrompt achieves state-of-the-art performance on the current knowledge probing benchmark. Moreover, since the prompt is readable, we discovered a misalignment between constructed prompts and knowledge, which is also present in current prompting methods verified by an attack experiment. We claim that the probing outcomes of the current prompting methods are unreliable that overestimate the knowledge contained within PLMs.

ReadPrompt: A Readable Prompting Method for Reliable Knowledge Probing
Zezhong Wang 12, Luyao Ye Hongru Wang 1.2 Wai-Chung Kwan David Hol Kam-Fai Wongl.2 "The Chinese University of Hong Kong, Hong Kong, China
MooE Key Laboratory of High Confidence Software Technologies, China *Central China Normal University, Wuhan, China
{zzwang, [Twong)ese.eo cuhk. hk, luyaoye@ccnu.edu enu.cn
Abstract
Fact: Iitaly (ehject)
located--npeetring
[urope
Knowledge probing is task to assess the knowledge encoded within pre-trained lan- guage models (PLMs) by having the PL.M com- plete prompts such as "Italy is located in _ model's prediction precision serves lower bound for the amount of knowledge it contains. Subsequent works explore training a series of vectors as prompts to guide PLMs towards more accurate predictions. However, these methods compromise the readability of the prompts. We camnot directly understand these prompts from their literal meaning, mak- ing it difficult to verify whether they are cor- rect. Consequently. the credibility of prob- ing results derived from these prompts is di- minished. To address the issue, we propose a novel method called READPROMPT, which aims to identify meaningful sentences to serve prompts. Experiments show that READ- PROMPT achieves state-of-the-arr performance on the current knowledge probing benchmark. Moreover, since the prompt is readable, dis- covered misalignment between constructed prompts and knowledge, which is also present in current prompting methods verified by attack experiment. We claim that the probing results obtained from the current prompting methods are unreliable and tend to overstate PLM's actual knowledge.'
LAMA Prompt
Asia
Italy is located in
X
Auto Prompt
Italy national art Petroleum outside
Europe
Soft Prompt
Europe
Italy V1
Europe
Italy is part continent of
Prompt
Prompts
PLM Outputs
Figure example of prevalent prompting methods with the fact (Italy, located-in, in, Europe), where LAMA prompt is constructed manually. The different parts of prompts are highlighted bold.
text in the rightmost column represents the output of the language model.
PLMs is crucial. One approach is to have the PLM complete prompts about factual knowledge (Hu et al., 2022: Chen et al., 2022; Petroni al., 2019). For instance, given the fact "(Italy, located-in, Eu- rope)," a prompt can be constructed by masking the object, like "Italy is located in [MASK]." If the PLM fills in "Europe" for the mask, it is considered to know the fact. The precision of PLMs on such factual questions serves lower bound for the amount of knowledge they encode.
1 Introduction
Pre-trained Language Models (PLMs), such as BERT (Devlin et al., 2018), contain factual knowl- edge from their training data. Recent studies plore the possibility of replacing traditional knowl- edge bases (KBs) with PLMMs (Safavi and Koutra, unclear
As the PLM is sensitive to the context, recent studies investigate ways to construct better prompts that can trigger the PLM to answer more factual questions. Shin et al. (2020) proposed a search- based method that automatically selects discrete words from the vocabulary to formulate prompts. Some works further believed that prompts are not necessarily composed of actual words and pro- posed tuning-based method optimize se- quence of vectors as prompts, i.c., continuous prompts (Liu et al., 2021; 1; Qin and Eisner, 2021;
2021; Petroni et al., 2019). However, if a PLM pre-trained on a corpus contains suffi- cient knowledge compared to a KB. Therefore, un- derstanding and probing the knowledge stored in
Equal contribution.
'The code is available at htaps:://githhn com/ MM-WANG/ReadPrompt

---

Zhong et al., 2021).
final prompt. The experimental results demonstrate that READPROMPT achieves state-of-the-arl per- formance on the current knowledge probing bench- mark. Furthermore, READPROMPT significantly improves the readability of prompts compared to other automated prompting methods. Upon ex- amining the constructed prompts, we found that prompts contradicting facts surprisingly led the PLM to answer more questions correctly. This in- dicates that PLMs have confusion in understanding certain knowledge, and the knowledge detected by these incorrect prompts should not be considered as knowledge mastered by PLMs. We further verified through attack experiments that the probing results of current prompting methods also contain knowl- edge that PLMs are confused about, leading to an overestimation of the PLMs' knowledge volume.
Although these prompts can encourage PLMs to answer more factual questions, compared to man- ual prompts, they sacrifice the readability of the prompts themselves, which reduces the reliability of their probing results. Due to the entire prompt series of
being replaced with disjoint tokens vectors, it is hard to discern the literal meaning of these prompts. For instance, in Figure 1, given "Iaay national art Petroleum outside" as a prompt, the PLM outputs the matching object "Europe." Nevertheless, it is difficult to claim that the PLM knows the fact (Italy, located-in, Europe) based on this prompt. The same issue arises with the SOFTPROMPT (Qin and Eisner, 2021) illustrated in Figure 1, which uses vectors prompts. Con- sequently, it is unreliable to judge whether a PLM knows a particular fact solely based on whether its output matches the object. We believe that ensuring the consistency between the prompt and the fact is a prerequisite for knowledge probing, and read- ability is fundamental for examining the prompt's meaning.
Related Work
Knowledge Probing, first introduced by the LAMA benchmark (Petroni et al., 2019), aims to assess the volume of factual information contained within PLM. In LAMA, facts are represented triples (Subject, Relation, Object) and are sourced from various databases such as Wikidata, Concept- Net (Speer al., 2012), and SQuAD (Rajpurkar et al., 2018). We follow recent knowledge probing research (Jiang et al., 2020; Shin et al., 2020) by focusing on the T-REx split (Elsahar et al., 2018), which comprises up to 1000 fact triples for each of the 41 Wikidata relation types. Each relation is associated with human wwitten prompt, for exam- ple, "[X] is located in [Y].", where [X] and [Y] are placeholders for subjects and objects. Given the prompt with subject, the PLM needs to predict a word for [Y]. its prediction matches the golden object, we claim that the PLM encodes information about the fact (Zhong et al., 2021). The PLM's pre- diction precision can be served as a lower bound on the amount of factual information it encodes. Prompting methods can be categorized into two types based on their composition: discrete prompts and continuous prompts. Discrete prompts are formulated by real words. For example, LPAQA (Jiang et al., 2020) mined prompts from Wikipedia. Several works proposed gradient-based method to search disjoint words as prompts (Wallace al., 2019; Shin et al., 2020). While continuous prompt is composed of trainable vectors sampled from word embedding space (Liu et al., 2021; Li and Liang, 2021; Qin and Eisner, 2021; Zhong et al.,
To this end, we aim to construct prompts with good readability for knowledge probing that allows humans to determine whether the combination of the PLM's output and the prompt constitutes a fact. Referring to Molnar (2020), we define readability as the degree to which it is understood by humans. However, assessing readability is challenging due its subjective nature. To maintain fairness, use the perplexity score (Shannon, 1951) of the PLM itself as metric. Intuitively, lower scores in- dicate that the prompt is closer to natural language and casier to understand (Gonen et al., 2022).
To enhance the readability of prompts, we pro- pose READPROMPT, method for searching sen- tences to construct prompts. Following the idea of Shin et al. (2020), we use first-order estima- tion predict the change in classification loss for each word in the vocabulary when selected into the prompt to generate a golden answer. We then select the word that most reduces the loss into the prompt. Moreover, we introduce the perplexity of the PLM as constraint, necessitating that the se- lected word reduces the overall perplexity of the prompt, ensuring its consistency with the current context and maintaining the readability of the en- tire prompt. We iteratively update each word in the prompt, repeating the process until all selected words remain unchanged, ultimately obtaining the

---

2021). In that way, the tokens in the prompt are not necessarily related to actual words. On the one hand, it brings higher degrees of freedom for prompts, which leads to better results. On the other hand, those prompts become completely unknown to humans (Khashabi et al., 2022).
3.2 Readable Prompt
READPROMPT strives to achieve two objectives: guiding PLMs to make more accurate predictions and ensuring the readability of the prompt. To accomplish these goals, we design two loss func- tions that assess the capability and readability of the prompts.
3 Methodology
3.2.1 Capability Loss
3.1 Problem Definition
With a good prompt, the PLM tends assign high probability to the correct word fill the [MASK]. Hence, cross-entropy loss is applied to the PLM's prediction of the [MASK]:
We first make clear the definition of the prompt. A prompt p is composed of three parts, includ- ing an input v, k trigger tokens f,, tt-- and a symbol of mask, i.c. = [z, t tg, [MASK]] (See Figure for an example). Fed the prompt p. the PLM F-)) predicts the word distribution for [MASK]:
L Σ log P(MASK] =y (0) (3)
Intuitively, a better trigger word can lead to a lower loss. However, replacing the trigger word with all possible words and computing loss are time- consuming. Hence, we adopt the idea of Shin al. (2020) that computes first-order approximation of the change in the loss by swapping the j-th trigger token tj with another token wy E V:
F(p) = P(MMASK] = =wlp), we E V
where V is the vocabulary that contains n unique words wy, The word with the largest probability is the prediction, i.e., w* = ougmaxx P(MASK] wclp)
As for the task of knowledge probing (Petroni al., 2019), fact is represented by triplet (x, r, y), where and y is the object, e.g., (Italy, Located-in, Europe). For the relation r, we design a prompt that con- tains the subject x and say that the PLM knows this fact if its prediction is true, i.e., w" = y. for the given p. Hence, the first goal is to find sequence of trigger tokens prompt that can make the PLM generate more ac- curate predictions for the given n facts, i...
of (w:)) = Vaaa.o
the subject, r' is the relation,
where e; is the word embedding of we And in this place, we take the i-th column of the weight in the embedding layer of the PLM as the word embedding 4, We prefer words that can cause the loss to have a large negative change.
[t,,.... tx. to formulate
The green part of Figure 2 illustrates this process. The PLM predicts the word distribution ý E R to the [MASK] and calculates the cross-entropy loss following Equation (3). Then step one backward (1)
max Σ
(1x = 10
computes the gradient V Cee E RPI and step two estimates the change of the loss Ao (w:) E RD
1
Besides the capability, the second goal is to en- sure the readability of the searched prompt. In this work, we evaluate the readability from the aspect of perplexity. We define the perplexity (PPL) of the bidirectional language model to given prompt as:
3.2.2 Readability Loss
As the prompt should be readable, we need to eval- uate whether the selected trigger words t,,... txk can formulate a meaningful sentence combined with the input words. Perplexity can be regarded as a metric to evaluate the rationality of sentences (Lee et al., 2021). Intuitively, well-trained PLM has low perplexity in fluent sentence. We calcu- late the perplexity according to Equation 2. We only consider the perplexity of Lj that is needed to update. Specifically, we compute the perplexity of l after being replaced with wi. Therefore, Equa- tion 2 degrades to the reciprocal of the probability:
PPL(p) = exp {
2-2
1
log p(p:lp/m))
1+2 1=1
(2)
where p; is the i-th token in the prompt and P/p. is the context of the i-th token. The value of k + 2 arises from taking into account both the subject and object during PPL calculation. For left-to-right PLMs such as LLaMA (Touvron et al., 2023), the calculation of PPL adheres to the standard defini- tion.
PPL, (w; p) = P(t, = w/lp/pp)- (4)

---

PPL
Loe
Vocabulary
backward
01
2
2. estimate
4. rank
part
Embedding (e, .... C.)
ALi (e:) ef Vellee 1-u
situated
1
3. combine
located
[Italy]
[T]
[T]
[T]
[MASK]
Relation: located-in
5. replace
Trigger words search of READPROMPT. This case is about the location of Italy. Taking the second trigger word search an example, compute the perplexity and cross-entropy loss according the word distribution fa and ÿ respectively. Then we compute the gradient according to the loss backward propagation, estimate the change loss, weight combine the loss with perplexity, rank words accordingly, and select the word with minimum loss as the new for the second trigger word.
Figure? 2:
The PLM assigns high probability wi if believes that putting u, into the position of to is reasonable for the whole sentence so that the perplexity of wy is small. From a practical view, P(t, = w))p) pp) is part of the output of PLMs which does not introduce extra computation.
ratio of 3:1:1 following the same setting Shin et al. (2020).
Prompts Comparison In this experiment, we compare READPROMPT with six prompts, includ- ing three discrete prompts LAMA (Petroni et al., 2019), LPAQA (Jiang et al., 2020), AUTOPROMPT (Shin et al., 2020), and three continuous prompts P-tuning (Liu et al., 2021), OPTIPROMPT (Zhong et al., 2021), SOFTPROMPT (Qin and Eisner, 2021), In particular, the prompts of LAMA are designed manually.
The yellow part of Figure 2 shows example of updating the trigger word f22 The f2 RPI is the word distribution predicted by the PLM. Then we calculate the perplexity for each word w; following Equation (4).
Pre-trained Language Models We apply prompts above with RoBERTa (roberta-base) (Liu et al.. 2019), and two variants of BERT (De- vlin et al., 2018), bert-base-cased and berta-large- cased. We further investigated the impact of READ- PROMPT on large language models (LLMs) by comparing LLaMA-7B (Touvron et al., 2023), Alpaca-7B B (Taori et al., 2023), and Vicuna-7B (Chi- ang et al., 2023). (Ouyang et al., 2022) and GPT-4 (OpenAI, 2023) combined with manual prompts (LAMA) as a base- line for comparison. Since these two models have not been open opunesourced, we did not construct READ- PROMPT for them.
3.2.3 Trigger Tokens Selection
We define the complete loss for swapping 1j with w as:
C(w,, p) a1 AU) + PPLy (w.,p
where a is a hyperparameter to control the effect of the perplexity. Theoretically, a large a leads to a more readable prompt but the effectiveness may not be optimum. We can compute L(w;, p) for all wo E V simultaneously and then rank the words according to the loss value. After that, we select & words with the smallest loss value as candidates for the trigger token lj. For each candidate, we take quick evaluation for the updated prompt the development according to Equation (1). The prompt that can retrieve most facts is selected as the new one.
addition, we used GPT-3.5
Evaluation Based on the LAMA knowledge probing framework, we construct prompts from facts using different prompting templates and query the PLMs. The objects in the facts serve ground truth for calculating the precision @ of the PLM responses. Moreover, we compute the perplexity according to Equation as a metric to assess the readability of the prompts.
4 Experiments
4.1 Settings
Dataset We conduct experiments on T-REx, sub- set of LAMA (Petroni et al., 2019). We split the data into training, development and test sets with
Searching Details READPROMPT There are several hyperparameters needed to be declared. Be- fore trigger words searching, we need to define

---

BEse
BERT-Large
RoBERTa
LAMA
Read.
Attack
P@1 PPL
P@1 1 PPL. P@1 1 PPL
P@I
PPL
P@1
PPL
P@ 11 PPL
LAMA
26.4
23.7 32.3
19.6 24.7 23.2
LLaMA
33.88 21.44 58 38.06 101.50 1.62
22.33
LPAQA
31.2
25.7
39.4
23.2
26.6t
29.3
Alpaca 36.65 19.32 58.88 114.23 1.65
20.85
Auto.
43.3
962.5
45.71
869.2
40.0
1006.1
Vicuna 30.79 20.71 57.93
103.86
2.33
22.09
Read.
52.8
305.3
53.1
222.6
49.2
299.4
GPT-3.5
87.02 N/A
N/A
N/A
0.00
N/A
+align
48.7 105.3
49.8
96.6
48.1
117.0
GPT-4
89.90 N/A N/A N/A
0.00
N/A
P-tuning
48.3
N/A 50.6 N/A
49.3
N/A
Table 2: Comparison results of P@ 1 (%) and PPL for different L.LMs on the T-REx dataset. The best results in each column have been highlighted in bold. The "Attack" column displays the results of LLM on attack samples. In this column, P@) represents the confusion rate, which is the precision of the model's output matching the object in the attack samples. A lower value is preferable.
Opti.
48.6
N/A
50.3
N/A
44.4
N/A
Soft.
50.7 N/A 51.6 N/A
40.6 N/A
Table 1: Comparison results of P@1 (%) and PPL for different prompts on the T-REx dataset. The best results in each column have been highlighted in bold. marks baseline results obtained from our reimplementations. The row of "Read." is our method. "+align" represent the results "ReadPrompt" after alignment.
4.2 Main Results
Table shows the results about precision @ and perplexity on T-REx dataset. As the continuous prompt is not composed of the actual words, we cannot compute their perplexity. So fill the corresponding value with N/A in Table 1. It is worth noting that the row marked "+align" in the table refers to the results of READPROMPT after applying our alignment strategy.
a template to indicate the number of the trigger words and the combination order among the object, subject, and trigger words, e.g..
[X] [T] [T] [T] [Y]
After that, we fill [X] and [Y] with the subject and object from the dataset. [T] remains the initial token of the trigger words. Then we apply READ- PROMPT to search for better trigger words in a sequential manner. Once a trigger word replaced with others, the context changes accordingly. may appear a better choice for the other trigger word under the new context. Therefore, we repeat this process iteratively until no better trigger tokens appear. We conduct grid search (see Appendix A) for and set it with 0.7 to achieve the best trade-off between capability and readability.
We can see that READPROMPT achieves the best results on all BERT-based PLMs and get over 2% improvements compared with the current state-of- the-art method SOFTPROMPT. Though it cannot surpass P-tuning when prompt RoBERTa, the gap is less than 0.1%. We further compare READ- PROMPT with baselines on Google-RE, Concept- Net, and UHN. The results show that though READ- PROMPT is composed of discrete real words, it can also achieve comparable results with contin- uous prompts (See Appendix B). On the other hand, READPROMPT decreased around 70% per- plexity over AUTOPROMPT. Despite the fact that the perplexity of the prompts searched by READ- PROMPT is higher than manual ones, i.e., LAMA and LPAQA, they are readable enough for humans to understand (See case study in section 4.3).
We conducted case-by-case analysis of the search results for READPROMPT and discovered that some prompts, although readable, deviate sig- nificantly from the relations defined in the knowl- edge. We refer to this issue as the misalignment of prompts and knowledge. To address this, we developed an alignment strategy based on the loss defined in Equation 3. During each search, we recorded the top 5 prompts that minimized the loss and caleulated the BertScore (Zhang et al., 2020) between each prompt and the manual prompt. We then selected the prompt with the highest BertScore as the aligned prompt. A higher BertScore in- dicates that the READPROMPT is more closely aligned with the manual prompt.
We further compared READPROMPT and LAMA Prompt with large language models (LLMs). To the best of our knowledge, we are the first to duct knowledge probing on LLMs using the T-Rex benchmark. It should be noted that the LLMs used in this experiment, unlike the BERT models from previous experiments, are all left-to-right language models. As a result, some prompts with objects in

---

Relation
Method
Prompt
P@1
PPL
LAMA
[X] is member of [Y].
13.14
8.19
P463
Auto.
[X] participated uncredited Millennium scarce of [Y].
43.80
1809.30
Read.
a project member of [Y].
45.26
15.68
LAMA
The original language of [X] is [Y].
39.00
11.01
P364
Auto.
[X] >hanna siblings speak panoramic [Y]-
39.00
526.07
Read.
[X] written has lyrics in [Y].
45.50
78.92
LAMA
[X] is subclass of [Y]-
29.00
5.31
P279
Auto.
X
polarppedpiconized [Y].
51.50
5848.57
Read.
a type of [Y].
48.00
7.71
LAMA
X and [Y] are twin cities.
2.79
15.68
P190
Auto.
X departed Istanbul microwave Marcos departed [Y].
0.00
3217.17
Read.
X airport flight then to [Y].
12.35
257.28
LAMA
X
is developed by [Y].
46.43
13.77
P178
Auto.
X product Sega merged Microsoft versus [Y].
62.50
748.23
Read.
X was a developed console by [Y].
67.86
23.83
LAMA
X plays [Y].
8.00
149.20
P1303
Auto.
X Ballet performances concepts radar versus [Y].
19.00
1815.02
Read,
[]] and concert player playing [Y].
22.50
198.27
LAMA
[X] is the capital of [Y].
38.33
6.39
P1376
Auto.
XX] previously olds nominally predominantly called [Y].
55.00
13345.02
Read.1
X has province called [Y].
58.66
19.78
Read.2
X capital of county of [Y].
53.23
8.51
Read.3
the city of [Y].
45.00
17.78
Table 3: The comparisons among the prompts from LAMA, AutoPrompt, and EXPROMPT terms of the precision@1 @1 and perplexity. We mark the keywords in blue, evidence in orange, and the false prompt in red.
the middle position, such as "[X] plays [Y] music," are suitable for these models. In left-to-right language models, the text following the predicted object [Y] does not contribute to the model's s pre- diction. Consequently, we modified eight prompts to place the object at the end of the prompt. Details are presented in Appendix C.
sion@ I scores of 87.02% and 89.90% respectively, These scores significantly surpass those of other models, representing the current state-of-the-art. The impressive results, obtained without modify- ing the prompts and solely relying on manually preset prompts, demonstrate not only the exten- sive knowledge embedded in both models but also their remarkable capacity to comprehend human language.
As shown in Table 2, Alpaca and Vicuna, which are fine-tuned from LLaMA, exhibit differences in their responses manual prompts. We argue that fine-tuning neither injects removes knowl- edge from the original model (LLaMA); instead, it leads alterations in the model's response to human language, thereby raising or lowering the lower bound of the probed knowledge encoded by the model. READPROMPT significantly improves the results of all three models. The performance of the three models combined with READPROMPT shows significant difference. suggesting that the amount of encoded knowledge in the three models is nearly identical. This shows that READPROMPT can effectively probe the maximum knowledge en- coded in different models. GPT-3.5 and GPT-4, when utilized with manual prompts, achieve Preci-
4.3 Case Study
Table 3 shows the prompts constructed by LAMA, AUTOPROMPT, and READPROMPT. Overall, READPROMPT significantly improves the readabil- ity of the searched prompts compared with AU- TOPROMPT. We can roughly know the meanings of the searched prompt, though they are not fluent enough. Besides that, we have three findings.
(1) READPROMPT retrieves prompts with higher readability. Although the prompts may not be strictly grammatically correct, they do not hinder human understanding. Most prompts are semanti- cally close to human suman-gennrated prompts and con- tain some the same keywords (as shown in blue

---

ID
Relation
P527
1.8%
34%
Original
PS27
[X] consists of [Y].
P407
20%
46%
Reversed
New Discovered
P407
[]] was written in [Y].
P364
12%
57%
P361
40%
58%
P364
The original language of [X] is [Y].
P30
18%
64%
P361
[X] is part of [Y].
P176
12%
78%
P138
4.3%
80%
P30
[X] is located in [Y].
P1376
27%
88%
P176
[X] is produced by [Y].
P136
17%
90%
P138
[X] is named after [Y]
P103
17%
94%
P1001
61%
99%
P1376
[X] is the capital of [Y].
100
P136
[X] plays [Y] music
Accuracy
P103
The native language of [X] is [Y].
Figure 3: The precision with original prompts (orange bar) versus attack prompts (blue bar). Notably, some facts are newly retrieved by the attack prompt but not by the original, and we highlight these parts in cyan.
P1001
[X] legal term in [Y]
Table 4: The selected 11 types of facts with asymmetric relations for attack experiments.
knowledge. However, due to the unreadability of these prompts, it is challenging to make intuitive judgments based on their literal meanings. To ad- dress this, we design an attack experiment to detect misalignment issues in unreadable prompts.
font).
(2) Part of prompts constructed by READ- PROMPT have a difference from the predefined relations the facts. For example, P190 describes a twin-city relation between the subject and object, while READPROMPT describes a flight-related rela- tion between them, which is a clear misalignment. Other examples include relations P364 and P1376, where marked misaligned prompts in orange.
4,4 Attack Experiment
In section 4.3, we demonstrated that some prompts construct relations that are semantically opposite to those defined in the facts. However, PLMs still generate a large number of matching subject-object pairs. This indicates that PLMs tend to output one when they see the other, regardless of the con- text. We believe that PLMs indeed capture the Co-occurrence relationship between subjects and objects, although the specific relationship may not be clear.
(3) A select few of the prompts even contra- dict the predefined relations in the facts. For in- stance, P1376 describes the subject as the capital of the object, while Read.1 describes the object as province of the subject. Filling the subject and object into Read.1 results in a completely wrong statement. Surprisingly, such prompts can deceive PLMs into producing 58.66% matching subject- object pairs.
We question whether unreadable prompts might implicitly change the mcaning and use co- occurrence relations to induce better results from the PLM. To test this, we generate attack samples by swapping subjects and objects, such as changing (Italy, located-in, Europe) to (Europe, located-in, Italy). We then query PLMs using these attack sam- ples and existing prompting methods. If the PLMM' response matched the swapped object in the attack sample (e.g., responding "Italy" when given "Eu- rope is located in"), it indicated confusion about the relationship between the subject and object. We calculated the Confusion Rate as the proportion of confused responses in the PLM's replies. In- tuitively, higher confusion rate is likely if the unreadable prompts are based on co-occurrence relationships.
The case study demonstrates that some prompts can be deceptive, leading the language model to generate more matching subject-object pairs. How- ever, the overall prompt may deviate from or con- tradict facts. Such results should not be considered as knowledge encoded by the language model. This indicates that simply striving for high accuracy may not yield effectiveness in certain cases. Before serting whether the language model knows fact, it is essential verify if the prompt used aligns with the knowledge. READPROMPT, due its readabil- ity, provides humans with a direct way to assess the alignment of the prompt. This helps to miti- gate potential issues arising from overestimating the knowledge storage capacity of PLMs due to misleading prompts.
We use SOFTPROMPT and BERT as an exam- ple and select 11 types of facts with asymmetric relations for attack experiments (details are shown
Further, we consider there are potential mis- alignment issues between unreadable prompts and

---

Read.
On the other hand, the 7B-sized model shows con- fusion regarding certain facts, suggesting that it might be more susceptible to adversarial attacks or misinformation.
LAMA
Soft.
+align
Precision @1
47.59%
71.63% %
63.27%
Confusion Rate
1.18%
25.93%
2.77%
Difference 46.41%
55.70%
60.50%
Discussion: Stochastic Parrot or Intelligence?
Table 5: The experimental results of LAMA Prompt, SOFTPROMPT, and READPROMPT 11 types of facts. Precision@ @1 is calculated based on normal samples, while confusion rate derived from attack samples. The difference refers to the gap between these metrics.
The emergence of large language models has at- tracted widespread attention. Many studies ques- tion whether large language models are truly intel- ligent or merely a "stochastic parrot" (Chen et al., 2022) that assemble sequences of linguistic forms based probabilistic information, without any ref- erence to meaning (Jin and Rinard, 2023; Mitchell and Krakauer, 2023).
in Table 4). First, trained continuous vectors prompts via SOFTPROMPT normal samples. Then, we combined the attack samples with the prompt to query the PLM and calculated the confu- sion rate. As shown in Figure 3, the results of the attack experiment suggest that SOFTPROMPT lead to high confusion rate (at least exceeding 10%) ) One possible reason is that the unreadable prompt itself describes a co-occurrence relationship. and swapping the subject and object positions does affect the overall meaning of the prompt. Another possibility that the PLM's performance sat- isfactory, and cannot distinguish the relationship between the two.
Our experiments with READPROMPT reveal con- fusion in some knowledge triplets within the PLM. Interestingly, using prompts with opposite mean- ings can help PLMs achieve better results on cur- rent knowledge probing benchmarks. Attack ex- periments confirm that some PLMs or LLMs only capture high-frequency co-occurrence relationships between parts of the subject and object, resembling stochastic parrot. In contrast, GPT-3.5 and GPT-4 successfully resist attack samples, demonstrating sufficient intelligence in the attack experiment. Al- though some studies (Liu and Low, 2023; Chiang et al., 2023; Geng et al., 2023; Liang et al., 2022) claim that moderately guage model can achieve results comparable to GPT-4, our knowledge probing experiments reveal that these models fall short in terms of the amount of knowledge they contain (lower bound), reliabil- ity, and their adherence human prompts.
To determine whether the confusion is caused by the unreadable prompt or is inherent in the PLM itself, we further conduct attack experiments on manually constructed prompts (LAMA). Since the manual prompts accurate and reliable, the con- fusion ratio can be considered PLM's insufficient performance. As shown in Ta- ble 5, the manual prompt gains a much lower con- fusion rate. In contrast, the high confusion rate exhibited by PLMS with SOITPROMPT can be at- tributed to the prompt. We conduct the same at- tack experiment on our method, READPROMPT, for comparison. The results show that the aligned prompt does not cause more confusion, and is only slightly higher by 1.59% compared the manual prompt. For each prompt, we calculate the differ- ence between Precision @1 and Confusion Rate. To some extent, this can represent the lower bound of the knowledge encoded in the model after de- ducting the confusion cases. The results show that READPROMPT still achieves the best performance.
(e.g., 7B) large lan-
result of the
Conclusion
In this work, we introduce READPROMPT, novel method to construct prompts for the knowledge probing task. On one hand, READPROMPT guide PLMs to answer more factual questions cor- rectly. On the other hand, READPROMPT read- able, providing a way for people to verify its cor- rectness. Our case studies reveal that PLMs oc- casionally confuse facts, which should not be re- garded as their inherent knowledge. Furthermore,
demonstrate through attack experiments that existing prompting methods intensify PLMs' con- fusion of facts, leading higher but unreliable results. Ultimately, we argue that for probing tasks, a reliable outcome is of greater importance than a merely "better" one.
We conduct the same attack experiments on larger language models and the results are pre- sented in the column of "tttack" of Table 2. both GPT-3.5 and GPT-4 demonstrate robust perfor- mance by successfully evading all attack samples.

---

Limitation
Xinyang Geng, Arnv Gudibande, Hao Liu, Eric Wal- lace, Pieter Abbeel, Sergey Levine, and Dawn Song. 2023. Koala: A dialogue model for academic re- search. Blog post.
We discuss the limitations of this work from three aspects. Firstl,, ReadPrompt searches for optimal trigger words within a given template. However, these templates are currently designed by humans, which may not always be the best choice. The second problem is the trade-off between readabil- ity and capability. ReadPrompt achieves certain readability that can be guessed and understood by humans. However, it cannot compare with human expressions from a strict grammar view. This is a reluctant compromise between readability and performance. Lastly, with the increasing size of language models, searching for readable prompts is becoming more time-consuming and resource- intensive. Therefore, there is an urgent need for more efficient search indicator, especially one that doesn't 't necessitate access the model itself.
Hila Gonen, Srini Iyer, Terra Blevins, Noah A. Smith, and Luke Zeutlmoyer. 2022. Demystifying prompts in language models via perplexity estimation.
R.A. Haddad and A.N. Akansu. 1991. A class of fast gaussian binomial filters for speech and image pro- cessing. IEEE Transactions on Signal Processing. 39(3):723-727.
Shengding Hu, Ning Ding, Huadong Wang, Zhiyuan Liu, Jingang Wang, Juanzi Li. Wei Wu, and Maosong Sun. 2022. Knowledgeable prompt-tuning: Incor- porating knowledge into prompt verbalizer for text classification. In Proceedings of the 60th Annual Meeting of the Association for Computational Lin- guistics (Volume I: Long Papers), pages 2225-2240, Dublin, Ireland. Association for Computational Lin- guistics.
A Acknowledgements
Zhengbao Jiang, Frank F Xu, Jun Araki, and Graham Neubig. 2020. How can we know what language models know? Transactions of the Association for Computational Linguistics, 8:423-438.
We express our sincere gratitude to the three anony- mous reviewers whose diligent and comprehensive assessments have significantly enhanced the qual- ity of our research. This research work is partially supported by CUHK under Project No. 3230377 (Ref. No. KPF23GW20).
Charles Jin and Martin Rinard. 2023. Evidence of mean- ing in language models trained programs.
Daniel Khashabi, Xinxi Lyu, Sewon Min, Lianhui Qin, Kyle Richardson, Sean Welleck, Hannaneh Ha- jishirzi, Tushar Khot, Ashish Sabharwal, Sameer Singh, and Yejin Choi. 2022. Prompt wayward- ness: The curious case of discretized interpretation of continuous prompts. In Proceedings of the 2022 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, pages 363 3631-3643, Seattle, United States. Association for Computational Lin- guistics.
References
Xiang Chen, Ningyu Zhang, Xin Xie, Shumin Deng, Yunzhi Yao, Chuanqi Tan, Fei Huang, Luo Si, and Huajun Chen. 2022. Knowprompt: Knowledge- aware prompt-tuning with synergistic optimization for relation extraction. In Proceedings of the ACM Web Conference 2022, WWW '22, page 2778-2788. New York, NY, USA. Association for Computing Machinery.
Naycon Lee, Yejin Bang. Andrea Madotto, and Pascale Fung. 2021. Towards few-shot fact-checking via perplexity. In Proceedings of the 2021 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, pages 1971-1981.
Wei-Lin Chiang, Zhuohan Li. Zi Lin, Ying Sheng, Zhanghao Wu, Hao Zhang, Lianmin Zheng, Siyuan Zhuang. Yonghao Zhuang, Joseph E. Gonzalez, Ion Stoica, and Eric Xing. 2023. Vicuna: An open- source chatbot impressing 4 with 90%* chatgpt quality.
Xiang Lisa Li and Percy Liang. 2021. Prefix-tuning: Optimizing continuous prompts for generation. In Proceedings of the 59th Annual Meeting of the Asso- ciation for Computational Linguistics and the I Itth International Joint Conference on Natural Language Processing ( (Volume 1: Long Papers), pages 4582- 4597.
Jacob Devlin, Ming-Wei Chang, Kenton Lee, and Kristina Toutanova. 2018. Bert: Pre-training of deep bidirectional transformers for language understand- ing. Xixiv preprint 1r1iv:1810.04000
Hady Elsahar, Pavlos Vougiouklis, Arslen Remaci, Christophe Gravier, Jonathon Hare, Frederique Lafor- est, and Elena Simperl. 2018. T-rex: A large scale alignment of natural language with knowledge base triples. In Proceedings of the Eleventh International Conference on Language Resources and Evaluation (LREC 2018).
Percy Liang, Rishi Bommasani, Tony Lee, Dimitris Tsipras, Dilara Soylu, Michihiro Yasunaga, Yian Zhang, Deepak Narayanan, Yuhuai Wu, Ananya Ku- mar, et al. 2022. Holistic evaluation language models. arXiv preprint (rXiv:2111.09) 10.

---

Tiedong Liu and Bryan Kian Hsiang Low. 2023. Goat: Fine-tuned llama outperforms gpt-4 on arithmetic tasks.
Claude E Shannon. 1951. of printed english. Bell system technical journal. 30( 1450-64.
Prediction and entropy
Xiao Liu, Yanan Zheng, Zhengxiao Du, Ming Ding. Yujie Qian, Zhilin Yang, and Jie Tang. 2021. Gpt understands, too. arXiv preprint arXiv:2103. 10385.
Taylor Shin, Yasaman Razeghi, Robert L Logan IV, Eric Wallace, and Sameer Singh. 2020. Autoprompt: Eliciting knowledge from language models with au- tomatically generated prompts. In Proceedings the 2020 Conference on Empirical Methods Natural Language Processing (EMNLP), pages 4222-4235.
Yinhan Liu, Myle Ott, Naman Goyal, Jingfei Du, Man- dar Joshi, Danqi Chen, Omer Levy, Mike Lewis, Luke Zettlemoyer, and Veselin Stoyanov. 2019. Roberta: A robustly optimized bert pretraining ap- proach. arXiv preprint arXiv: 1907.11692.
Robyn Speer, Catherine Havasi, et al. 2012. Represent- ing general relational knowledge in conceptnet 5. In LREC, volume 2012, pages 3679-86.
Melanie Mitchell and David C. Krakauer. 2023. The debate understanding in Al's large language models. Proceedings of the National Academy of Sciences, 120(13).
Rohan Taori, Ishaan Gulrajani, Tianyi Zhang, Yann Dubois, Xuechen Li, Carlos Guestrin, Percy Liang, and Tatsunori B. Hashimoto. 2023. Stanford alpaca: An destucttionfollowiig llama model. https: gi thub. coom/tatsu-laat wat.tssullabstttood.... stanford_alppas paacc..
Christoph Molnar. 2020. Interpretable machine learn- ing. Lulu, com.
Hugo Touvron, Thibaut Lavril, Gautier Izacard, Xavier Martinet. Marie-Anne Lachaux. Timothée Lacroix. Baptiste Rozière, Naman Goyal, Eric Hambro, Faisal Azhar, Aurelien Rodriguez, Armand Joulin, Edouard Grave, and Guillaume Lample. 2023. Llama: Open and efficient foundation language models.
OpenAI. 2023. Gpt-4 technical report.
Long Ouyang, Jeff Wu. Xu Jiang, Diogo Almeida, Car- roll Wainwright, Pamela Mishkin. Chong Zhang. Sandhini Agarwal, Katarina Slama, Alex Ray, et al. 2022. Training language models to follow in- structions with human feedback. arXiv preprint arXiv:2203.02155.
Eric Wallace, Shi Feng. Nikhil Kandpal, Matt Gard- ner, and Sameer Singh. 2019. Universal adversarial triggers for attacking and analyzing NLP. In Proceed- ings of the 2019 Conference on Empirical Methods in Natural Language Processing and the 9th Inter- national Joint Conference on Natural Language Pro- cessing (EMNLP-JJCNIPP) pages 2153-2162, Hong Kong, China. Association for Computational Linguis- tics.
Fabio Petroni, Tim Rocktäschel, Sebastian Riedel, Patrick Lewis, Anton Bakhtin, Yuxiang Wu, and Alexander Miller. 2019. Language Models as Knowl- edge Bases? In Proceedings of the 2019 Confer- ence on Empirical Methods in Natural Language Pro- cessing and the 9th International Joint Conference on Natural Language Processing (EMNLP-IJCNLP), pages 2463-2473, Hong Kong, China, Association for Computational Linguistics.
Tianyi Zhang, Varsha Kishore, Felix Wu, Kilian Q. Weinberger, and Yoav Artzi. 2020. Bertscore: Evalu- ating text generation with bert.
Nina Poerner, Ulli Waltinger, and Hinrich Schüizee 2019, Bert is not a knowledge base (yet): Factual knowl- edge vs. name-based reasoning in unsupervised qa. ArXiv, abs/191110366
7exuan Zhong, Dan Friedman, and Danqi Chen. 2021. Factual probing is [mask]: Learning vs. learning recall. In Proceedings of the 2021 Conference of the North American Chapter of the Association for Computational Linguisties: Human Language Tech- nologies, pages 5033-
Guanghui Qin and Jason Eisner. 2021. Learning how to ask: Querying lms with mixtures of soft prompts. In Proceedings of the 2021 Conference of the North American Chapter of the Association for Computa- tional Linguistics: Human Language Technologies, pages
Pranav Rajpurkar, Robin Jia, and Percy Liang. 2018. Know what you don't 't know: Unanswerable questions for squad. arXiv preprint arXiv:1806.0333 03822.
Tara Safavi and Danai Koutra. 2021. Relational World
Knowledge Representation in Contextual Language Models: Review. In Proceedings of the 2021 Conference on Empirical Methods in Natural Lan- guage Processing, pages 1053-1067, Online and Punta Cana, Dominican Republic, Association for Computational Linguistics.

---

A Hyperparameters Grid Search
We investigate how the hyperparameters influence the results aiming suggest the best choice of the hyperparameters and gain a better understanding of the role of perplexity in searching. We conduct a limited hyperparameter grid search for each re- lation, with a E (0.0, 0.1, 0.3, 0.5, 0.7, 0.9) and with the number of trigger tokens from 3 to 15. We follow the settings of AutoPrompt and fix the number of candidates to 10. To optimize the ef- ficiency of the hyperparameter search, we apply the following settings for the maximum iteration: when the number trigger tokens stands 10 or less, we establish the maximum iteration as 500; when the number exceeds 10, we incorporate 50 ad- ditional iterations for each extra trigger token. This methodology ensures a thorough search for each token. The strategy is mathematically represented follows:
48-
66
11
the number of tr Irager toktens
Figure 4: The relation between precision @ I and the number of trigger tokens regarding different a.
example, "##ally", symbols, which leads to a high perplexity.
3200
2-533
2000
600
200
3
13
13
15
the number trigger totees
Figure 5: The relation between perplexity and num- ber of trigger tokens regarding different
We also plot the relation between capability and readability of READPROMPT in Figure 6, which can help us find the best compromise on capability and readability.
500
k < 10
iter
50k k > 10"
where & denotes the trigger token number. After determining the optimal combination of hyperpa- rameters, i.e., k 10 and 0.7, we proceed by increasing the maximum iteration to 1000. Sub- sequently, we conduct prompt searches utilizing these hyperparameters, performe evaluations, and report the results in Table 1.
40
In order to illustrate the trend clearly, we smooth the results by Gaussian filter (Haddad and Akansu, 1991) with
Figure 6: The relationship between performance and the perplexity of prompts with different a. The green point represents ideal result with the lowest perplexity and highest performance. For cach a, we mark the points closest to the ideal result with crosses.
before visualization.
We compare the performance of READPROMPT with different a in Figure 4, where a is the ab- lation study of the readability loss. As for the value of a, we suggest the best choice with 0.7. Over- all, the performance increased with But when reached 0.9, the performance decreased, which implies the impact of perplexity is overweight in the search. For the number of trigger tokens, nearly all experiments show that 10 enough. Increasing the number has little impact on the improvement results and even has a negative effect.
All hyperparameters for the other prompting methods reproduced in this work are referenced from the original papers with the recommended settings.
B More Experiment Results
Besides T-REx, we compare READPROMPT with baselines on three datasets.
Figure 5 compares perplexity with different There are two clear trends: First, higher leads overall lower perplexity, which shows that the readability loss defined in Equation 2 works as desired. Second, more trigger words cause higher perplexity. We checked the prompts and found that long prompts contain more word pieces, for
Google-RF. It contains facts manually extracted from Wikipedia by Petroni al. (2019). It covers three relations, namely place-of-birth with 2937 facts, date-of-birth with 1825 facts, and place-of- death with 766 facts.
ConceptNet. (Speer et al., 2012) is a multi- lingual knowledge base, initially built on top of

---

Code
LAMA Prompt
Modified Prompt
P413
[X] plays in [Y] position.
The position played by [X] is [Y].
P1923
[Y] participated in the [X]
[X] was participated in by [Y].
P106
[X] [Y] by profession.
[X]'s I's occupation a/an [Y].
P102
[X] member of the [Y] political party.
X is member of the political party [Y].
P27
[X] is [Y] citizen.
[X] is a citizen of [Y].
P136
[X] plays [Y] music.
The music played by [X] is [Y].
P140
[X] is affiliated with the [Y] religion.
The religion affiliated with [X] is [Y].
P190 [X] and [Y] are twin cities.
The twin city of [X] is [Y]
Table 6: Modifications on LAMA prompts that move the object to the end of the prompt.
Method
Google-RE
ConceptNet
UHN.
C LAMA Prompt Modifications
LAMA
9.71
0.1
21.8F
We modified eight prompts to place the object at the end of the prompt. The details are presented in Table 6.
LPAQA
10.6
28.74
Auto. 11.0
12.2
31.3
Opti. 12.9
38.4
Soft.
14.5
Read. 11.8
16.6
37.2
Table 7: Results on datasets of Google-RE, ConcepNet, and UHN. The best results are marked in bold. and mark baseline results obtained from Qin and Eisner (2021) and Zhong et al. (2021), respectively. "_" denotes the results that are not reported by current works. The results of AUTOPROMPT Google-Ro Reg and Concept- Net are reimplemented by us.
Open Mind Common Sense (OMCS) sentences. OMCS represents commonsense relationships be- tween words and/or phrases. Petroni et al. (2019) extract facts from the English part of ConceptNet that have single-token objects covering 16 relations benchmark of knowledge probing.
UHN. Poerner et al. (2019) note that some facts in LAMA can be recalled solely based on surface forms of entities without memorizing facts. They filter out those casy-to-guess facts and create more diffieult benchmark, denoted LAMA-UHN (Jiang et al., 2020). In this experiment, we use one its subsets, T-REx-UHN.
READPROMPT achieves the best results on Con- ceptNet, slightly lower than SOFTPROMPT on Google-RE and OPTIPROMPT UHN. The re- sults show that though READPROMPT is composed of discrete real words, it can also achieve compa- rable results as the continuous prompts, which consistent with the conclusion obtained in the main results (section 4.2).


Reviews:
[{"review_id": "aBh1MOHA0n", "reviewer": "Reviewer_S7sg", "summary": "", "questions": "A: Why does the equation on line 200 have a k+2 if there are k tokens being optimized? Are we including both the subject x and the target y? If so, this should be explicitly stated.\n\nB: For Equation (6), should p rather be p_{\\backslash p_i}, since we're considering the \"masked\" probability of replacing the i-th token?\n\nC: In the description of the setup, I could not find discussion or mention of the number of candidate \"replacement\" tokens that were selected at each iteration to be ranked with the perplexity augmented loss function. Was this number also optimized through hyper-parameter search or simply fixed throughout the experiments? Is it the same number as that used by AutoPrompt? Note, that this value is distinct from the prompt length.\n\nD: The discussion on the setup for methods being compared to is somewhat limited. For example, we don't know the length of the prompts used for either AutoPrompt or any of the continuous prompting methods. Were these parameters optimized as well or possibly chosen based on recommendations in the original papers?", "limitations": "", "rating": 4, "confidence": 4, "soundness": 4, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "We would like to express our heartfelt appreciation for your thorough review and insightful feedback. We have diligently examined each of your reviews and summarized ten responses. Rather than positioning this as a mere defense of our work, we perceive it as a valuable exchange with a professional expert of your stature. To facilitate your review process, we include the following table of contents:\n\n### TOC\n- [1. The order of updating trigger tokens](#1-the-order-of-updating-trigger-tokens)\n- [2. The reason why ReadPrompt outperforms AutoPrompt](#2-the-reason-why-readprompt-outperforms-autoprompt)\n- [3. The data for hyper-parameter search](#3-the-data-for-hyper-parameter-search)\n- [4. The Ideal Point in Figure 6](#4-the-ideal-point-in-figure-6)\n- [5. Limitation: ReadPrompt not applying to closed-source LLMs (GPT3.5, GPT4)](#5-limitation-readprompt-not-applying-to-closed-source-llms-gpt35-gpt4)\n- [6. (A) Line 200: the k+2 in Equation (3)](#6-a-line-200-the-k2-in-equation-3)\n- [7. (B) Changing $p$ to $p_{/p_i}$ in Equation (6)](#7-b-changing-to-in-equation-6)\n- [8. (C) Hyperparameter: the number of candidates](#8-c-hyperparameter-the-number-of-candidates)\n- [9. (D) The experiment settings and hyperparameters of baseline methods](#9-d-the-experiment-settings-and-hyperparameters-of-baseline-methods)\n- [10. Typos Grammar Style And Presentation Improvements](#10-typos-grammar-style-and-presentation-improvements)\n\n### 1. The order of updating trigger tokens\nThe order of updating trigger tokens is **sequential**. In lines 324-327 of our work, we state *\"Then we apply ReadPrompt to search for better trigger words one by one.\"* We intend to convey that we employ the ReadPrompt to search for trigger words sequentially. We have made the revisions in the next version according to your suggestion to eliminate ambiguity. Our chosen sequential update approach, distinct from the random update of AutoPrompt, is motivated by **two key considerations**:\n\n1. Drawing from our experience with AutoPrompt, the impact of random token updates on final outcomes is minimal. Furthermore, updating tokens sequentially does not result in local optima, even in the absence of randomness.\n2. Updating tokens from left to right in sequence aligns with the decoding process of left-to-right LLMs, such as LLaMA, and coincides with the calculation of standard PPL. In the preliminary stages, we undertook diverse efforts to formulate the PPL constraint. The process of sequentially updating tokens from left to right expedited our endeavors in devising PPL constraints tailored to left-to-right LLMs. \n\n### 2. The reason why ReadPrompt outperforms AutoPrompt\nWe wholeheartedly support your hypothesis and would like to provide an additional perspective on the construction of pre-trained corpus and contexts. The subject and object always appeared as a fluent natural language in the pre-training corpus. Therefore, we believe that maintaining a fluent context is more likely to guide the PLM in accurately identifying the appropriate object for a given subject. The prompt discovered by AutoPrompt constitutes a token sequence absent from the pre-training corpus coverage. **Conversely, the context formulated by ReadPrompt exhibits greater alignment with sentences present within the pre-training corpus.** This alignment potentially confers an advantage to ReadPrompt.\n\nFurthermore, current research has shown that prompts with **low perplexity are more beneficial for the performance** of LLM in downstream tasks. The paper \"Demystifying prompts in language models via perplexity estimation\" (cited by our work, lines 645-647) allows GPT3 to paraphrase some manually written seed prompts. Experimental results demonstrate that prompts with low perplexity significantly improve the performance of LLM in word-level translation and text classification tasks.\n\n### 3. The data for hyper-parameter search\nWe employed **ALL** the data in the training set for hyperparameter search, with the selection of hyperparameters based on the results from the development set. Due to the limited training data for each relation (fewer than 800 data points), no sampling was performed.\n\nTo optimize the efficiency of the hyperparameter search, we refined the approach for setting the **maximum iteration**. In our initial experiments, we discovered that the optimal prompt identified within 500 iterations yields performance comparable to that of the optimal prompt identified within 1000 iterations. Building upon this insight, we developed a strategic approach: when the number of trigger tokens stands at 10 or less, we established the maximum iteration as 500; when the number exceeds 10, we incorporated 50 additional iterations for each extra trigger token. This methodology ensures a thorough search for each token. The strategy is mathematically represented as follows: $iter_{max} = 500$,  if $k \\le 10$; $iter_{max} = 50k$, if $k \\gt 10$, where $k$ denotes the trigger token number. This approach, more time-efficient than AutoPrompt's recommended 1000 iterations, significantly expedites our hyperparameter search process. In the next version, we will provide this information in the Appendix, section A.\n\nAfter determining the optimal combination of hyperparameters ($k=10$ and $\\alpha=0.7$), we proceeded by increasing the maximum iteration to 1000. Subsequently, we conducted prompt searches utilizing these hyperparameters, performed evaluations, and reported the results in Table 1.\n\n### 4. The Ideal Point in Figure 6\nThe term \"ideal point\" refers to a prompt that possesses **both good readability and capability**.  Unfortunately, we have not encountered such a prompt in our experiments. The prompt offering the highest readability might not result in the best performance, and conversely, the most effective prompt might not possess the highest readability. We will include an explanation of the ideal point in Appendix Figure 6.\n\n### 5. Limitation: ReadPrompt not applying to closed-source LLMs (GPT-3.5, GPT-4)\nThis may **NOT** be a limitation of ReadPrompt in the future. During our recent literature review, we came across an insightful paper that tackles the challenge of finding specific prompts for closed-source LLMs accessible only through APIs. We are delighted to share this paper with you.\n\nThe paper titled \"Universal and Transferable Adversarial Attacks on Aligned Language Models\" was publicly released on 7-27 in its preprint version. They also present a modification to the AutoPrompt, which is aimed at discovering adversarial prompts for open-source LLMs like Vicuna. Guided by those adversarial prompts, the LLMs are induced to provide answers to questions that they should not (a.k.a. jailbreak). The authors highlight that the prompts identified through their method have a certain level of generalizability. **Notably, the adversarial prompts searched for Vicuna can also be effectively applied to jailbreak GPT-3.5, as well as GPT-4.**\n\nWhile the paper's primary focus is not directly on the knowledge probing task, their methods are highly relevant to our work, as both methods are derived from AutoPrompt. This demonstrates the potential for the searched prompts to be applied in a generalized manner across multiple LMs.\n\nWe have also recently been exploring the generalization of the readable prompts searched on moderately sized (7b) LLMs to GPT-4, thereby improving the effectiveness of GPT-4 on downstream tasks. Given that GPT-4 already performs very well on the knowledge probing task (Table 2 shows that GPT-4's performance is 89.9%), our recent work attempts to try this idea on other more challenging tasks.\n\n### 6. (A) Line 200: the k+2 in Equation (3)\nYour understanding is correct. As you mentioned, the value of $k+2$ arises from taking into account both the subject and object during PPL calculation. We intend to incorporate this clarification in line 202.\n\n### 7. (B) Changing $p$ to $p_{/p_i}$ in Equation (6)\nYour suggestion is correct. Equation (6) would be more accurate by modifying $p$ to $p_{/p_i}$. The given context should be tokens except for token $i$. We modified Equation (6) according to your suggestion. We will also consider exchanging the subscripts $i$ and $j$ in Equation (6) and its related description to maintain consistency with the use of subscripts in Equation (3).\n\n### 8. (C) Hyperparameter: the number of candidates\nWe follow the settings of AutoPrompt and **fix the number of candidates to 10**. We did not conduct a search for this hyperparameter before. Inspired by your questions, we realize it is a crucial hyperparameter that may infect both the efficiency and optimum prompts of the search. Therefore, in the past week, we conducted a search for the best practices regarding the number of candidates, with the aim of investigating the impact of this hyperparameter.\n\nWe conducted the experiments on BERT-base with the number of candidates $l\\in\\{10, 30, 50, 70, 90\\}$ . Due to the urgent time constraints, we set the maximum iteration to 300 to accelerate the search process. The other settings are consistent with the main experiments presented in this work. The results, including P@1 and PPL, are presented in the table below. Furthermore, we report the time required for each setting to complete the search for all 41 relations. The experiments were conducted on the server with a single GeForce RTX 2080 Ti. We acknowledge that various factors may affect the time-consuming, but we believe it can provide a rough estimate of the efficiency of each setting. We did not monitor the usage of memory, as we assume it to be consistent across all settings based on the code development.\n\n\n|  _l =_ | 10   | 30  | 50   | 70   | 90   |\n| ----   | ---- | ----| ---- | ---- | ---- |\n|  P@1   | 40.55%   | 48.30%  | 48.43%   | 48.40%   | 49.00%   |\n|  PPL   | 266.7    | 290.3   | 245.5    | 323.8    | 257.1    |\n|  Time (h:m:s) | 1:02:57  | 2:22:08 | 3:40:54  | 4:58:44  | 6:16:41  |\n\nIn terms of capability (P@1), more candidates can enable ReadPrompt to identify a better local optimum within a limited number of iterations. However, an increased number of candidates also results in an increase in time consumption. Specifically, when $l=90$, the time taken is six times longer than when $l=10$. By increasing the maximum iteration to 1000 （around 3.5 hours）or even longer, we can attain comparable or superior performance with $l=10$ (as illustrated in Table 1). **We believe ReadPrompt will exhibit consistent performance in its final output, regardless of the varying settings of $l$, as long as we extend the maximum iterations.** But currently, we cannot claim which setting is more efficient from the comparisons.\n\nAs for the readability (PPL), there is no discernible trend in its variation with respect to the value of $l$, indicating that **PPL remains unaffected by $l$**.\n\n### 9. (D) The experiment settings and hyperparameters of baseline methods\nYes, we refer to the recommended hyperparameters and settings in the original paper, e.g., 7 trigger tokens for AutoPrompt. We will supplement the explanation in section 4.1 Searching Details of ReadPrompt (line 334), by stating that all hyperparameters for the other prompting methods reproduced in this work are referenced from the original papers with the recommended settings. \n\n### 10. Typos Grammar Style And Presentation Improvements\nWe extend our sincere gratitude for your meticulous suggestions for revision. We have diligently incorporated your feedback into the local draft, effecting the necessary changes.\n\nRegarding sentence 455, _\"This indicates that blindly pursuing high accuracy is meaningless,\"_ we have rephrased it to _\"This indicates that simply striving for high accuracy may not yield effectiveness in certain cases.\"_\n\nRegarding the typo in Table 6, \"EXP.\" refers to our proposed ReadPrompt. Therefore, it should be corrected to \"Read.\" We previously considered using Explainable prompt (EXPrompt) to name this method. Nevertheless, the use of \"Explainable\" to describe the method appeared somewhat excessive. Therefore, we finally named it ReadPrompt. We accidentally missed correcting this cell during the revision stage.", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "The results of the paper are quite convincing and thorough. In my experience, the lack of readability and interpretability of discrete prompt optimization approaches, like AutoPrompt and GrIPS, is quite surprising for practitioners and gives many pause in using such prompts, even if they produce better performance than human designed ones. Enforcing readability while maintaining utility is a very important task. This paper takes a solid step in this direction, while still maintaining the efficiency and straightforward nature of AutoPrompt.\n\nI think the experimental design of the \"attack\" prompts is also a very useful consideration in understanding how the un-readable prompts, discrete or continuous, produce increased performance. While the design is fairly specific to the anti-symmetric relation setting, I can imagine it inspiring other approaches for understanding uninterpretable prompt performance. \n\nThe paper also attempts to provide quite a lot of detail around reproducibility by providing the prompt designs for all of their experiments. I think this is helpful in future work to use or extend the work presented here. However, there is a bit of a lack of discussion on hyper-parameters used for training and those used for the other methods to which ReadPrompt is compared.", "reasons_to_reject": "Overall, the paper is well written, but there are still some questions that I believe should be answered in order to fully describe the experimental setup both for reproducibility and to better understand the setup of the methods being compared with. I would also say that the authors missed one or two places to discuss some of the results in more detail, which could help the reader understand the results better and interpret why the method is performing so well. Some of these are stated below or as questions to the authors.\n\nThe authors say that they iteratively update the trigger tokens, but do not state whether this process is random, as in AutoPrompt, or sequential.\n\nIt would be useful for the authors to speculate as to why the constrained ReadPrompt search out-performs AutoPrompt in many instances. I would hypothesize that it enforces some global prompt consistency in the search, potentially preventing the model from becoming stuck in less valuable local minima due to exclusively optimizing using a single-token first-order search.\n\nIn the hyper-parameter search, it is unclear whether all relations where used in the search or whether a subset of of these where used to obtain the results.\n\nThe authors mentioned marking \"ideal\" points in Figure 6, but there is no discussion as to what ideal means specifically.\n\nAs a minor comment, I think it's important to note, in the limitations section, that this method requires calculation of a gradient w.r.t the model embeddings. This requires model access, which is not always possible, for example with the higher versions of GPT.", "questions_to_authors": "A: Why does the equation on line 200 have a k+2 if there are k tokens being optimized? Are we including both the subject x and the target y? If so, this should be explicitly stated.\n\nB: For Equation (6), should p rather be p_{\\backslash p_i}, since we're considering the \"masked\" probability of replacing the i-th token?\n\nC: In the description of the setup, I could not find discussion or mention of the number of candidate \"replacement\" tokens that were selected at each iteration to be ranked with the perplexity augmented loss function. Was this number also optimized through hyper-parameter search or simply fixed throughout the experiments? Is it the same number as that used by AutoPrompt? Note, that this value is distinct from the prompt length.\n\nD: The discussion on the setup for methods being compared to is somewhat limited. For example, we don't know the length of the prompts used for either AutoPrompt or any of the continuous prompting methods. Were these parameters optimized as well or possibly chosen based on recommendations in the original papers?", "paper_topic_and_main_contributions": "This paper proposes a method for discrete prompt optimization with more human readable prompts through the lens of knowledge probing and demonstrates its utility in maximizing knowledge probing effectiveness through a number of experiments with various model types, primarily focused on the T-REx dataset. The approach is an extension of the first-order optimization procedure proposed by Shin et al. (2020) where the discrete search is constrained by a penalty based on the perplexity resulting from a candidate token substitution. The method, named ReadPrompt, may also be augmented with an alignment step based on the BertScore of the resulting prompt. In experiments the method outperforms several other discrete and continuous prompt optimization approaches while maintaining a fair bit of human readability. Finally, the paper also constructs some knowledge probing investigation of optimum prompts through so-called \"attack\" prompts to determine whether optimized prompts are true knowledge probes.", "excitement": 4, "reproducibility": 4, "ethical_concerns": "No", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {"Missing_References": "Another relevant reference that backs up the paper's case for prompt interpretability being difficult is Prompt Waywardness: The Curious Case of Discretized Interpretation of Continuous Prompts"}, "extra_text": {"Typos_Grammar_Style_and_Presentation_Improvements": "Lines 026-029: \"We claim that the probing outcomes of the current prompting methods are unreliable that overestimate the knowledge contained within PLMs.\" This phrasing is a bit awkward and could be improved.\n\nLine 088: \"Referring Molnar (2020),\" is likely missing a \"to\"\n\nLine 112: Typo, missing a \"t\"\n\nLine 133 introduces the acronym for PLM, but this is redundant as it has been introduced earlier.\n\nSome equation are numbered when they need not be, as they are never referred to anywhere in the manuscript.\n\nThe notation \\mathcal{R} is never introduced. If it is meant as the set of real numbers, then I would recommend the more standard notation of \\mathbb{R} be used.\n\nIn the discussion on perplexity on lines 243-244, \"Intuitively, a well-trained PLM has low perplexity in a smooth sentence.\" I might suggest changing \"smooth\" to something like \"fluent.\"\n\nThere is inconsistent casing for equation references. Either \"equation x\" or \"Equation x\" should be used throughout. it appears the latter is preference for Table and Figure references, so I would recommend continuing with that.\n\nCapitalization of LLaMA model is not correct throughout the paper.\n\nLine 440: I would suggest rephrasing \"Very few of the prompts even contradict...\" to something like \"A select few of the prompts even contradict...\" or the like.\n\nI would suggest softening this phrasing, \"This indicates that blindly pursuing high accuracy is meaningless\" on lines 455-456. I don't believe that pursuing accuracy is meaningless, but rather the context that produces high accuracy is also useful in determining the knowledge volume of the PLM.\n\nLine 527, 'PLM\" to \"PLMs\" or the \"PLM\"\n\nIn Table 6, it is not very clear where the ReadPrompt results are reported. My guess is that \"EXP.\" is the ReadPrompt results for the three datasets, but that is inconsistent with the rest of the tables in the paper and not mentioned anywhere.\n\nLine 855, typo in reference to Table 9."}}, {"review_id": "3R9p8PVEDs", "reviewer": "Reviewer_5PKH", "summary": "", "questions": "1. I would like to request clarification on whether your definition of perplexity takes into account its applicability to all models without introducing any inaccuracies or errors.\n2. In section 4.1, I noticed that you provided a comparison between different prompting strategies in Table 1. However, the 'align' module is only incorporated in the theory proposed in this paper. I would like to understand why you did not include the 'align' module for comparison in all the presented theories.", "limitations": "", "rating": 3, "confidence": 3, "soundness": 2, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "We sincerely appreciate your feedback on our paper. \n\n## Reasons To Reject \nWe are grateful for the opportunity to address your inquiries and concerns, particularly concerning the novelty and experimental outcomes of our research.\n\n### 1. The paper lacks novelty\nRegarding novelty, our paper focuses on the task of knowledge probing and highlights the issue of current methods constructing unreadable prompts, resulting in misalignment between the prompts and real-world knowledge. This problem leads to overestimated and even erroneous probing results, which introduce potential reliability concerns for downstream tasks. A running example can be found in line 69 and Figure 1: given _\"Italy national art Petroleum outside\"_ as a prompt, the PLM outputs the matching object _\"Europe\"_. Nevertheless, it is difficult to claim that the PLM knows the fact (Italy, located-in, Europe) based on this prompt. Current methods cannot guarantee the prompts they constructed are still consistent with the original meaning. The issue was further confirmed by an attack experiment (section 4.4), which shows the urgent need to improve the reliability of current prompts. \n\nWe are the first to identify this problem and propose a corresponding solution. We believe our work is sufficiently novel from the motivation and method itself.\n\n### 2. The experimental results are not compelling\nRegarding experimental results,  it is important to note that our paper does not focus on presenting a trick to improve the performance of PLMs. Instead, we propose ReadPrompt to improve the reliability of knowledge probing results, rather than blindly pursuing overestimated results. We compared ReadPrompt with six prompting methods, which encompass all the current state-of-the-art methods for knowledge probing, including discrete prompts (AutoPrompt) and continuous prompts (SoftPrompt etc.)  According to the results in Table 1 and Section 4.3 Case Study, ReadPrompt not only achieves state-of-the-art performance, but more importantly, ensures that the searched prompts can be understood by humans, making the probing results more convincing. \n\nWe sincerely hope that these clarifications provide a clearer understanding of the essence of our work. We kindly request that if you have any specific questions or suggestions pertaining to the novelty and experimental results, please share them with us. Your expertise and insights are invaluable to us, and we welcome any further input that could enhance the quality and impact of our work.\n\n## Questions For The Authors\n\n### 1. The definition of Perplexity\nFor masked PLMs such as BERT, their attention mechanism is bidirectional. However, the conventional definition of PPL only conditions the log-likelihood of the i-th token based on the context preceding it, which is not consistent with the bidirectional attention mechanism employed by these PLMs. Therefore, for these PLMs, we define a new way of calculating PPL, as shown in Equation (6). \n\nFor left-to-right PLMs such as GPT-2 and LLaMA, the calculation of PPL adheres to the standard definition. \n\nIn other words, we only applied the new PPL definition for masked PLMs (see line 198), and experiments showed that using this new PPL definition as a constraint for these PLMs is effective. \n\nPlease note that the primary focus of our experiment is to compare the performance of the prompting method, rather than evaluating the various PLMs themselves. The definition of PPL for each PLM remains constant and unchanged, ensuring a fair comparison.\n\n### 2. Reason for not aligning other prompting methods\nThe ReadPrompt faces a challenge: while prompts are readable and possess a certain meaning, the meaning they convey may deviate from real-world knowledge. To address this issue, we employed alignment for the prompts searched by the ReadPrompt. Here, we utilized the BERTScore to select the most similar prompt in comparison to real-world knowledge among the five available candidates.\n\nHowever, for continuous prompts, BERTScore cannot be used to measure the similarity between vector sequences (i.e., the prompts are composed of new token embeddings).\n\nFor discrete prompts, manual prompts do not necessitate alignment, whereas AutoPrompt's retrieved prompts comprise meaningless token sequences. While it's technically possible to align these token sequences with the knowledge, such alignment is not crucial, as the resulting aligned prompts would still lack meaningful context.\n\nThanks again for your review. We have already corrected the **typo** in the next version.", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "The paper addresses the issue of misalignment between prompts and real-world knowledge and presents an outline for future research directions.", "reasons_to_reject": "The overall logic of the paper is relatively clear, but it lacks novelty. The proposed 'readprompt' theory does not provide significant reference value, and the experimental results on existing models are not sufficiently compelling. Therefore, I recommend rejecting the paper.", "questions_to_authors": "1. I would like to request clarification on whether your definition of perplexity takes into account its applicability to all models without introducing any inaccuracies or errors.\n2. In section 4.1, I noticed that you provided a comparison between different prompting strategies in Table 1. However, the 'align' module is only incorporated in the theory proposed in this paper. I would like to understand why you did not include the 'align' module for comparison in all the presented theories.", "paper_topic_and_main_contributions": "The paper under review aims to address the challenges of accuracy and consistency in large model prompt engineering. It proposes a theoretical framework called \"readprompt\" and provides experimental evidence demonstrating its ability to achieve State-of-the-Art (SOTA) performance.Furthermore, the paper addresses the issue of misalignment between prompts and real-world knowledge and presents an outline for future research directions.", "excitement": 3, "reproducibility": 4, "ethical_concerns": "No", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {"Typos_Grammar_Style_and_Presentation_Improvements": "I'd like to point out that there is a missing \"T\" in the word \"The\" on line 112."}, "extra_text": {}}, {"review_id": "rzTYzdDMHA", "reviewer": "Reviewer_bnqg", "summary": "", "questions": "", "limitations": "", "rating": 3, "confidence": 1, "soundness": 4, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "We sincerely appreciate your valuable review suggestions. We are pleased to acknowledge that your grasp of the paper's content is accurate, and your summary is on point.  We would like to initiate a discussion with you concerning both the **dataset** and the **models** employed in our experiments, along with some **supplementary experiment results**.\n\nWe present experiment results on **four knowledge probing datasets**, namely T-Rex, Google-RE, ConceptNet, and UHN. Notably, among these datasets, T-Rex enjoys widespread adoption and serves as a common choice for numerous related knowledge probing works. As such, we opted to use the evaluation results on this dataset as our main results. Importantly, our deductions drawn from the four datasets are consistent: ReadPrompt achieves state-of-the-art performance. Therefore, we place the results of other experiments in the Appendix (refer to section B, Table 6). Kindly take note of a typo in Table 6 where \"EXP.\" should be rectified to \"Read.\"\n\nRegarding the **models**, we report the complete experimental results of six models, including BERT-Base, BERT-Large, RoBERTa, LLaMA, Alpaca, and Vicuna (see Tables 1 and 2). Additionally, we tested GPT-3.5 and GPT-4, two closed-source models, as baseline. This brings the total number of considered LMs to **eight**. Noteworthy is the high recognition achieved by LLaMA, Alpaca, and Vicuna since 2023, whereas GPT4 is currently recognized as one of the preeminent LLMs. We firmly believe that the language models covered in our experiment are sufficient.\n\nWe also conducted **experiments** on two models, OPT (125M) and T5-base (220M). However, the current methods of prompting do not effectively suit these two models. Considering the rapid growth of large language models (LLM), we strongly believe that providing dependable and precise analysis for LLM holds utmost significance. Consequently, we have decided to discontinue our experiments with these two models, reserving more space for the analysis of LLM. Nonetheless, we are happy to share the preliminary results with you.\n\n|         | OPT (125M)  | T5-base (220M)|\n|  ----   | ----        | ----          |\n|         | P@1 / PPL   | P@1 / PPL     |\n| LAMA    | 28.8 / 28.3 | 33.6 / 30.5   |\n| Auto.   | 40.3 / 753.4| 45.8 / 1033.8 |\n| Read.   | 46.8 / 212.9| 48.2 / 340.3  |\n| + align | 46.1 / 144.2| 45.9 / 135.6  |\n\nThe hyperparameters employed in this set of experiments have not been optimized through search, leading to generally low numerical values for the outcomes. Nonetheless, the results of the comparison of various methods are generally consistent with the results reported in Table 1. We will add this table to Appendix, section B.\n\nThanks again for your careful review. We have already corrected the **typo** in the next version.", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "Prompt readability leads to a new discovery, which means a misalignment between constructed prompts and knowledge. The author claim that the probing outcomes of the current prompting methods are unreliable that overestimate the knowledge contained within PLMs, which is a direction worth continuing to explore.", "reasons_to_reject": "The dataset and pre-training model used by the author are not very sufficient, and some current representative work should be supplemented.", "questions_to_authors": "", "paper_topic_and_main_contributions": "This paper addresses the problem of prompt readability in the knowledge probin task. The authors propose READPROMPT, a method for searching sentences to construct prompts. The READPROMPT achieves state-of-the-art performance on the current knowledge probing bench-mark and significantly improves the readability of prompts compared to other automated prompting methods.", "excitement": 3, "reproducibility": 3, "ethical_concerns": "No", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {"Typos_Grammar_Style_and_Presentation_Improvements": "Line 112: 'he experimental results'  loses the letter T."}, "extra_text": {}}]

Author rebuttal:

Meta-review: **Strengths**:

1. Discrete prompt optimization with more human readable prompts is clearly an important problem.

2. "attack" prompts analysis to determine whether optimized prompts are true knowledge probes is a good contribution. Discovery of misalignment between constructed prompts and knowledge in current prompting methods is intriguing.

3. The discussion provides a lot of clarifications, and extra results.

**Suggestions**: There is a wealth of clarifications and new information in your rebuttal. Request you to incorporate the same in your draft.
Decision: Accept-Findings