Original: Decoupling Knowledge from Memorization: Retrieval-augmented Prompt Learning
Revised: Decoupling Knowledge from Memorization: Retrieval-augmented Prompt Learning

Original: Anonymous Author(s)

Original: Abstract
Revised: Abstract

Original: Prompt learning approaches have made waves in natural language processing byinducing better few-shot performance while they still follow a parametric-basedlearning paradigm; the oblivion and rote memorization problems in learning mayencounter unstable generalization issues.
Revised: Prompt learning approaches have made waves in natural language processing by inducing better few-shot performance while they still follow a parametric-based learning paradigm; the oblivion and rote memorization problems in learning may encounter unstable generalization issues.

Original: Specifically, vanilla prompt learning maystruggle to utilize atypical instances by rote during fully-supervised training or overfit shallow patterns with low-shot data.
Revised: Specifically, vanilla prompt learning may struggle to utilize atypical instances by rote during fully-supervised training or overfit shallow patterns with low-shot data.

Original: To alleviate such limitations, we develop
Revised: To alleviate such limitations, we develop

Original: R ETRO P ROMPT with the motivation of decoupling knowledge from memorizationto help the model strike a balance between generalization and memorization.
Revised: R ETRO P ROMPT with the motivation of decoupling knowledge from memorization to help the model strike a balance between generalization and memorization.

Original: Incontrast with vanilla prompt learning, R ETRO P ROMPT constructs an open-bookknowledge-store from training instances and implements a retrieval mechanismduring the process of input, training and inference, thus equipping the model withthe ability to retrieve related contexts from the training corpus as cues for enhancement.
Revised: In contrast with vanilla prompt learning, R ETRO P ROMPT constructs an open-book knowledge-store from training instances and implements a retrieval mechanism during the process of input, training and inference, thus equipping the model with the ability to retrieve related contexts from the training corpus as cues for enhancement.

Original: Extensive experiments demonstrate that R ETRO P ROMPT can obtain betterperformance in both few-shot and zero-shot settings.
Revised: Extensive experiments demonstrate that R ETRO P ROMPT can obtain better performance in both few-shot and zero-shot settings.

Original: Besides, we further illustratethat our proposed R ETRO P ROMPT can yield better generalization abilities withnew datasets.
Revised: Besides, we further illustrate that our proposed R ETRO P ROMPT can yield better generalization abilities with new datasets.

Original: Detailed analysis of memorization indeed reveals R ETRO P ROMPTcan reduce the reliance of language models on memorization; thus, improvinggeneralization for downstream tasks 1 .
Revised: Detailed analysis of memorization indeed reveals R ETRO P ROMPT can reduce the reliance of language models on memorization; thus, improving generalization for downstream tasks 3 .

Original:  Introduction
Revised:  Introduction

Original: Large parametric language models [45, 7, 22, 31] have achieved dramatic empirical success innatural language processing (NLP).
Revised: Large parametric language models [46, 7, 23, 32] have achieved dramatic empirical success in natural language processing (NLP).

Original: Notably, pre-trained language models (PLMs) have learned asubstantial amount of in-depth knowledge from data, and have archived tremendous promise infew-shot/zero-shot learning ability with the natural language prompts [13, 50, 55].
Revised: Notably, pre-trained language models (PLMs) have learned a substantial amount of in-depth knowledge from data, and have archived tremendous promise in few-shot/zero-shot learning ability with the natural language prompts [13, 51, 58].

Original: However, Recentstudies [37, 39, 57] observe that prompt learning with PLMs usually generalizes unstably in anextremely low-resource setting or emerging domains.
Revised: However, Recent studies [38, 40, 60] observe that prompt learning with PLMs usually generalizes unstably in an extremely low-resource setting or emerging domains.

Original: One potential reason is that, it is non-trivialfor parametric models to learn rare or hard patterns well with rote memorization , thus, resulting ininefficient generalizable performance.
Revised: One potential reason is that, it is non-trivial for parametric models to learn rare or hard patterns well with rote memorization , thus, resulting in inefficient generalizable performance.

Original: Intuitively, if we regard the whole training data as a book and the test phase as the examination ,the current training-test procedure of prompt learning (based on batch data training) can be viewedas page-by-page memorization and closed-book examination [42].
Revised: Intuitively, if we regard the whole training set as a book and the test phase as the examination , the current training-test procedure of prompt learning (based on batch data training) can be viewed as page-by-page memorization and closed-book examination [43].

Original: During training, vanilla promptlearning may struggle to memorize atypical instances in a fully-supervised setting or overfit shallowpatterns with low-shot data [60, 10].
Revised: During training, vanilla prompt learning may struggle to memorize atypical instances in a fully-supervised setting or overfit shallow patterns with low-shot data [64, 10].

Original: Specifically, recent studies[11, 12] have proposed a long-tail theory, which states that if training data form a long-tail distribution and have small “sub-populations”with atypical instances, then PLMs indeed predict on the test data through rote memorizing theseatypical instances rather than learning the common patterns [60, 54].
Revised: Specifically, recent studies[11, 12] have proposed a longtail theory, which notes that when the training set has a long-tail distribution and contain small “sub-populations” with atypical instances, then PLMs indeed predict on the test data through rote memorizing these atypical instances rather than learning the common patterns [64, 57].

Original: The limitations of rote memorization remind us of the human learning process of “learn by analogy”and the proverb that “the palest ink is better than the best memory” .
Revised: The limitations of rote memorization remind us of the human learning process of “learn by analogy” and the proverb that “the palest ink is better than the best memory” .

Original: Note that humans can performassociative learning to recall relevant skills in deep memories for reinforcing each other, thus, owningthe extraordinary abilities to solve few-shot and zero-shot tasks.
Revised: Note that humans can perform associative learning to recall relevant skills in deep memories for reinforcing each other, thus, owning the extraordinary abilities to solve few-shot and zero-shot tasks.

Original: Motivated by these, we endeavor toimprove the generalization ability of prompt learning with retrieval and association.
Revised: Motivated by these, we endeavor to improve the generalization ability of prompt learning with retrieval and association.

Original: Our intuition isthat the difficulty of resolving the above limitations can be substantially alleviated if we can decouplethe knowledge from memorization by constructing an open-book knowledge-store from the trainingdata; thus, referring to related knowledge could provide a strong enhancement signal to help themodel strike a balance between generalization and memorization.
Revised: Our intuition is that the difficulty of resolving the above limitations can be substantially alleviated if we can decouple the knowledge from memorization by constructing an open-book knowledge-store from the training data; thus, referring to related knowledge could provide a strong enhancement signal to help the model strike a balance between generalization and memorization.

Original: [Figure]
Revised: [Figure]

Original: Specifically, we introduce a novel retrieval- augmented framework based on promptlearning ( R ETRO P ROMPT ) as shown in
Revised: Specifically, we introduce a novel retrievalaugmented framework based on prompt learning ( R ETRO P ROMPT ) as shown in

Original: Figure 1.
Revised: Figure 1.

Original: The open-book knowledge store( K , V ) , defined as the set of key: prompt- based example embeddings and value: corresponding label words constructed fromthe training data, are served as additionalreferences for the model to decouple knowledge from pure memorization to some extent.
Revised: The open-book knowledge store ( K , V ) , defined as the set of key: promptbased example embeddings and value: corresponding label words constructed from the training data, are served as additional references for the model to decouple knowledge from pure memorization to some extent.

Original: Specifically, to integrate retrievedknowledge into the input, Firstly , we design to incorporate neural demonstrations into the inputsequences as in-context augmentation, where the demonstration is retrieved from the knowledge-store.
Revised: Specifically, to integrate retrieved knowledge into the input, Firstly , we design to incorporate neural demonstrations into the input sequences as in-context augmentation, where the demonstration is retrieved from the knowledge-store.

Original: Then , we apply a non-parametric algorithm k NN over the input query and knowledge store, andregard k NN results as an indication of easy vs. hard examples in the training set.
Revised: Then , we apply a non-parametric algorithm k NN over the input query and knowledge store, and regard k NN results as an indication of easy vs. hard instances.

Original: More specifically,we automatically force the model to focus on the hard examples identified by k NN by assigning ascaling during training.
Revised: Moreover, we automatically force the model to emphasize the hard instances identified by k NN by assigning a scaling during training.

Original: Lastly , the k NN results are further employed at the output of the PLM headto participate in masked prediction during inference.
Revised: Lastly , the k NN results are further employed at the output of the PLM head to participate in masked prediction.

Original: The model retrieves Top- k nearest referenceinstances as cues from ( K , V ) and makes inference by linearly interpolating the output of promptlearning with a non-parametric nearest neighbor distribution.
Revised: The model conducts inference through linearly interpolating the non-parametric nearest neighbor distribution with the output of prompt learning, which regards the Top- k nearest reference instances as cues from ( K , V ) .

Original: The considerable performance gains on nine tasks in few-shot and zero-shot settings demonstrate thatour systemic retrieval mechanism helps the model generalize better with scarce data.
Revised: The considerable performance gains on nine tasks in few-shot and zero-shot settings demonstrate that our systemic retrieval mechanism helps the model generalize better with scarce data.

Original: Experiments inthe fully-supervised setting with long-tail distribution illustrate that our R ETRO P ROMPT can dealwith atypical instances more robustly.
Revised: Experiments in the fully-supervised setting with long-tail distribution illustrate that our R ETRO P ROMPT can deal with atypical instances more robustly.

Original: We further adopt self-influence [27] as our memorizationscoring function to analyze the memorization process between fine-tuning, prompt learning andour R ETRO P ROMPT .
Revised: We further adopt self-influence [28] as our memorization scoring function to analyze the memorization process between fine-tuning, prompt learning and our R ETRO P ROMPT .

Original: The final analysis results show that 1) the training instances with the highestmemorization scores tend to be atypical, 2) R ETRO P ROMPT generalize better than fine-tuning andconvention prompt-tuning with decoupling knowledge from memorization to alleviate the rote of
Revised: The final analysis results show that 1) the training samples having the highest memorization scores are mostly atypical, 2) R ETRO P ROMPT generalize better than fine-tuning and convention prompt-tuning with decoupling knowledge from memorization to alleviate the rote of

Original: PLMs.
Revised: PLMs.

Original: In a nutshell, our work may open up new avenues to improve the generalization of prompting
Revised: In a nutshell, our work may open up new avenues to improve the generalization of prompting

Original: PLMs by decoupling knowledge from memorization.
Revised: PLMs by decoupling knowledge from memorization.

Original:  Preliminaries of Prompt Learning
Revised:  Preliminaries of Prompt Learning

Original: Assuming that M , T respectively denotes the PLM and the template function for prompt tuning.
Revised: Assuming that M , T respectively denotes the PLM and the template function for prompt tuning.

Original: Formally, the text classification task takes a query sentence x
Revised: Formally, the text classification task takes a query sentence x

Original: = ( x 0 , x 1 , ..., x n ) as input, and classifyit into a class label y ∈ Y .
Revised: = ( x 0 , x 1 , ..., x n ) as input. Then, classify it into the label y ∈ Y .

Original: While prompt learning converts classification task into a masked languagemodeling problem with cloze-style objectives.
Revised: While prompt learning converts the task into a MLM problem with cloze-style objectives.

Original: Specifically, the template function T inserts  piecesoftexts into x as ˆ x = T ( x ) , where ˆ x is thecorresponding input of M with a [MASK] token in it.
Revised: Specifically, the template function T inserts text pieces into x as ˆ x = T ( x ) , where ˆ x refers to the input of M with a [MASK] token.

Original: Forexample, assuming we need to classify the sentence x =“The movie makes absolutely no sense.”
Revised: For instances, when we have to classify the text x =“The movie makes absolutely no sense.”

Original: into[Figure] label N EGATIVE (labeled as 0) or P OSITIVE (labeled as 1), we wrap it into [Equation]
Revised: into label N EGATIVE (labeled as 0) or P OSITIVE (labeled as 1), we wrap it into ˆ x = [CLS] x It was [MASK][SEP] (1)

Original: CLS

Original: [Equation]
Revised: [Figure]

Original: It was [MASK][SEP] (1) The verbalizer f : Y (cid:55)→ V is defined as a mapping from the label space Y to a few words in thevocabulary, which form the label word set V .
Revised: The verbalizer f : Y (cid:55)→ V is defined as a mapping from the label space Y to those words in the vocabulary, which constructs the label word set V .

Original: The base component of M produces the sequencerepresentation over ˆ x , and we choose the hidden vector at the [MASK] position as the contextualrepresentation h ˆ x ∈ R d , where d is the dimension of hidden states.
Revised: The base component of M produces the sequence representation over ˆ x , and we choose the hidden vector at the [MASK] position as the contextual representation h ˆ x ∈ R d , where d is the dimension of hidden states.

Original: Then the MLM head of M canoperate on h ˆ x to calculatethe probability of each word v  in the vocabulary being filled in [MASK]token P M ( [MASK] = v | ˆ x ) .
Revised: Then the MLM head of M can operate on h ˆ x to calculate each word v ’s probability in the vocabulary being filled in [MASK] P M ( [MASK] = v | ˆ x ) .

Original: We let V y to represent the subset of V that is connected with a specificlabel y , ∪ y ∈Y V y = V .
Revised: We let V y to represent the subset of V which is connected with a unique label y , ∪ y ∈Y V y = V .

Original: Then the probability distribution over the label y is calculated as:
Revised: Finally, the probability distribution over the label y is calculated as:

Original: [Equation]

Original: [Equation]
Revised: [Equation]

Original: ) where g is a function transforming the probability of label words into the probability ofthe classes.
Revised: where g refers to the function converting the probability of label words to the probability of classes.

Original:  R ETRO P ROMPT : Retrieval-augmented Prompt Learning
Revised:  R ETRO P ROMPT : Retrieval-augmented Prompt Learning

Original: We introduce a simple and general retrieval-augmented framework for prompt learning, named
Revised: We introduce a simple and general retrieval-augmented framework for prompt learning, named

Original: R ETRO P ROMPT , whose basis is the dense retriever (§3.1) with an open-book knowledge-store todecouple knowledge from memorization.
Revised: R ETRO P ROMPT , whose basis is the dense retriever (§3.1) with an open-book knowledge-store to decouple knowledge from memorization.

Original: As shown in Figure 2, R ETRO P ROMPT consists of threecomponents: retrieval of neural demonstration for enhancing input (§3.2), the k NN guided training(§3.3) and the k NN-based probability for cloze-style prediction (§3.4).
Revised: As shown in Figure 2, R ETRO P ROMPT consists of three components: retrieval of neural demonstration for enhancing input (§3.2), the k NN guided training (§3.3) and the k NN-based probability for cloze-style prediction (§3.4).

Original: 3.1 Dense Retriever
Revised: 3.1 Dense Retriever

Original: Open-book Knowledge-store
Revised: Open-book Knowledge-store

Original: The first step of our proposed framework is to build a knowledgestore for retrieval that can decouple from memorization and captures the semantics of the instance fromthe training set C .
Revised: The first step of our proposed framework is to build a knowledgestore for retrieval that can decouple from memorization and captures the semantics of the instance from the training set C .

Original: Specifically, we utilize the encoder to embedprompt-based instance representationover the C to construct the knowledge-store.
Revised: Specifically, we leverage the encoder to embed instance representation over the C to construct the knowledge-store.

Original: Given the i -th example ( c i , y i ) in the training data C ,we compute the key-value pair ( h ˆ c i , v i ) , in which ˆ c i
Revised: Given the i -th example ( c i , y i ) in the training data C , we obtain the key-value pair ( h ˆ c i , v i ) , in which ˆ c i

Original: = T ( c i ) , 
Revised: = T ( c i ) , h

Original: h ˆ c i ∈ R
Revised: ˆ c i ∈ R

Original: d is the embedding of the[MASK] token in the last layer of theunderlying PLM, and v i =
Revised: d is the embedding of the [MASK] token in the last layer of the PLM, and v i =

Original: f ( y i ) denotes the label word ofthe i -th example.
Revised: f ( y i ) denotes the label word of the i -th example.

Original: Compared with kNN-LM [26] that constructing with sliding generative corpusand tokens, our knowledge-store is more suitable for prompt learning.
Revised: Compared with kNN-LM [27] that constructing with sliding generative corpus and tokens, our knowledge-store is more suitable for prompt learning.

Original: We store all pairs ( h ˆ c , v ) in akey-value datastore ( K , V )
Revised: We store all pairs ( h ˆ c , v ) in a key-value datastore ( K , V )

Original: where
Revised: where

Original: h ˆ c serves as key and v as value as follows:
Revised: h ˆ c serves as key and v as value as follows:

Original: [Equation]
Revised: [Equation]

Original: The knowledge-store is flexible to add, edit or delete any instances and can be asynchronously updatedduring the training procedure.
Revised: The knowledge-store maye be flexible to edit, add or delete any instances and can be asynchronously updated during the training procedure.

Original: Note that our knowledge-store is constructed from few-shot trainsetsin the corresponding few-shot settings rather than the whole available training data.
Revised: Note that our knowledge-store is constructed from few-shot trainsets in the corresponding few-shot settings rather than the whole available training data.

Original: Efficient Searching Considering that the size of the training data C can be enormous, we mustensure an efficient retrieval process.
Revised: Efficient Searching Considering that the size of the training data C can be enormous, we must ensure an efficient retrieval process.

Original: As shown in the above creation of open-book knowledge-store,we can build the matrix D ∈ R |C|× d as the index of training examples.
Revised: As shown in the above creation of open-book knowledge-store, we can build the matrix D ∈ R |C|× d as the index of training examples.

Original: Given a query set Q , wefirst encode each query example with template mapping function T ( · ) to get a set of prompt-basedquery vectors h ˆ q for retrieval augmentation on the fly.
Revised: Given a query set Q , we first encode each query example with template mapping function T ( · ) to get a set of prompt-based query vectors h ˆ q for retrieval augmentation on the fly.

Original: Then, we utilize query vectors to search forthe closest examples over the index D via maximum inner product search (MIPS).
Revised: Then, we utilize query vectors to search for the closest examples over the index D via maximum inner product search (MIPS).

Original: For the retrievalprocess, we choose FAISS [21] to query the open-book knowledge-store efficiently.
Revised: For the retrieval process, we choose FAISS [22] to query the open-book knowledge-store efficiently.

Original: FAISS is anexcellent open-source library for fast nearest neighbor retrieval in high-dimensional spaces.
Revised: FAISS is an excellent open-sourced toolkit for fast nearest neighbor retrieval.

Original: Asynchronous Refresh of the Knowledge-store Since the neural demonstration may lead to thevariable contextual representation of instance as the parameters of the PLM are continually updated,we thus propose to “refresh” the index of retrieval by asynchronously re-embedding and re-indexingall embeddings in an open-book knowledge-store every j training epochs 2 .
Revised: Asynchronous Refresh of the Knowledge-store Since the neural demonstration may lead to the variable contextual representation of instance as the parameters of the PLM are continually updated, we thus propose to “refresh” the index of retrieval by asynchronously re-embedding and re-indexing all embeddings in an open-book knowledge-store every j training epochs 4 .

Original: In § 4.6, we empiricallydemonstrate that this procedure results in performance improvement.
Revised: In § 4.6, we empirically demonstrate that this procedure results in performance improvement.

Original: 3.2 Retrieval of Neural Demonstration
Revised: 3.2 Retrieval of Neural Demonstration

Original: To enhance the PLMs with the ability to learn by analogy through the knowledge-store, we furtherpropose neural demonstrations that can be concatenated with input instance at the embedding layer toimprove the generalization ability of our R ETRO P ROMPT .
Revised: To enhance the PLMs with the ability to learn by analogy through the knowledge-store, we further propose neural demonstrations that can be concatenated with input instance at the embedding layer to improve the generalization ability of our R ETRO P ROMPT .

Original: For the t -th query instance q t , we firstutilize prompt-based representation h ˆ q t to query the cached representations of open-book knowledgestore.
Revised: For the t -th query instance q t , we first utilize prompt-based representation h ˆ q t to query the cached representations of open-book knowledgestore.

Original: Then we retrieve m nearest neighbors {{ c (1)1 , ..., c (1) m } , ..., { c ( L ) 1 , ..., c ( L ) m }} of q t for eachclass, where the superscript L denotes the total number of the classes and the c ( l ) i is retrieved as thei -th nearest neighbor in the l -th class.
Revised: Then we retrieve m nearest neighbors {{ c (1)1 , ..., c (1) m } , ..., { c ( L ) 1 , ..., c ( L ) m }} of q t for each class, where the superscript L denotes the total number of the classes and the c ( l ) i is retrieved as the i -th nearest neighbor in the l -th class.

Original: After the model retrieves the Top- m candidates for each class,their corresponding representation h ( l ) ˆ c i and label word v ( l ) from knowledge-store will be incorporatedinto the encoder to act as a demonstration learning.
Revised: After the model retrieves the Top- m candidates for each class, their corresponding representation h ( l ) ˆ c i and label word v ( l ) from knowledge-store will be incorporated into the encoder to act as a demonstration learning.

Original: Since the h ( l ) ˆ c i is already vector, we intuitivelyaggregate the m neighbor vectors for each class according to their similarity and incorporate thedemonstration into the input representation of ˆ x after the word embedding layer of the M as follows:
Revised: Since the h ( l ) ˆ c i is already vector, we intuitively aggregate the m neighbor vectors for each class according to their similarity and incorporate the demonstration into the input representation of ˆ x after the word embedding layer of the M as follows:

Original: [Equation]
Revised: [Equation]

Original: where e ( · ) represents the word embedding layer of M , ⊕ denotes the concatenation of input sequences, α ( l ) i is the softmax score for the i -th retrieval belonging to l -th class label to denote theirrelevance with ˆ q , and I is the sequence features for inputting the next layer of PLM.
Revised: where e ( · ) represents the word embedding layer of M , ⊕ denotes the concatenation of input sequences, α ( l ) i is the softmax score for the i -th retrieval belonging to l -th class label to denote their relevance with ˆ q , and I is the sequence features for inputting the next layer of PLM.

Original: As shown inthe above equation, we encode demonstration representation with the weighted sum of the retrievalrepresentation.
Revised: As shown in the above equation, we encode demonstration representation with the weighted sum of the retrieval representation.

Original: Thus, retrieval scores are directly used in the final representation, making the framework differentiable.
Revised: Thus, retrieval scores are directly used in the final representation, making the framework differentiable.

Original: To this end, we denote this style of demonstration as neural demonstration ,significantly different from prior work of discrete demonstration [13].
Revised: To this end, we denote this style of demonstration as neural demonstration , significantly different from prior work of discrete demonstration [13].

Original: Neural vs. Discrete Demonstration Compared with prior discrete demonstrations described in[13, 35, 49, 28], retrieving weighted neural demonstrations from the knowledge-store to augmentprompt learning has advantages in the following three major aspects: (1) neural demonstrationscould be more tolerant of the model’s maximum input length than discrete demonstrations, while thediscrete demonstration is usually not suitable for multi-class classification tasks due to the limitationof input length, such as relation extraction, etc.
Revised: Neural vs. Discrete Demonstration Compared with prior discrete demonstrations described in [13, 36, 50, 29], retrieving weighted neural demonstrations from the knowledge-store to augment prompt learning has advantages in the following three major aspects: (1) neural demonstrations could be more tolerant of the model’s maximum input length than discrete demonstrations, while the discrete demonstration is usually not suitable for multi-class classification tasks due to the limitation of input length, such as relation extraction, etc.

Original: (2) the model needs to deal with large retrieval tokensfor discrete demonstration, making it time-consuming and computationally intensive to performcross-attention operations due to the quadratic attention complexity.
Revised: (2) the model needs to deal with large retrieval tokens for discrete demonstration, making it time-consuming and computationally intensive to perform cross-attention operations due to the quadratic attention complexity.

Original: In contrast, dealing with muchshorter instance representations as neural demonstrations unleashes the potential of cross-attentionand accelerates the inference.
Revised: In contrast, dealing with much shorter instance representations as neural demonstrations unleashes the potential of cross-attention and accelerates the inference.

Original: (3) when sampling examples based on the similarity between instances,our cloze-style contextual representation is more informative and consistent than the contextualrepresentation from [CLS] of Sentence-BERT [47] (adopted in LM-BFF).
Revised: (3) when sampling examples based on the similarity between instances, our cloze-style contextual representation is more informative and consistent than the contextual representation from [CLS] of Sentence-BERT [48] (adopted in LM-BFF).

Original: 3.3 Retrieve k NN for Guiding Training
Revised: 3.3 Retrieve k NN for Guiding Training

Original: Eager learners, such as PLMs, are trained to provide a globalapproximating function that maps frominput to output space.
Revised: Eager learners (e.g., PLMs) are optimized to learn a global function that maps from the text to semantic label space.

Original: Lazy learners such as k -nearest neighbor classifiers, on the contrary, focuson approximating the neighborhoods around  test examples [2].
Revised: Lazy learners such as k -nearest neighbor classifiers, on the contrary, aims to approximating the neighborhoods around those test examples [2].

Original: Since k NN can easily predict foreach encountered query instance based on pre-trained representation without an extra classifier, it isintuitively to leverage the k NN’s classification results as the prior external knowledge to guide the
Revised: Since k NN can easily predict for each encountered query instance based on pre-trained representation without an extra classifier, it is intuitively to leverage the k NN’s classification results as the prior external knowledge to guide the

Original: PLMs’ parameters attending to hard examples (hard samples usually refer to atypical samples) duringthe training process (also referred as k NN-train for the abbreviation).
Revised: PLMs’ parameters attending to hard examples (hard samples usually refer to atypical samples) during the training process (also referred as k NN-train for the abbreviation).

Original: Particularly, our intuition isto differentiate between easy and hard examples according to the prediction of k NN.
Revised: Particularly, our intuition is to differentiate between easy and hard examples according to the prediction of k NN.

Original: Given the t -thquery instance q t , we leverage the h q t querying the open-book knowledge-store ( K , V ) to retrievethe k -nearest neighbors N of q t according to a similarity function d ( · , · ) , where d ( · , · ) typically adoptthe inner product similarity.
Revised: Given the t -th query instance q t , we leverage the h q t querying the open-book knowledge-store ( K , V ) to retrieve the k -nearest neighbors N of q t according to a similarity function d ( · , · ) , where d ( · , · ) typically adopt the inner product similarity.

Original: Then, we compute a distribution over neighbors based on a softmax oftheir similarities and aggregate probability mass for each label word acrossall its occurrences in theretrieved targets:
Revised: Then, we compute the distribution over neighbors according to the softmax of their similarities and aggregate probability mass for each label word across its occurrences in the retrieved targets:

Original: [Equation]
Revised: [Equation]

Original: Given the probability p k NN of the query instance q t being predicted as the gold class (also as theprobability value of the gold class in the P k NN ), we propose to retrieve the k NN for guiding thetraining process of prompt learning.
Revised: Given the probability p k NN of the query instance q t being predicted as the gold class (also as the probability value of the gold class in the P k NN ), we propose to retrieve the k NN for guiding the training process of prompt learning.

Original: The k NN guider reweights the cross-entropy loss L CE byadjusting the relative loss for the correctly-classified or misclassified instances identified by k NN,respectively.
Revised: The k NN guider reweights the cross-entropy loss L CE by adjusting the relative loss for the correctly-classified or misclassified instances identified by k NN, respectively.

Original: Specifically, we apply the negative log-likelihood as the modulating factor F ( p k NN ) .
Revised: Specifically, we apply the negative log-likelihood as the modulating factor F ( p k NN ) .

Original: The final loss L is defined as:
Revised: The final loss L is defined as:

Original: [Equation]
Revised: [Equation]

Original: where β denotes a scalar to determine the proportion of each loss term.
Revised: where β denotes a scalar to determine the proportion of each loss term.

Original: Note that p k NN is computedusing the leave-one-out distribution on the training set due to the fact that each example in the trainingset cannot retrieve itself.
Revised: Note that p k NN is computed using the leave-one-out distribution on the training set due to the fact that each example in the training set cannot retrieve itself.

Original: The motivation of modulating factor is inspired by Focal-loss [34], whilewe focus on exploit leveraging k-NN’s results for calibrating the training of LMs.
Revised: The motivation of modulating factor is inspired by Focal-loss [35], while we focus on exploit leveraging k-NN’s results for calibrating the training of LMs.

Original: 3.4 k NN based probability for Cloze-style Prediction
Revised: 3.4 k NN based probability for Cloze-style Prediction

Original: Apart from the neural demonstration on the input side and k NN guided training process (also referredas k NN-test for the abbreviation), we further present k NN based probability for Cloze-style predictionon the inference process, providing the PLM ability to retrieve nearest neighbors for decisions ratherthan making predictions only based on memorized parameters.
Revised: Apart from the neural demonstration on the input side and k NN guided training process (also referred as k NN-test for the abbreviation), we further present k NN based probability for Cloze-style prediction on the inference process, providing the PLM ability to retrieve nearest neighbors for decisions rather than making predictions only based on memorized parameters.

Original: Given the non-parametric k nearestneighbor distribution P k NN of the query instance q t being predicted as y , we follow [14, 26, 17]to reformulate the P ( y | q t ) by interpolating the P k NN with the already-trained base PLM’s MLMprediction P M using parameter λ to produce the final probability of the label:
Revised: Given the non-parametric k nearest neighbor distribution P k NN of the query instance q t being predicted as y , we follow [14, 27, 17] to reformulate the P ( y | q t ) by interpolating the P k NN with the already-trained base PLM’s MLM prediction P M using parameter λ to produce the final probability of the label:

Original: [Equation]
Revised: [Equation]

Original: Different from k NN-LM [26, 17] that mainly retrieve tokens to augment the language modeling, wefocus on leveraging prompt-based kNN’s distribution for reference at test time, which can unlock themodel prediction process as an open-book examination for prompt learning.
Revised: Different from k NN-LM [27, 17] that mainly retrieve tokens to augment the language modeling, we focus on leveraging prompt-based kNN’s distribution for reference at test time, which can unlock the model prediction process as an open-book examination for prompt learning.

Original:  Experiments 
Revised:  Experiments 4.1 Datasets and Baselines

Original: 4.1 Datasets and Baselines

Original: Datasets We evaluate R ETRO P ROMPT on several types of natural language understanding tasks,including single sentence classification tasks (SST-2 [53], MR [43], and CR [19]) and sentence pairclassification tasks (MNLI [56], QNLI [46], and QQP 3 ).
Revised: Datasets We evaluate R ETRO P ROMPT on several types of natural language understanding tasks, including single sentence classification tasks (SST-2 [55], MR [44], and CR [20]) and sentence pair classification tasks (MNLI [59], QNLI [47], and QQP 5 ).

Original: To further evaluate the effectiveness of theproposed approach with multi-class classification, we also conduct experiments on the informationextraction tasks, including FewNERD [9], SemEval 

Revised: To further evaluate the effectiveness of the proposed approach with multi-class classification, we also conduct experiments on the information extraction tasks, including FewNERD [9], SemEval

Original: Task 8 (SemEval)
Revised: Task 8 (SemEval)

Original: [18], and TACRED [59].
Revised: [18], and TACRED [63].

Original: [Table]
Revised: The detailed statistics of the datasets are shown in Appendix A.

Original: Baselines We compare with LM-BFF [13] for single sentence and sentence pair classification tasksand adopt SOTA prompt learning model KnowPrompt [6] as the baseline for information extractiontasks.
Revised: Baselines We compare with LM-BFF [13] for single sentence and sentence pair classification tasks and adopt SOTA prompt learning model KnowPrompt [6] as the baseline for information extraction tasks.

Original: Note that the discrete demonstration method cannot be applied to multi-class classificationtasks due to the input length limitations; thus, we leave out the experimental table about the resultsof KnPr (D-demo).
Revised: Note that the discrete demonstration method cannot be applied to multi-class classification tasks due to the input length limitations; thus, we leave out the experimental table about the results of KnPr (D-demo).

Original: We also compare our R ETRO P ROMPT with the knowledge-enhanced promptlearning method KPT [20] since KPT leverages the external knowledge base for enhancing promptlearning while we focus on utilizing internal trainsets as a knowledge-store.
Revised: We also compare our R ETRO P ROMPT with the knowledge-enhanced prompt learning [Table] [Table] [Table]

Revised: [Table]

Revised: Ours

Revised: [Table]

Revised: [Table]

Revised: [Table]

Revised: [Table]

Revised: LOTClass ♣

Revised: [Table]

Revised: [Table]

Revised: ♣

Revised: [Table]

Revised: FT

Revised: [Table]

Revised: [Table]

Revised: [Table]

Revised: [Table]

Revised: FT

Revised: [Table]

Revised: [Table]

Revised: Ours

Revised: [Table]

Revised: method KPT [21] since KPT leverages the external knowledge base for enhancing prompt learning while we focus on utilizing internal trainsets as a knowledge-store.

Revised: You can refer to Appendix B for detailed introduction of baseline methods.

Original: 4.2 Evaluation protocols and details
Revised: 4.2 Evaluation protocols and details

Original: The experiments are implemented on 1 NVIDIA V100 and utilize Pytorch [44] as the base library.
Revised: The experiments are implemented on 1 NVIDIA V100 and utilize Pytorch [45] as the base library.

Original: We adopt RoBERTa large [38] as the PLM and employ AdamW as the optimizer for all experiments.
Revised: We adopt RoBERTa large [39] as the PLM and employ AdamW as the optimizer for all experiments.

Original: To mitigate the influence of diverse templates, we conduct baselines and R ETRO P ROMPT with thesame templates for each dataset.
Revised: To mitigate the influence of diverse templates, we conduct baselines and R ETRO P ROMPT with the same templates for each dataset.

Original: The specific templates we use for each dataset are in Appendix. Asfor few-shot and zero-shot experiments, we leverage different settings, respectively. 
Revised: We list the specific experimental settings and tuning retrieve parameters in Appendix C and D. As for few-shot and zero-shot experiments, we leverage different settings, respectively. [Figure]

Original: Few-shot Setting.
Revised: Few-shot Setting.

Original: We follow the few-shot setting of LM-BFF [13] to conduct 4-shot and 16-shotexperiments and evaluate the average performance with a fixed set of seeds, S seed , across five differentsampled D train for each task.
Revised: We follow the few-shot setting of LM-BFF [13] to conduct 4-shot and 16-shot experiments and evaluate the average performance with a fixed set of seeds, S seed , across several different sampled D train for each task.

Original: Note that our knowledge-store is constructed with the few-shot trainingset in this setting.
Revised: Note that our knowledge-store is constructed with the few-shot training set in this setting.

Original: Zero-shot Setting 4 .
Revised: Zero-shot Setting 6 .

Original: We leverage vanilla RoBERTa large for all baselines (except LOTClass [41])to directly inference on the test set.
Revised: We leverage vanilla RoBERTa large for all baselines (except LOTClass [42]) to directly inference on the test set.

Original: To take advantage of retrieval mechanism, R ETRO P ROMPTfollows LOTClass [41] to utilize unlabeled trainsets for retrieval.
Revised: To take advantage of retrieval mechanism, R ETRO P ROMPT follows LOTClass [42] to utilize unlabeled trainsets for retrieval.

Original: Specifically, we take the vanilla RoBERTa large to tag the pseudo labels on unlabeled trainset and create the open-book knowledge-storewith the unlabeled trainsets and pseudo labels.
Revised: Specifically, we take the vanilla RoBERTa large to tag the pseudo labels on unlabeled trainset and create the open-book knowledge-store with the unlabeled trainsets and pseudo labels.

Original: Lastly, R ETRO P ROMPT make predictions on the testset based on the constructed datastore without tuning any of the model parameters .
Revised: Lastly, R ETRO P ROMPT make predictions on the test set based on the constructed datastore without tuning any of the model parameters .

Original: 4.3 Experimental Results
Revised: 4.3 Experimental Results

Original: Few-shot Results.
Revised: Few-shot Results.

Original: As shown in Table 1, we find R ETRO P ROMPT consistently outperforms baselinemethod LM-BFF and KnowPrompt, both in 4-shot and 16-shot experiments.
Revised: As shown in Table 1, we find R ETRO P ROMPT consistently outperforms baseline method LM-BFF and KnowPrompt, both in 4-shot and 16-shot experiments.

Original: Especially for information extraction tasks with multiple classes, discrete demonstrations cannot be applied to the input due to the limited input sequence length, while our neural demonstration can also work and achievesimprovement on these multi-class datasets.
Revised: Especially for information extraction tasks with multiple classes, discrete demonstrations cannot be applied to the input due to the limited input sequence length, while our neural demonstration can also work and achieves improvement on these multi-class datasets.

Original: Moreover, R ETRO P ROMPT obtain better performancecompared with KPT.
Revised: Moreover, R ETRO P ROMPT obtain better performance compared with KPT.

Original: Compared with KPT with external knowledge, we only focus on referencing theinternal few-shot trainsets without visiting the external knowledge base.
Revised: Compared with KPT with external knowledge, we only focus on referencing the internal few-shot trainsets without visiting the external knowledge base.

Original: Besides, we observe that
Revised: Besides, we observe that

Original: R ETRO P ROMPT has a relatively lower standard deviation than the baselines.
Revised: R ETRO P ROMPT has a relatively lower standard deviation than the baselines.

Original: The reason may lie thatthe retrieval mechanism can compensate for instabilities in parametric predictions.
Revised: The reason may lie that the retrieval mechanism can compensate for instabilities in parametric predictions.

Original: [Figure]
Revised: [Table]

Original: Zero-shot Results.
Revised: Zero-shot Results.

Original: From Table 1, we also observe that R ETRO- [Figure] P ROMPT achieves improvements in the zero-shot setting.
Revised: From Table 1, we also observe that R ETRO P ROMPT achieves improvements in the zero-shot setting.

Original: Anothernotable point is that R ETRO P ROMPT performs even better than KPTin the zero-shot setting, revealing that exploring own data to decoupleknowledge from memorization has more potential than leveragingexternal knowledge.
Revised: Another notable point is that R ETRO P ROMPT performs even better than KPT in the zero-shot setting, revealing that exploring own data to decouple knowledge from memorization has more potential than leveraging external knowledge.

Original: Moreover, we achieve superior performance to LOTClass even though we utilize the vanilla RoBERTa large withoutany training.
Revised: Moreover, we achieve superior performance to LOTClass even though we utilize the vanilla RoBERTa large without any training.

Original: Fully-supervised Results.
Revised: Fully-supervised Results.

Original: As shown in Figure 3, the experimentsin fully-supervised settings with long-tail distribution illustrate that R ETRO P ROMPT achieves improvement compared with baselines.
Revised: As shown in Figure 3, the experiments in fully-supervised settings with long-tail distribution illustrate that R ETRO P ROMPT achieves improvement compared with baselines.

Original: Thisindicates that our retrieval mechanism extends the LM’s ability to learn hard examples in the fullysupervised datasets.
Revised: This indicates that our retrieval mechanism extends the LM’s ability to learn hard examples in the fullysupervised datasets.

Original: 4.4 Model Generalization to New Domains
Revised: 4.4 Model Generalization to New Domains

Original: [Figure]

Original: [Table]

Original: [Figure]

Original: [Table] [Figure] [Table] The scarce data may bring the overfitting problem for the lotsof memory parameters of PLMs, even though prompt learning.
Revised: The scarce data may bring the overfitting problem for the lots of memory parameters of PLMs, even though prompt learning.

Original: Thus, we conduct cross-domain experiments to validate thegeneralization of our R ETRO P ROMPT .
Revised: Thus, we conduct cross-domain experiments to validate the generalization of our R ETRO P ROMPT .

Original: Specifically, we utilizethe model trained on the source datasets and directly test on theother target datasets.
Revised: Specifically, we utilize the model trained on the source datasets and directly test on the other target datasets.

Original: From Table 2, we can find that our methodconsistently outperforms baselines.
Revised: From Table 2, we can find that our method consistently outperforms baselines.

Original: This finding illustrates that R ETRO P ROMPT achieves great model generalization to newdomains.
Revised: This finding illustrates that R ETRO P ROMPT achieves great model generalization to new domains.

Original: 4.5 Analysis of Memorization
Revised: 4.5 Analysis of Memorization

Original: It is necessary and interesting to further explore the memorization mechanism to help us betterunderstand the utility of retrieval for memorization in NLP.
Revised: It is necessary and interesting to further explore the memorization mechanism to help us better understand the utility of retrieval for memorization in NLP.

Original: Definition of Memorization Measurement.
Revised: Definition of Memorization Measurement.

Original: Inspired by the idea of [11] in the computer visionarea, we define memorization measures as to how the classification varies when a training instance zis deleted from the trainset.
Revised: Inspired by the idea of [11] in the computer vision area, we define memorization measures as to how the classification varies when a training instance z is deleted from the trainset.

Original: We follow [27, 60] to define and derive the memorization score for atraining instance z as follows:
Revised: We follow [28, 64] to define and derive the memorization score for a training instance z as follows:

Original: [Equation]
Revised: [Equation]

Original: where ˆ θ ξ, − z denotes the parametersof the model trained with the instance z down-weighted by ξ , ˆ θ isthe parameters of the model trained with all instances and H ˆ θ = 1 n (cid:80) ni
Revised: where ˆ θ ξ, − z denotes the parameters trained with the instance z down-weighted by ξ , ˆ θ refers to the parameters of the model trained with all instances and H ˆ θ = 1 n (cid:80) ni

Original: =1 ∇ 2 θ L ( z i , ˆ θ ) .
Revised: =1 ∇ 2 θ L ( z i , ˆ θ ) .

Original: Thus S delate ( z )is the amount of change of P ( y | x ; θ ) when the instance z is down-weighted bya small amount ξ .
Revised: Thus S delate ( z ) refers to the amount of change of P ( y | x ; θ ) when the instance z is down-weighted by ξ .

Original: Top-memorized Instances: Typical or Atypical?
Revised: Top-memorized Instances: Typical or Atypical?

Original: Since the SST-2 dataset provides the annotationsof phrase-level sentiment polarity labels, we adopt SST-2 to analyze the memorization by judging theatypical of an instance by checking the percentage of positive phrases.
Revised: Since the SST-2 dataset provides the annotations of phrase-level sentiment polarity labels, we adopt SST-2 to analyze the memorization by judging the atypical of an instance by checking the percentage of positive phrases.

Original: We collect such statistics from SST-2 and find that a typical positive instance has a relatively high percentage of positive phrases,and a typical negative instance should have a relatively low percentage of positive phrases.
Revised: We achieve such statistics from SST-2 and observe that a typical positive instance has a relatively high percentage of positive phrases, and a typical negative instance should have a relatively low percentage of positive phrases.

Original: Basedon the above observation, we apply the memorization score defined in Eq.
Revised: Based on the above observation, we apply the memorization score defined in Eq.

Original: 8 to select Top-10% and Bottom-10% memorized instances from the trainset and collect the average percentage of positivephrases in these instances.
Revised: 8 to select Top-10% and Bottom-10% memorized instances from the trainset and collect the average percentage of positive phrases in these instances.

Original: [Table] As shown in Table 3,we can conclude following findings: (1) The PLM tends to give atypical samplesdeeper memory attention.
Revised: As shown in Table 3,we can conclude following findings: (1) The PLM tends to give atypical samples deeper memory attention.

Original: Specifically, no matter LM-BFF or our method, the top-10% memorizednegative instances have a higher percentage of positive phrases than the average percentage of positive phrases of all negative instances.
Revised: Specifically, no matter LM-BFF or our method, the top- [Table] 10% memorized negative instances have a higher percentage of positive phrases than the average percentage of positive phrases of all negative instances.

Original: 2) LM-BFFhas lower memorization scores on hard samples thanfine-tuning.
Revised: 2) LM-BFF has lower memorization scores on hard samples than fine-tuning.

Original: We think it owns to prompt learningcan help PLMs recall what they learned from pretraining without strengthening memory for downstream data. 
Revised: We think it owns to prompt learning can help PLMs recall what they learned from pre-training without strengthening memory for downstream data. 3)

Original: 3) R ETRO P ROMPT further has loweraverage memorization scores than fine-tuning and LM-BFF, which illustrates that our method is less memory dependent.
Revised: R ETRO P ROMPT further has lower average memorization scores than fine-tuning and LM-BFF, which illustrates that our method is less memory dependent.

Original: This result may be attributedto decoupling knowledge from memorization through retrieval to alleviating the rote of PLMs.
Revised: This result may be attributed to decoupling knowledge from memorization through retrieval to alleviating the rote of PLMs.

Original: [Table]
Revised: [Table]

Original: Case Analysis.
Revised: Case Analysis.

Original: As shown in Table 6, we manually list thetop-ranked and bottom-ranked traininginstances of SST-2 according to our model.
Revised: As shown in Table 6, we manually list the bottom-ranked and top-ranked training instances of SST-2 according to our model.

Original: It reveals that the top-ranked memorized instances seemto show universal opinions indirectly.
Revised: It reveals that the top-ranked memorized instances seem to show universal opinions indirectly.

Original: Thus, we inspect them as atypical/hard for sentiment classification.
Revised: Thus, we inspect them as atypical/hard for sentiment classification.

Original: While those instances with 0 memorizationscores are straightforward to show their opinion forsentiment classification, representing the typical instance.
Revised: While those instances with 0 memorization scores are straightforward to show their opinion for sentiment classification, representing the typical instance.

Original: Note that F ( p kNN ) is defined to representthe difficulty of the sample discriminated by k NN distribution.
Revised: Note that F ( p kNN ) is defined to represent the difficulty of the sample discriminated by k NN distribution.

Original: And the Table 6 also shows that F ( p kNN )indeed reflect atypicality of examples, which validate the effectiveness of the k NN guided training.
Revised: And the Table 6 also shows that F ( p kNN ) indeed reflect atypicality of examples, which validates the effectiveness of the k NN guided training.

Original: 4.6 Ablation Study
Revised: 4.6 Ablation Study

Original: Component Ablation
Revised: Component Ablation.

Original: As shown in Table 4, the performance of component ablation experimentswith four variants has a clear drop, which proves the effectiveness of our retrieval component.
Revised: As shown in Table 4, the performance of component ablation experiments with four variants has a clear drop, which validates the power of our retrieval component.

Original: Wealso find that neural demonstration and k NN-train have more improvement in the few-shot settingthan k NN-test.
Revised: We also find that neural demonstration and k NN-train have more improvement in the few-shot setting than k NN-test.

Original: Note that k NN-test is similar to k NN-LM [26, 17] and the results reveals that simplyincorporate k NN in the test process of prompt learning has little influence in a few-shot setting.
Revised: Note that k NN-test is similar to k NN-LM [27, 17] and the results reveals that simply incorporate k NN in the test process of prompt learning has little influence in a few-shot setting.

Original: [Table]
Revised: [Table]

Original: Key Representation and k NN Acquisition.
Revised: Key Representation and k NN Acquisition.

Original: We studythe effect of using different representations of the key in theknowledge-store.
Revised: We study the effect of using different representations of the key in the knowledge-store.

Original: We experiment with two types of representations: (1) prompt-based representation, which is thedefault setting, and (2) [CLS] based representation of current
Revised: We experiment with two types of representations: (1) prompt-based representation, which is the default setting, and (2) [CLS] based representation of current

Original: LM.
Revised: LM.

Original: We also experiment with two types of calculation ofk NN distribution: (1) representation based similarity score(refer as rep-similar), which is the default setting, and (2)
Revised: We also experiment with two types of calculation of k NN distribution: (1) representation [Table] similarity score (refer as rep-similar), which is the default setting, and (2)

Original: BM25 based score , which calculates the correlation scorebetween the query and each key examples with BM25 [48]algorithm.
Revised: BM25 based score , which calculates the correlation score between the query and each key examples with BM25 [49] algorithm.

Original: Results in Table 5 show that using prompt-based representations for key and representationbased similarity scores for k NN leads to the best performance.
Revised: Results in Table 5 show that using prompt-based representations for key and representation based similarity scores for k NN leads to the best performance.

Original: It suggests that prompt learn betterrepresentations for context similarity and the representation similarity based k NN distribution isbetter than BM25 based scores.
Revised: It suggests that prompt learn better representations for context similarity and the representation similarity based k NN distribution is better than BM25 based scores.

Original: [Table]
Revised: [Table]

Original:  Related Work
Revised:  Related Work

Original: [Table]

Original: Retrieval-enhanced PLMs.
Revised: Retrieval-enhanced PLMs.

Original: Our pipeline is partly inspired by discrete demonstration methodssuch as [13, 35, 28, 29] that retrieves few training examples in a natural language prompt, whilewe propose neural demonstration for enhancing the input to alleviate the limitations of input length.
Revised: Our pipeline is partly inspired by discrete demonstration methods such as [13, 36, 29, 30] that retrieves few training examples in a natural language prompt, while we propose neural demonstration for enhancing the input to alleviate the limitations of input length.

Original: Another line researches of retrieval augmentation [15, 23, 32, 49, 3] retrieve useful information froma external knowledge corpus (e.g., Wikipedia) for a particular task (e.g., an open-domain question).
Revised: Another line researches of retrieval augmentation [15, 24, 33, 50, 3] retrieve useful information from a external knowledge corpus (e.g., Wikipedia) for a particular task (e.g., an open-domain question).

Original: Unlike these works, we focus on retrieving examples from the internal training data.
Revised: Unlike these works, we focus on retrieving examples from the internal training data.

Original: Besides, semiparametric methods [26, 17,  25,24, 1, 42] have risen to leverage k -nearest neighbor classifier thatmakes the prediction based on representation similarities, to enhance pre-trained language models. However, unlike these models using nearest neighbors only for augmenting the process of prediction,we aim to develop a comprehensive retrieval mechanism for input, training and test process.
Revised: Besides, semiparametric methods [27, 17, 26, 25, 1, 43] have risen to leverage k -nearest neighbor classifier, a classic non-parametric algorithm that makes the prediction based on representation similarities, to enhance pre-trained language models in various tasks However, unlike these models using nearest neighbors only for augmenting the process of prediction, we aim to develop a comprehensive retrieval mechanism for input, training and test process.

Original: Prompt learning for PLMs.
Revised: Prompt learning for PLMs.

Original: With the birth of GPT-3 [4], prompt learning [36] has recentlyarisen to fill the gap between masked LM objective of PLMs and downstream fine-tuning objective.
Revised: With the birth of GPT-3 [4], prompt learning [37] has recently arisen to fill the gap between masked LM objective of PLMs and downstream fine-tuning objective.

Original: Prompt learning has achieves very impressive performance on various tasks [51, 52, 5, 40, 16, 6],especially under the setting of few-shot learning.
Revised: Prompt learning has achieves very impressive performance on various tasks, such as text classification [52, 54], named entity recognition [5, 41], relation extraction [16, 6], event extraction [19, 61], machine translation [56] and language generation [53], especially under the setting of few-shot learning.

Original: Moreover, continuous prompts have also beenproposed [33, 30, 37] to reduce prompt engineering, which directly appends a series of learnablecontinuous embeddings as prompts into the input sequence.
Revised: Moreover, continuous prompts have also been proposed [34, 31, 38] to reduce prompt engineering, which directly appends a series of learnable continuous embeddings as prompts into the input sequence.

Original: Our work is orthogonal to previousprompt learning approaches, which aim to optimize prompts, while we focus on the systematic studyof retrieving related examples from training data to enhance prompt learning.
Revised: Our work is orthogonal to previous prompt learning approaches, which aim to optimize prompts, while we focus on the systematic study of retrieving related examples from training data to enhance prompt learning.

Original:  Conclusion and Future Work
Revised:  Conclusion and Future Work

Original: We propose R ETRO P ROMPT that decouples knowledge from memorization by introducing retrievalaugmentation to further improve the generalization ability of prompt learning on the input side andthe whole process of model training and prediction.
Revised: We propose R ETRO P ROMPT that decouples knowledge from memorization by introducing retrieval augmentation to further improve the generalization ability of prompt learning on the input side and the whole process of model training and prediction.

Original: R ETRO P ROMPT , is a straightforward yet effectiveretrieval method that combines both neural demonstrations, k NN guider for training and prediction.
Revised: R ETRO P ROMPT , is a straightforward yet effective retrieval method that combines both neural demonstrations, k NN guider for training and prediction.

Original: Our extensive results show that it outperforms other demonstration-enhanced prompt methods andknowledge-enhanced prompt methods in few-shot, zero-shot and fully-supervised settings.
Revised: Our extensive results show that it outperforms other demonstration-enhanced prompt methods and knowledge-enhanced prompt methods in few-shot, zero-shot and fully-supervised settings.

Original: Analyzingthe essence of memorization validates the effectiveness of decoupling knowledge from memorization.
Revised: Analyzing the essence of memorization validates the effectiveness of decoupling knowledge from memorization.

Original: Interesting future directions include: 1) apply to other tasks, such as QA and NLG, 2) explore thenoise data mining for unsupervised learning, 3) further improve the retrieve efficiency for largedatasets, etc. 
Revised: Interesting future directions include: 1) apply to other tasks, such as QA and NLG, 2) explore the noise data mining for unsupervised learning, 3) further improve the retrieve efficiency for large datasets, etc. Acknowledgments

Revised: We want to express gratitude to the anonymous reviewers for their kind comments.

Revised: This work was supported by National Natural Science Foundation of China (No.62206246, 91846204 and

Revised: U19B2027), Zhejiang Provincial Natural Science Foundation of China (No. LGG22F030011),

Revised: Ningbo Natural Science Foundation (2021J190), and Yongjiang Talent Introduction Programme (2021A-156-G).

Original: A Datasets and Templates
Revised: Checklist

Revised: For all authors... (a) Do the main claims made in the abstract and introduction accurately reflect the paper’s contributions and scope? [Yes] (b) Did you describe the limitations of your work? [Yes] See Appendix.

Revised: (c) Did you discuss any potential negative societal impacts of your work?

Revised: [Yes] See Appendix.

Revised: (d) Have you read the ethics review guidelines and ensured that your paper conforms to them?

Revised: [Yes]

Revised: If you are including theoretical results... (a) Did you state the full set of assumptions of all theoretical results?

Revised: [N/A] (b) Did you include complete proofs of all theoretical results?

Revised: [N/A] 3.

Revised: If you ran experiments... (a) Did you include the code, data, and instructions needed to reproduce the main experimental results (either in the supplemental material or as a URL)?

Revised: [Yes] We include the source code and data in our supplemental material submission, and we outline the data generation procedure, the evaluation protocol, the training regime, and everything else necessary for reproduction either in the main body of the paper or in the appendix.

Revised: (b) Did you specify all the training details (e.g., data splits, hyperparameters, how they were chosen)?

Revised: [Yes] See Subsection 4.2 and Appendix.

Revised: (c) Did you report error bars (e.g., with respect to the random seed after running experiments multiple times)?

Revised: [Yes] We list the standard deviation for few-shot setting.

Revised: (d) Did you include the total amount of compute and the type of resources used (e.g., type of GPUs, internal cluster, or cloud provider)?

Revised: [Yes] We introduce type of resources in Section 4.2. 4.

Revised: If you are using existing assets (e.g., code, data, models) or curating/releasing new assets... (a) If your work uses existing assets, did you cite the creators?

Revised: [Yes] (b) Did you mention the license of the assets?

Revised: [No] The code and the data are proprietary.

Revised: (c) Did you include any new assets either in the supplemental material or as a URL?

Revised: [No] (d) Did you discuss whether

Revised: and how consent was obtained from people whose data you’re using/curating?

Revised: [No] The code and the data are proprietary.

Revised: (e) Did you discuss whether the data you are using/curating contains personally identifiable information or offensive content?

Revised: [No] 5.

Revised: If you used crowdsourcing or conducted research with human subjects... (a) Did you include the full text of instructions given to participants and screenshots, if applicable?

Revised: [N/A] (b) Did you describe any potential participant risks, with links to Institutional Review Board (IRB) approvals, if applicable?

Revised: [N/A] (c) Did you include the estimated hourly wage paid to participants and the total amount spent on participant compensation?

Original:  In this section, we introduce theinformation of datasets as shown in Table 7 and list the templates weuse in experiments as follows. 
Revised: [N/A] A Datasets and Templates In this section, we introduce the datasets as shown in Table 7 and list the templates we use in experiments as follows. [Table]

Original: SST-2, MR, CR .
Revised: SST-2, MR, CR .

Original: For the single sentence classification tasks, we follow the LM-BFF [13] to designthe templates:
Revised: For the single sentence classification tasks, we follow the LM-BFF [13] to design the templates:

Original: T ( x ) =
Revised: T ( x ) =

Original: [CLS] x It was [MASK] .
Revised: [CLS] x It was [MASK] .

Original: We set Verbalizer: (great/terrible) →
Revised: We set Verbalizer: (great/terrible) →

Original: (positive/negative) for SST-2 MR and CR.
Revised: (positive/negative) for SST-2 MR and CR.

Original: For the Yahoo dataset,we assign the Verbalizer following the original labels.
Revised: For the Yahoo dataset, we assign the Verbalizer following the original labels.

Original: MNLI, QNLI, QQP .
Revised: MNLI, QNLI, QQP .

Original: For the sentence pair classification tasks, we follow LM-BFF [13] to set
Revised: For the sentence pair classification tasks, we follow LM-BFF [13] to set

Original: Verbalizer: (Yes/Maybe/No) →
Revised: Verbalizer: (Yes/Maybe/No) →

Original: (entailment/neutral/contradiction), and define the following templates:
Revised: (entailment/neutral/contradiction), and define the following templates:

Original: [Table]
Revised: T ( x 1 , x 2 ) = [CLS] x 1 ? [MASK] , x

Original: FewNERD, SemEval, TACRED .
Revised: FewNERD, SemEval, TACRED .

Original: FewNERD, SemEval and TACRED are datasets for informationextraction, which require inserting the entity into the template.
Revised: FewNERD, SemEval and TACRED are datasets for information extraction, which require inserting the entity into the template.

Original: Therefore, we follow [8] and [6] todefine the template and verbalizers.
Revised: Therefore, we follow [8] and [6] to define the template and verbalizers.

Original: B Compared Baselines
Revised: B Compared Baselines

Original: In this subsection, we introduce the baselines we compare with and re-produce them under the samesettings with their open-source codes.
Revised: In this subsection, we introduce the baselines we compare with and re-produce them under the same settings with their open-source codes.

Original: LM-BFF uses several other tricks, such as prompt ensemble, while KPT utilizes tremendous externalknowledge.
Revised: LM-BFF uses several other tricks, such as prompt ensemble, while KPT utilizes tremendous external knowledge.

Original: We do not use any of these tricks and external knowledge since we get the most out of thedata to decouple part of knowledge from parametric memorization.
Revised: We do not use any of these tricks and external knowledge since we get the most out of the data to decouple part of knowledge from parametric memorization.

Original: Our R ETRO P ROMPT mechanismis orthogonal to other methodological improvements of prompt-tuning (such as continuous prompt in
Revised: Our R ETRO P ROMPT mechanism is orthogonal to other methodological improvements of prompt-tuning (such as continuous prompt in

Original: P-tuning [37] and DART [58] ) and can be combined with other prompt-tuning methods in futurework.
Revised: P-tuning [38] and DART [62] ) and can be combined with other prompt-tuning methods in future work.

Original: Fine-tuning (FT).
Revised: Fine-tuning (FT).

Original: The traditional fine-tuning method regard the hidden embedding of [CLS]token of the PLM as the representation of the sentence and then feeds them into a classification layerto make predictions.
Revised: The traditional fine-tuning method regard the hidden embedding of [CLS] token of the PLM as the representation of the sentence and then feeds them into a classification layer to make predictions.

Original: LM-BFF.
Revised: LM-BFF.

Original: LM-BFF [13] is a typical prompt-tuning method wrapping an input sentence into ahandcrafted template.
Revised: LM-BFF [13] is a typical prompt-tuning method wrapping an input sentence into a handcrafted template.

Original: Here we re-produce LM-BFF based on their open-source codes 5 with thesame manual prompts as R ETRO P ROMPT for a fair comparison.
Revised: Here we re-produce LM-BFF based on their open-source codes 7 with the same manual prompts as R ETRO P ROMPT for a fair comparison.

Original: LM-BFF (+Demo).
Revised: LM-BFF (+Demo).

Original: This approach is the above LM-BFF [13] combined with the demonstration [4].
Revised: This approach is the above LM-BFF [13] combined with the demonstration [4].

Original: Different from R ETRO P ROMPT , it uses examples of natural language as demonstrations, which isrestricted by the input length of the language model.
Revised: Different from R ETRO P ROMPT , it uses examples of natural language as demonstrations, which is restricted by the input length of the language model.

Original: Thus, LM-BFF (+demo) is not suitable formulti-class classification tasks.
Revised: Thus, LM-BFF (+demo) is not suitable for multi-class classification tasks.

Original: KnowPrompt.
Revised: KnowPrompt.

Original: KnowPrompt [6] is a SOTA prompt-tuning method for relation extraction taskswith multiple classes.
Revised: KnowPrompt [6] is a SOTA prompt-tuning method for relation extraction tasks with multiple classes.

Original: We apply our R ETRO P ROMPT over KnowPrompt on information extractiontasks for comparison, aiming to verify the broad applicability of our method.
Revised: We apply our R ETRO P ROMPT over KnowPrompt on information extraction tasks for comparison, aiming to verify the broad applicability of our method.

Original: Incorporating Knowledge into Prompt (KPT).
Revised: Incorporating Knowledge into Prompt (KPT).

Original: KPT [20] focuses on incorporating external knowledge into the verbalizer by refining the expanded label word space to improve and stabilize prompttuning, which is a solid baseline for comparison.
Revised: KPT [21] focuses on incorporating external knowledge into the verbalizer by refining the expanded label word space to improve and stabilize prompttuning, which is a solid baseline for comparison.

Original: We follow their public codes 6 to conduct experimentsin the same setting for a fair comparison.
Revised: We follow their public codes 8 to conduct experiments in the same setting for a fair comparison.

Original: LOTClass.
Revised: LOTClass.

Original: LOTClass [41] is the SOTA method in unsupervised text classification that utilizes the
Revised: LOTClass [42] is the SOTA method in unsupervised text classification that utilizes the

Original: PLM to extract the label-related words from the whole unlabeled training corpus.
Revised: PLM to extract the label-related words from the whole unlabeled training corpus.

Original: Then it leveragesthe Masked Category Prediction task to train on the unlabeled corpus with pseudo labels.5 https://github.com/princeton-nlp/LM-BFFhttps://github.com/ShengdingHu/KnowledgeablePromptTuning [Figure]
Revised: Then it leverages the Masked Category Prediction task to train on the unlabeled corpus with pseudo labels.

Original: C Experimental Settings
Revised: C Experimental Settings

Original: We report the hyper-parameters in Table 8.
Revised: We report the hyper-parameters in Table 8.

Original: Most of the hyper-parameters are the default parametersof LM-BFF 7 .
Revised: Most of the hyper-parameters are the default parameters of LM-BFF 9 .

Revised: D Tuning Retrieve Parameters

Revised: The final distribution of the label is affected by the hyperparameters of β , k and λ when conducting k NN-train and k NN-test.

Revised: Thus, we provide insight into the effect of β , k and λ on the final results.

Original: [Table]
Revised: [Figure]

Revised: β varies: Figure 4(a) shows the performance of the model when the β increases and reveals that the model performs worse as the β increases on a 16-shot CR dataset.

Revised: This finding indicates that a moderate degree of k NN guiding training is essential since k NN can help the model attend to hard examples, but excessive attendance of k NN-train also can bring the noise.

Revised: λ varies: From Figure 4(b), we observe that model achieves optimal results on a 16-shot MR dataset when λ is set to be 0.2 while attaining the best results on MR in the zero-shot setting when λ is set to be 0.7.

Revised: We think the model may require more reference when there is no data for training.

Revised: k varies: As shown in Figure 4(c), the model performance in the 16-shot MR dataset fluctuates very little.

Revised: In contrast, the result in the zero-shot MR dataset continues to improve as k increases until it converges when reaching a threshold ( k = 256 ).

Revised: It illustrates that the k -NN retrieval provides more evidence for reference in zero-setting.

Original: D Discussion of Limitation
Revised: E Discussion of Limitation

Original: Analysis of Efficiency We make the comparison between LM-BFF (man), LM-BFF (+demo) and
Revised: Analysis of Efficiency We make the comparison between LM-BFF (man), LM-BFF (+demo) and

Original: R ETRO P ROMPT in speed on the MR dataset for the 16-shot setting.
Revised: R ETRO P ROMPT in speed on the MR dataset for the 16-shot setting.

Original: We observe that the speed of
Revised: We observe that the speed of

Original: R ETRO P ROMPT and LM-BFF (+demo) are approximately 1.12 and 20 times slower than LM-BFF(man) on the 16-shot MR dataset.
Revised: R ETRO P ROMPT and LM-BFF (+demo) are approximately 1.12 and 20 times slower than LM-BFF (man) on the 16-shot MR dataset.

Original: The slow inference of LM-BFF (+demo) is due to the fact thatthey sample from the top r % instances ( r = 50 ) for each class to use as demonstrations and vastlyincrease the length of the input, thus, increasing computational complexity significantly.
Revised: The slow inference of LM-BFF (+demo) is due to the fact that they sample from the top r % instances ( r = 50 ) for each class to use as demonstrations and vastly increase the length of the input, thus, increasing computational complexity significantly.

Original: And thebottleneck of computational speed is general limitations of retrieval methods, and our method is noexception.
Revised: And the bottleneck of computational speed is general limitations of retrieval methods, and our method is no exception.

Original: We will leave the engineering optimization about retrieval speed in our future work.
Revised: We will leave the engineering optimization about retrieval speed in our future work.

Original: Analysis of memory usage Actually, our method adopt FAISS tools for retrieval.
Revised: Analysis of memory usage Actually, our method adopt FAISS tools for retrieval.

Original: FAISS is anexcellent open-source library for fast nearest neighbor retrieval in high-dimensional spaces, whichsupports searching only from RAM, which involves k-means clustering for improving memory usageefficiency.
Revised: FAISS is an excellent open-source library for fast nearest neighbor retrieval in high-dimensional spaces, which supports searching only from RAM, which involves k-means clustering for improving memory usage efficiency.

Original: Memory usage is negligible in the few-shot settings and acceptable in the full-data settings.
Revised: Memory usage is negligible in the few-shot settings and acceptable in the full-data settings.

Original: Our retrieval process is performed mainly on CPU, and we compare the utilization of CPU with andwithout retrieval in the SST-2 full setting as follows:
Revised: Our retrieval process is performed mainly on CPU, and we compare the utilization of CPU with and without retrieval in the SST-2 full setting as follows:

Original: • The CPU utilization was 46.2% with the retrieval process and 2.5% without it 
Revised: • The CPU utilization was 46.2% with the retrieval process and 2.5% without it (Our CPU is Intel(R)

Original: (Our CPU isIntel(R) Xeon(R)
Revised: Xeon(R)

Original: Silver 4210R CPU @
Revised: Silver 4210R CPU @

Original: 2.40GHz with 40 cores).• In terms of memory usage, adding retrieval requires about 2.5G more memory than not.
Revised: 2.40GHz with 40 cores). • In terms of memory usage, adding retrieval requires about 2.5G more memory than not.

Original: Oneway to reduce resource usage is to store the datastore on the disk in advance, then read andrelease it in the retrieval process.
Revised: One way to reduce resource usage is to store the datastore on the disk in advance, then read and release it in the retrieval process.

Original: E Tuning Retrieve Parameters

Original: The final distribution of the label is affected by the hyperparameters of β , k and λ when conductingk NN-train and k NN-test.

Original: Thus, we provide insight into the effect of β , k and λ on the final results.β varies: Figure 4(a) shows the performance of the model when the β increases and reveals thatthe model performs worse as the β increases on a 16-shot CR dataset.

Original: This finding indicates that amoderate degree of k NN guiding training is essential since k NN can help the model attend to hardexamples, but excessive attendance of k NN-train also can bring the noise.λ varies: From Figure 4(b), we find that model achieves optimal results on a 16-shot MR datasetwhen λ is set to be 0.2 while attaining the best results on MR in the zero-shot setting when λ is set tobe 0.7.

Original: We think the model may require more reference when there is no data for training.k varies: As shown in Figure 4(c), the model performance in the 16-shot MR dataset fluctuates verylittle.

Original: In contrast, the result in the zero-shot MR dataset continues to improve as k increases until itconverges when reaching a threshold ( k = 256 ).

Original: It illustrates that the k -NN retrieval provides moreevidence for reference in zero-setting.

Original: F More Fully-supervised Results

Original: As shown in the Figure 5, we have complemented the performance of R ETRO P ROMPT and baselinein full-data settings.

Original: [Figure]

Original: Checklist

Original: For all authors...

Original: (a) Do the main claims made in the abstract and introduction accurately reflect the paper’scontributions and scope?

Original: Yes]

Original: (b) Did you describe the limitations of your work?

Original: [Yes] See Appendix.

Original: (c) Did you discuss any potential negative societal impacts of your work?

Original: [Yes] See

Original: Appendix.

Original: (d) Have you read the ethics review guidelines and ensured that your paper conforms to

Original: them?

Original: [Yes]2.

Original: If you are including theoretical results...

Original: (a) Did you state the full set of assumptions of all theoretical results?

Original: [N/A](b)

Original: Did you include complete proofs of all theoretical results?

Original: [N/A]3.

Original: If you ran experiments...

Original: (a) Did you include the code, data, and instructions needed to reproduce the main experi-

Original: mental results (either in the supplemental material or as a URL)?

Original: [Yes] We include the

Original: source code and data in our supplemental material submission

Original: and we outline the data

Original: generation procedure, the evaluation protocol, the training regime, and everything else

Original: necessary for reproduction either in the main body of the paper or in the appendix.

Original: (b) Did you specify all the training details (e.g., data splits, hyperparameters,

Original: how they

Original: were chosen)?

Original: [Yes] See Subsection 4.2 and Appendix.

Original: (c) Did you report error bars (e

Original: g., with respect to the random seed after running experi-

Original: ments multiple times)?

Original: [Yes] We list the standard deviation for few-shot setting.

Original: (d) Did you include the total amount of compute and the type of resources used (e.g., typeof GPUs, internal cluster, or cloud provider)?

Original: [Yes] We introduce type of resources in

Original: Section 4.2.

Original: If you are using existing assets (e

Original: g., code, data, models) or curating/releasing new assets...

Original: (a) If your work uses existing assets, did you cite the creators?

Original: [Yes]

Original: (b) Did you mention the license of the assets?

Original: [No] The code and the data are proprietary.

Original: (c) Did you include any new assets either in the supplemental material or as a URL?

Original: [No](d) Did you discuss whether

Original: and how consent was obtained from people whose data you’reusing/curating?

Original: [No] The code and the data are proprietary.

Original: (e) Did you discuss whether the data you are using/curating contains personally identifiable

Original: information or offensive content?

Original: [No]

Original: If you used crowdsourcing or conducted research with human subjects...

Original: (a) Did you include the full text of instructions given to participants and screenshots, if

Original: applicable?

Original: [N/A]

Original: (b) Did you describe any potential participant risks, with links to Institutional Review

Original: Board (IRB) approvals, if applicable?

Original: [N/A]

Original: (c) Did you include the estimated hourly wage paid to participants and the total amount

Original: spent on participant compensation?

Original: [N/A]