Original: Decoupling Knowledge from Memorization: Retrieval-augmented Prompt Learning
Revised: Decoupling Knowledge from Memorization: Retrieval-augmented Prompt Learning

Original: Anonymous Author(s)
Revised: Anonymous Author(s)

Original: Abstract
Revised: Abstract

Original: Prompt learning approaches have made waves in natural language processing byinducing better few-shot performance while they still follow a parametric-basedlearning paradigm; the oblivion and rote memorization problems in learning mayencounter unstable generalization issues.
Revised: Prompt learning approaches have made waves in natural language processing byinducing better few-shot performance while they still follow a parametric-basedlearning paradigm; the oblivion and rote memorization problems in learning mayencounter unstable generalization issues.

Original: Specifically, vanilla prompt learning maystruggle to utilize atypical instances by rote during fully-supervised training or overfit shallow patterns with low-shot data.
Revised: Specifically, vanilla prompt learning maystruggle to utilize atypical instances by rote during fully-supervised training or overfit shallow patterns with low-shot data.

Original: To alleviate such limitations, we develop
Revised: To alleviate such limitations, we develop

Original: R ETRO P ROMPT with the motivation of decoupling knowledge from memorizationto help the model strike a balance between generalization and memorization.
Revised: R ETRO P ROMPT with the motivation of decoupling knowledge from memorizationto help the model strike a balance between generalization and memorization.

Original: Incontrast with vanilla prompt learning, R ETRO P
Revised: Incontrast with vanilla prompt learning, R ETRO P

Original: ROMPT constructs an open-bookknowledge-store from training instances and implements a retrieval mechanismduring the process of input, training and inference, thus equipping the model withthe ability to retrieve related contexts from the training corpus as cues for enhancement.
Revised: ROMPT constructs an open-bookknowledge-store from training instances and implements a retrieval mechanismduring the process of input, training and inference, thus equipping the model withthe ability to retrieve related contexts from the training corpus as cues for enhancement.

Original: Extensive experiments demonstrate that R ETRO P ROMPT can obtain betterperformance in both few-shot and zero-shot settings.
Revised: Extensive experiments demonstrate that R ETRO P ROMPT can obtain betterperformance in both few-shot and zero-shot settings.

Original: Besides, we further illustratethat our proposed R ETRO P ROMPT can yield better generalization abilities withnew datasets.
Revised: Besides, we further illustratethat our proposed R ETRO P ROMPT can yield better generalization abilities withnew datasets.

Original: Detailed analysis of memorization indeed reveals R ETRO P ROMPTcan reduce the reliance of language models on memorization; thus, improvinggeneralization for downstream tasks 1 .
Revised: Detailed analysis of memorization indeed reveals R ETRO P ROMPTcan reduce the reliance of language models on memorization; thus, improvinggeneralization for downstream tasks 1 .

Original:  Introduction
Revised:  Introduction

Original: Large parametric language models [42, 6, 19, 28] have achieved dramatic empirical success innatural language processing (NLP).
Revised: Large parametric language models [45, 7, 22, 31] have achieved dramatic empirical success innatural language processing (NLP).

Original: Notably, pre-trained language models (PLMs) have learned asubstantial amount of in-depth knowledge from data, and have archived tremendous promise infew-shot/zero-shot learning ability with the natural language prompts [11, 47, 52].
Revised: Notably, pre-trained language models (PLMs) have learned asubstantial amount of in-depth knowledge from data, and have archived tremendous promise infew-shot/zero-shot learning ability with the natural language prompts [13, 50, 55].

Original: However, Recentstudies [34, 36, 54] observe that prompt learning with PLMs usually generalizes unstably in anextremely low-resource setting or emerging domains.
Revised: However, Recentstudies [37, 39, 57] observe that prompt learning with PLMs usually generalizes unstably in anextremely low-resource setting or emerging domains.

Original: One potential reason is that, it is non-trivialfor parametric models to learn rare or hard patterns well with rote memorization , thus, resulting ininefficient generalizable performance.
Revised: One potential reason is that, it is non-trivialfor parametric models to learn rare or hard patterns well with rote memorization , thus, resulting ininefficient generalizable performance.

Original: Intuitively, if we regard the whole training data as a book and the test phase as the examination ,the current training-test procedure of prompt learning (based on batch data training) can be viewedas page-by-page memorization and closed-book examination [39].
Revised: Intuitively, if we regard the whole training data as a book and the test phase as the examination ,the current training-test procedure of prompt learning (based on batch data training) can be viewedas page-by-page memorization and closed-book examination [42].

Original: During training, vanilla promptlearning may struggle to memorize atypical instances in a fully-supervised setting or overfit shallowpatterns with low-shot data [56, 8].
Revised: During training, vanilla promptlearning may struggle to memorize atypical instances in a fully-supervised setting or overfit shallowpatterns with low-shot data [60, 10].

Original: Specifically, recent studies[9, 10] have proposed a long-tail theory, which states that if training data form a long-tail distribution and have small “sub-populations” withatypical instances, then PLMs indeed predict on the test data through rote memorizing these atypicalinstances rather than learning the common patterns [56, 51].
Revised: Specifically, recent studies[11, 12] have proposed a long-tail theory, which states that if training data form a long-tail distribution and have small “sub-populations”with atypical instances, then PLMs indeed predict on the test data through rote memorizing theseatypical instances rather than learning the common patterns [60, 54].

Original: The limitations of rote memorization remind us of the human learning process of “learn by analogy”and the proverb that “the palest ink is better than the best memory” .
Revised: The limitations of rote memorization remind us of the human learning process of “learn by analogy”and the proverb that “the palest ink is better than the best memory” .

Original: Note that humans can performassociative learning to recall relevant skills in deep memories for reinforcing each other, thus, owningthe extraordinary abilities to solve few-shot and zero-shot tasks.
Revised: Note that humans can performassociative learning to recall relevant skills in deep memories for reinforcing each other, thus, owningthe extraordinary abilities to solve few-shot and zero-shot tasks.

Original: Motivated by these, we endeavor toimprove the generalization ability of prompt learning with retrieval and association.
Revised: Motivated by these, we endeavor toimprove the generalization ability of prompt learning with retrieval and association.

Original: Our intuition isthat the difficulty of resolving the above limitations can be substantially alleviated if we can decouplethe knowledge from memorization by constructing an open-book knowledge-store from the trainingdata; thus, referring to related knowledge could provide a strong enhancement signal to help themodel strike a balance between generalization and memorization.
Revised: Our intuition isthat the difficulty of resolving the above limitations can be substantially alleviated if we can decouplethe knowledge from memorization by constructing an open-book knowledge-store from the trainingdata; thus, referring to related knowledge could provide a strong enhancement signal to help themodel strike a balance between generalization and memorization.

Original: [Figure]
Revised: [Figure]

Original: Specifically, we introduce a novel retrievalaugmented framework based on promptlearning ( R ETRO P ROMPT ) as shown in
Revised: Specifically, we introduce a novel retrieval- augmented framework based on promptlearning ( R ETRO P ROMPT ) as shown in

Original: Figure 1.
Revised: Figure 1.

Original: The open-book knowledge store( K , V ) , defined as the set of key: promptbased example embeddings and value: corresponding label words constructed fromthe training data, are served as additionalreferences for the model to decouple knowledge from pure memorization to some extent.
Revised: The open-book knowledge store( K , V ) , defined as the set of key: prompt- based example embeddings and value: corresponding label words constructed fromthe training data, are served as additionalreferences for the model to decouple knowledge from pure memorization to some extent.

Original: Specifically, to integrate retrievedknowledge into the input, Firstly , we design to incorporate neural demonstrations into the inputsequences as in-context augmentation, where the demonstration is retrieved from the knowledge-store.
Revised: Specifically, to integrate retrievedknowledge into the input, Firstly , we design to incorporate neural demonstrations into the inputsequences as in-context augmentation, where the demonstration is retrieved from the knowledge-store.

Original: Then , we apply a non-parametric algorithm k NN over the input query and knowledge store, andregard k NN results as an indication of easy vs. hard examples in the training set.
Revised: Then , we apply a non-parametric algorithm k NN over the input query and knowledge store, andregard k NN results as an indication of easy vs. hard examples in the training set.

Original: More specifically,we automatically force the model to focus on the hard examples identified by k NN by assigning ascaling during training.
Revised: More specifically,we automatically force the model to focus on the hard examples identified by k NN by assigning ascaling during training.

Original: Lastly , the k NN results are further employed at the output of the PLM headto participate in masked prediction during inference.
Revised: Lastly , the k NN results are further employed at the output of the PLM headto participate in masked prediction during inference.

Original: The model retrieves Top- k nearest referenceinstances as cues from ( K , V ) and makes inference by linearly interpolating the output of promptlearning with a non-parametric nearest neighbor distribution.
Revised: The model retrieves Top- k nearest referenceinstances as cues from ( K , V ) and makes inference by linearly interpolating the output of promptlearning with a non-parametric nearest neighbor distribution.

Original: The considerable performance gains on nine tasks in few-shot and zero-shot settings demonstrate thatour systemic retrieval mechanism helps the model generalize better with scarce data.
Revised: The considerable performance gains on nine tasks in few-shot and zero-shot settings demonstrate thatour systemic retrieval mechanism helps the model generalize better with scarce data.

Original: Experiments inthe fully-supervised setting with long-tail distribution illustrate that our R ETRO P ROMPT can dealwith atypical instances more robustly.
Revised: Experiments inthe fully-supervised setting with long-tail distribution illustrate that our R ETRO P ROMPT can dealwith atypical instances more robustly.

Original: We further adopt self-influence [24] as our memorizationscoring function to analyze the memorization process between fine-tuning, prompt learning andour R ETRO P ROMPT .
Revised: We further adopt self-influence [27] as our memorizationscoring function to analyze the memorization process between fine-tuning, prompt learning andour R ETRO P ROMPT .

Original: The final analysis results show that 1) the training instances with the highestmemorization scores tend to be atypical, 2) R ETRO P ROMPT generalize better than fine-tuning andconvention prompt-tuning with decoupling knowledge from memorization to alleviate the rote of
Revised: The final analysis results show that 1) the training instances with the highestmemorization scores tend to be atypical, 2) R ETRO P ROMPT generalize better than fine-tuning andconvention prompt-tuning with decoupling knowledge from memorization to alleviate the rote of

Original: PLMs.
Revised: PLMs.

Original: In a nutshell, our work may open up new avenues to improve the generalization of prompting
Revised: In a nutshell, our work may open up new avenues to improve the generalization of prompting

Original: PLMs by retrieving knowledge from memorization.
Revised: PLMs by decoupling knowledge from memorization.

Original:  Preliminaries of Prompt Learning
Revised:  Preliminaries of Prompt Learning

Original: Assuming that M , T respectively denotes the PLM and the template function for prompt tuning.
Revised: Assuming that M , T respectively denotes the PLM and the template function for prompt tuning.

Original: Formally, the text classification task takes a query sentence x
Revised: Formally, the text classification task takes a query sentence x

Original: = ( x 0 , x 1 , ..., x n ) as input, and classifyit into a class label y ∈ Y .
Revised: = ( x 0 , x 1 , ..., x n ) as input, and classifyit into a class label y ∈ Y .

Original: While prompt learning converts classification task into a masked languagemodeling problem with cloze-style objectives.
Revised: While prompt learning converts classification task into a masked languagemodeling problem with cloze-style objectives.

Original: Specifically, the template function T inserts pieces oftexts into x as ˆ x = T ( x ) , where ˆ x is the corresponding input of M with a [MASK]
Revised: Specifically, the template function T inserts pieces oftexts into x as ˆ x = T ( x ) , where ˆ x is the corresponding input of M with a [MASK]

Original: token in it.
Revised: token in it.

Original: Forexample, assuming we need to classify the sentence x =“The movie makes absolutely no sense.”
Revised: Forexample, assuming we need to classify the sentence x =“The movie makes absolutely no sense.”

Original: into
Revised: into

Original: [Figure]
Revised: [Figure]

Original: label N EGATIVE (labeled as 0) or P OSITIVE (labeled as 1), we wrap it into
Revised: label N EGATIVE (labeled as 0) or P OSITIVE (labeled as 1), we wrap it into

Original: [Equation]
Revised: [Equation]

Original: [CLS]
Revised: CLS

Original: [Equation]
Revised: [Equation]

Original: It was [MASK][SEP] (1)
Revised: It was [MASK][SEP] (1)

Original: The verbalizer f : Y (cid:55)→ V is defined as a mapping from the label space Y to a few words in thevocabulary, which form the label word set V .
Revised: The verbalizer f : Y (cid:55)→ V is defined as a mapping from the label space Y to a few words in thevocabulary, which form the label word set V .

Original: The base component of M produces the sequencerepresentation over ˆ x , and we choose the hidden vector at the [MASK] position as the contextualrepresentation
Revised: The base component of M produces the sequencerepresentation over ˆ x , and we choose the hidden vector at the [MASK] position as the contextualrepresentation

Original: h ˆ x ∈ R d , where d is the dimension of hidden states.
Revised: h ˆ x ∈ R d , where d is the dimension of hidden states.

Original: Then the MLM head of M canoperate on h ˆ x to calculate the probability of each word v in the vocabulary being filled in [MASK]token P M ( [MASK] = v | ˆ x ) .
Revised: Then the MLM head of M canoperate on h ˆ x to calculate the probability of each word v in the vocabulary being filled in [MASK]token P M ( [MASK] = v | ˆ x ) .

Original: We let V y to represent the subset of V that is connected with a specificlabel y , ∪ y ∈Y V y = V .
Revised: We let V y to represent the subset of V that is connected with a specificlabel y , ∪ y ∈Y V y = V .

Original: Then the probability distribution over the label y is calculated as:
Revised: Then the probability distribution over the label y is calculated as:

Original: [Equation] [Equation]
Revised: [Equation] [Equation]

Original:  where g is a function transforming the probability of label words into the probability of the classes.
Revised: ) where g is a function transforming the probability of label words into the probability of the classes.

Original:  R ETRO P ROMPT : Retrieval-augmented Prompt Learning
Revised:  R ETRO P ROMPT : Retrieval-augmented Prompt Learning

Original: We introduce a simple and general retrieval-augmented framework for prompt learning, named
Revised: We introduce a simple and general retrieval-augmented framework for prompt learning, named

Original: R ETRO P ROMPT , whose basis is the dense retriever (§3.1) with an open-book knowledge-store todecouple knowledge from memorization.
Revised: R ETRO P ROMPT , whose basis is the dense retriever (§3.1) with an open-book knowledge-store todecouple knowledge from memorization.

Original: As shown in Figure 2, R ETRO P ROMPT consists of threecomponents: retrieval of neural demonstration for enhancing input (§3.2), the k NN guided training(§3.3) and the k NN-based probability for cloze-style prediction (§3.4).
Revised: As shown in Figure 2, R ETRO P ROMPT consists of threecomponents: retrieval of neural demonstration for enhancing input (§3.2), the k NN guided training(§3.3) and the k NN-based probability for cloze-style prediction (§3.4).

Original: 3.1 Dense Retriever
Revised: 3.1 Dense Retriever

Original: Open-book Knowledge-store
Revised: Open-book Knowledge-store

Original: The first step of our proposed framework is to build a knowledgestore for retrieval that can decouple from memorization and captures the semantics of the instance fromthe training set C .
Revised: The first step of our proposed framework is to build a knowledgestore for retrieval that can decouple from memorization and captures the semantics of the instance fromthe training set C .

Original: Specifically, we utilize the encoder to embed prompt-based instance representationover the C to construct the knowledge-store.
Revised: Specifically, we utilize the encoder to embed prompt-based instance representationover the C to construct the knowledge-store.

Original: Given the i -th example ( c i , y i ) in the training data C ,we compute the key-value pair ( h ˆ c
Revised: Given the i -th example ( c i , y i ) in the training data C ,we compute the key-value pair ( h ˆ c

Original: i , v i ) , in which ˆ c i
Revised: i , v i ) , in which ˆ c i

Original: = T ( c i ) ,
Revised: = T ( c i ) ,

Original: h ˆ c i ∈ R
Revised: h ˆ c i ∈ R

Original: d is the embedding of the[MASK] token in the last layer of the underlying PLM, and v i =
Revised: d is the embedding of the[MASK] token in the last layer of the underlying PLM, and v i =

Original: f ( y i ) denotes the label word of thei -th example.
Revised: f ( y i ) denotes the label word ofthe i -th example.

Revised: Compared with kNN-LM [26] that constructing with sliding generative corpusand tokens, our knowledge-store is more suitable for prompt learning.

Original: We store all pairs ( h ˆ c , v ) in a key-value datastore ( K , V )
Revised: We store all pairs ( h ˆ c , v ) in akey-value datastore ( K , V )

Original: where
Revised: where

Original: h ˆ c serves as key and vas value as follows:
Revised: h ˆ c serves as key and v as value as follows:

Original: [Equation]
Revised: [Equation]

Original: The knowledge-store is flexible to add, edit or delete any instances and can be asynchronously updatedduring the training procedure.
Revised: The knowledge-store is flexible to add, edit or delete any instances and can be asynchronously updatedduring the training procedure.

Original: Note that our knowledge-store is constructed from few-shot trainsetsin the corresponding few-shot settings rather than the whole available training data.
Revised: Note that our knowledge-store is constructed from few-shot trainsetsin the corresponding few-shot settings rather than the whole available training data.

Original: Efficient Searching Considering that the size of the training data C can be enormous, we mustensure an efficient retrieval process.
Revised: Efficient Searching Considering that the size of the training data C can be enormous, we mustensure an efficient retrieval process.

Original: As shown in the above creation of open-book knowledge-store,we can build the matrix D ∈ R |C|× d as the index of training examples.
Revised: As shown in the above creation of open-book knowledge-store,we can build the matrix D ∈ R |C|× d as the index of training examples.

Original: Given a query set Q , wefirst encode each query example with template mapping function T ( · ) to get a set of prompt-basedquery vectors
Revised: Given a query set Q , wefirst encode each query example with template mapping function T ( · ) to get a set of prompt-basedquery vectors

Original: h ˆ q for retrieval augmentation on the fly.
Revised: h ˆ q for retrieval augmentation on the fly.

Original: Then, we utilize query vectors to search forthe closest examples over the index D via maximum inner product search (MIPS).
Revised: Then, we utilize query vectors to search forthe closest examples over the index D via maximum inner product search (MIPS).

Original: For the retrievalprocess, we choose FAISS [18] to query the open-book knowledge-store efficiently.
Revised: For the retrievalprocess, we choose FAISS [21] to query the open-book knowledge-store efficiently.

Original: FAISS is anexcellent open-source library for fast nearest neighbor retrieval in high-dimensional spaces.
Revised: FAISS is anexcellent open-source library for fast nearest neighbor retrieval in high-dimensional spaces.

Original: Asynchronous Refresh of the Knowledge-store Since the neural demonstration may lead to thevariable contextual representation of instance as the parameters of the PLM are continually updated,we thus propose to “refresh” the index of retrieval by asynchronously re-embedding and re-indexingall embeddings in an open-book knowledge-store every j training epochs 2 .
Revised: Asynchronous Refresh of the Knowledge-store Since the neural demonstration may lead to thevariable contextual representation of instance as the parameters of the PLM are continually updated,we thus propose to “refresh” the index of retrieval by asynchronously re-embedding and re-indexingall embeddings in an open-book knowledge-store every j training epochs 2 .

Original: In § 4.6, we empiricallydemonstrate that this procedure results in performance improvement.
Revised: In § 4.6, we empiricallydemonstrate that this procedure results in performance improvement.

Original: 3.2 Retrieval of Neural Demonstration
Revised: 3.2 Retrieval of Neural Demonstration

Original: To enhance the PLMs with the ability to learn by analogy through the knowledge-store, we furthercombine R ETRO P ROMPT with neural demonstrations, an orthogonal technique enhancing languagemodels, to improve the generalization ability of our model.
Revised: To enhance the PLMs with the ability to learn by analogy through the knowledge-store, we furtherpropose neural demonstrations that can be concatenated with input instance at the embedding layer toimprove the generalization ability of our R ETRO P ROMPT .

Original: For the t -th query instance q t , we firstutilize prompt-based representation h ˆ q t to query the cached representations of open-book knowledgestore.
Revised: For the t -th query instance q t , we firstutilize prompt-based representation h ˆ q t to query the cached representations of open-book knowledgestore.

Original: Then we retrieve m nearest neighbors {{ c (1)1 , ..., c (1) m } , ..., { c ( L ) 1 , ..., c ( L ) m }} of q t for eachclass, where the superscript L denotes the total number of the classes and the c ( l ) i is retrieved as thei -th nearest neighbor in the l -th class.
Revised: Then we retrieve m nearest neighbors {{ c (1)1 , ..., c (1) m } , ..., { c ( L ) 1 , ..., c ( L ) m }} of q t for eachclass, where the superscript L denotes the total number of the classes and the c ( l ) i is retrieved as thei -th nearest neighbor in the l -th class.

Original: After the model retrieves the Top- m candidates for each class,their corresponding representation h ( l ) ˆ c i and label word v ( l ) from knowledge-store will be incorporatedinto the encoder to act as a demonstration learning.
Revised: After the model retrieves the Top- m candidates for each class,their corresponding representation h ( l ) ˆ c i and label word v ( l ) from knowledge-store will be incorporatedinto the encoder to act as a demonstration learning.

Original: Since the h ( l ) ˆ c i is already vector, we intuitivelyaggregate the m neighbor vectors for each class according to their similarity and incorporate thedemonstration into the input representation of ˆ x after the word embedding layer of the M as follows:
Revised: Since the h ( l ) ˆ c i is already vector, we intuitivelyaggregate the m neighbor vectors for each class according to their similarity and incorporate thedemonstration into the input representation of ˆ x after the word embedding layer of the M as follows:

Original: [Equation]
Revised: [Equation]

Original: where e ( · ) represents the word embedding layer of M , ⊕ denotes the concatenation of input sequences, α ( l ) i is the softmax score for the i -th retrieval belonging to l -th class label to denote theirrelevance with ˆ q , and I is the sequence features for inputting the next layer of PLM.
Revised: where e ( · ) represents the word embedding layer of M , ⊕ denotes the concatenation of input sequences, α ( l ) i is the softmax score for the i -th retrieval belonging to l -th class label to denote theirrelevance with ˆ q , and I is the sequence features for inputting the next layer of PLM.

Original: As shown inthe above equation, we encode demonstration representation with the weighted sum of the retrievalrepresentation.
Revised: As shown inthe above equation, we encode demonstration representation with the weighted sum of the retrievalrepresentation.

Original: Thus, retrieval scores are directly used in the final representation, making the framework differentiable.
Revised: Thus, retrieval scores are directly used in the final representation, making the framework differentiable.

Original: To this end, we denote this style of demonstration as neural demonstration ,significantly different from prior work of discrete demonstration [11].
Revised: To this end, we denote this style of demonstration as neural demonstration ,significantly different from prior work of discrete demonstration [13].

Original: Neural vs. Discrete Demonstration Compared with prior discrete demonstrations described in[11, 32, 46, 25], retrieving weighted neural demonstrations from the knowledge-store to augmentprompt learning has advantages in the following three major aspects: (1) neural demonstrationscould be more tolerant of the model’s maximum input length than discrete demonstrations, while thediscrete demonstration is usually not suitable for multi-class classification tasks due to the limitationof input length, such as relation extraction, etc.
Revised: Neural vs. Discrete Demonstration Compared with prior discrete demonstrations described in[13, 35, 49, 28], retrieving weighted neural demonstrations from the knowledge-store to augmentprompt learning has advantages in the following three major aspects: (1) neural demonstrationscould be more tolerant of the model’s maximum input length than discrete demonstrations, while thediscrete demonstration is usually not suitable for multi-class classification tasks due to the limitationof input length, such as relation extraction, etc.

Original: (2) the model needs to deal with large retrieval tokensfor discrete demonstration, making it time-consuming and computationally intensive to performcross-attention operations due to the quadratic attention complexity.
Revised: (2) the model needs to deal with large retrieval tokensfor discrete demonstration, making it time-consuming and computationally intensive to performcross-attention operations due to the quadratic attention complexity.

Original: In contrast, dealing with muchshorter instance representations as neural demonstrations unleashes the potential of cross-attentionand accelerates the inference.
Revised: In contrast, dealing with muchshorter instance representations as neural demonstrations unleashes the potential of cross-attentionand accelerates the inference.

Original: (3) when sampling examples based on the similarity between instances,our cloze-style contextual representation is more informative and consistent than the contextual representation from [CLS] of Sentence-BERT [44] (adopted in LM-BFF).
Revised: (3) when sampling examples based on the similarity between instances,our cloze-style contextual representation is more informative and consistent than the contextualrepresentation from [CLS] of Sentence-BERT [47] (adopted in LM-BFF).

Original: 3.3 Retrieve k NN for Guiding Training
Revised: 3.3 Retrieve k NN for Guiding Training

Original: Eager learners, such as PLMs, are trained to provide a global approximating function that maps frominput to output space.
Revised: Eager learners, such as PLMs, are trained to provide a global approximating function that maps frominput to output space.

Original: Lazy learners such as k -nearest neighbor classifiers, on the contrary, focuson approximating the neighborhoods around test examples [2].
Revised: Lazy learners such as k -nearest neighbor classifiers, on the contrary, focuson approximating the neighborhoods around test examples [2].

Original: Since k NN can easily predict foreach encountered query instance based on pre-trained representation without an extra classifier, it isintuitively to leverage the k NN’s classification results as the prior external knowledge to guide the
Revised: Since k NN can easily predict foreach encountered query instance based on pre-trained representation without an extra classifier, it isintuitively to leverage the k NN’s classification results as the prior external knowledge to guide the

Original: PLMs’ parameters attending to hard examples (hard samples usually refer to atypical samples) duringthe training process (also referred as k NN-train for the abbreviation).
Revised: PLMs’ parameters attending to hard examples (hard samples usually refer to atypical samples) duringthe training process (also referred as k NN-train for the abbreviation).

Original: Particularly, our intuition isto differentiate between easy and hard examples according to the prediction of k NN.
Revised: Particularly, our intuition isto differentiate between easy and hard examples according to the prediction of k NN.

Original: Given the t -thquery instance q t , we leverage the h q t querying the open-book knowledge-store ( K , V ) to retrievethe k -nearest neighbors N of q t according to a similarity function d ( · , · ) , where d ( · , · ) typically adoptthe inner product similarity.
Revised: Given the t -thquery instance q t , we leverage the h q t querying the open-book knowledge-store ( K , V ) to retrievethe k -nearest neighbors N of q t according to a similarity function d ( · , · ) , where d ( · , · ) typically adoptthe inner product similarity.

Original: Then, we compute a distribution over neighbors based on a softmax oftheir similarities and aggregate probability mass for each label word across all its occurrences in theretrieved targets:
Revised: Then, we compute a distribution over neighbors based on a softmax oftheir similarities and aggregate probability mass for each label word across all its occurrences in theretrieved targets:

Original: [Equation]
Revised: [Equation]

Original: Given the probability p k NN of the query instance q t being predicted as the gold class , we proposeto retrieve the k NN for guiding the training process of prompt learning.
Revised: Given the probability p k NN of the query instance q t being predicted as the gold class (also as theprobability value of the gold class in the P k NN ), we propose to retrieve the k NN for guiding thetraining process of prompt learning.

Original: The k NN guider reweightsthe cross-entropy loss L CE by adjusting the relative loss for the correctly-classified or misclassifiedinstances identified by k NN, respectively.
Revised: The k NN guider reweights the cross-entropy loss L CE byadjusting the relative loss for the correctly-classified or misclassified instances identified by k NN,respectively.

Original: Specifically, we apply the negative log-likelihood as themodulating factor F ( p k NN ) .
Revised: Specifically, we apply the negative log-likelihood as the modulating factor F ( p k NN ) .

Original: The final loss L is defined as:
Revised: The final loss L is defined as:

Original: [Equation]
Revised: [Equation]

Original: where β denotes a scalar to determine the proportion of each loss term.
Revised: where β denotes a scalar to determine the proportion of each loss term.

Original: Note that p k NN is computedusing the leave-one-out distribution on the training set due to the fact that each example in the trainingset cannot retrieve itself.
Revised: Note that p k NN is computedusing the leave-one-out distribution on the training set due to the fact that each example in the trainingset cannot retrieve itself.

Original: The motivation of modulating factorhere is similar to Focal-loss [31], whilewe focus on exploit  the application of k NN in tuning PLMs.
Revised: The motivation of modulating factor is inspired by Focal-loss [34], whilewe focus on exploit leveraging k-NN’s results for calibrating the training of LMs.

Original: 3.4 k NN based probability for Cloze-style Prediction
Revised: 3.4 k NN based probability for Cloze-style Prediction

Original: Apart from the neural demonstration on the input side and k NN guided training process (also referredas k NN-test for the abbreviation), we further present k NN based probability for Cloze-style predictionon the inference process, providing the PLM ability to retrieve nearest neighbors for decisions ratherthan making predictions only based on memorized parameters.
Revised: Apart from the neural demonstration on the input side and k NN guided training process (also referredas k NN-test for the abbreviation), we further present k NN based probability for Cloze-style predictionon the inference process, providing the PLM ability to retrieve nearest neighbors for decisions ratherthan making predictions only based on memorized parameters.

Original: Given the non-parametric k nearestneighbor distribution P k NN of the query instance q t being predicted as y ,  the P ( y | q t ) is reformulatedby interpolating the P k NN with the already-trained base PLM’s MLM prediction P M using parameterλ to produce the final probability of the label:
Revised: Given the non-parametric k nearestneighbor distribution P k NN of the query instance q t being predicted as y , we follow [14, 26, 17]to reformulate the P ( y | q t ) by interpolating the P k NN with the already-trained base PLM’s MLMprediction P M using parameter λ to produce the final probability of the label:

Original: [Equation]
Revised: [Equation]

Original: Different from k NN-LM [14] that uses tokens to augment the language modeling directly, weexplicitly take advantage of prompt-based instance representation for classification tasks, whichismore deeply rooted in prompt learning. In this way, we can unlock the model prediction process asan open-book examination.
Revised: Different from k NN-LM [26, 17] that mainly retrieve tokens to augment the language modeling, wefocus on leveraging prompt-based kNN’s distribution for reference at test time, which can unlock themodel prediction process as an open-book examination for prompt learning.

Original:  Experiments
Revised:  Experiments

Original: 4.1 Datasets and Baselines
Revised: 4.1 Datasets and Baselines

Original: Datasets We evaluate R ETRO P ROMPT on several types of natural language understanding tasks,including single sentence classification tasks (SST-2 [50], MR [40], and CR [16]) and sentence pairclassification tasks (MNLI [53], QNLI [43], and QQP 3 ).
Revised: Datasets We evaluate R ETRO P ROMPT on several types of natural language understanding tasks,including single sentence classification tasks (SST-2 [53], MR [43], and CR [19]) and sentence pairclassification tasks (MNLI [56], QNLI [46], and QQP 3 ).

Original: To further evaluate the effectiveness of theproposed approach with multi-class classification, we also conduct experiments on the informationextraction tasks, including FewNERD [7], SemEval 2010{+
+} Task 8 (SemEval) [15], and TACRED [55].
Revised: To further evaluate the effectiveness of theproposed approach with multi-class classification, we also conduct experiments on the informationextraction tasks, including FewNERD [9], SemEval {+
+} Task 8 (SemEval) [18], and TACRED [59].

Original: [Table]
Revised: [Table]

Original: Baselines We compare with LM-BFF [11] for single sentence and sentence pair classification tasksand adopt SOTA prompt learning model
Revised: Baselines We compare with LM-BFF [13] for single sentence and sentence pair classification tasksand adopt SOTA prompt learning model

Original: KnowPrompt [5] as the baseline for information extractiontasks.
Revised: KnowPrompt [6] as the baseline for information extractiontasks.

Original: Note that the discrete demonstration method cannot be applied to multi-class classificationtasks due to the input length limitations; thus, we leave out the experimental table about the resultsof KnPr (D-demo).
Revised: Note that the discrete demonstration method cannot be applied to multi-class classificationtasks due to the input length limitations; thus, we leave out the experimental table about the resultsof KnPr (D-demo).

Original: We also compare our R ETRO P ROMPT with the knowledge-enhanced promptlearning method KPT [17] since KPT leverages the external knowledge base for enhancing promptlearning while we focus on utilizing internal trainsets as a knowledge-store.
Revised: We also compare our R ETRO P ROMPT with the knowledge-enhanced promptlearning method KPT [20] since KPT leverages the external knowledge base for enhancing promptlearning while we focus on utilizing internal trainsets as a knowledge-store.

Original: 4.2 Evaluation protocols and details
Revised: 4.2 Evaluation protocols and details

Original: The experiments are implemented on 1 NVIDIA V100 and utilize Pytorch [41] as the base library.
Revised: The experiments are implemented on 1 NVIDIA V100 and utilize Pytorch [44] as the base library.

Original: We adopt RoBERTa large [35] as the PLM and employ AdamW as the optimizer for all experiments.
Revised: We adopt RoBERTa large [38] as the PLM and employ AdamW as the optimizer for all experiments.

Original: To mitigate the influence of diverse templates, we conduct baselines and R ETRO P ROMPT with thesame templates for each dataset.
Revised: To mitigate the influence of diverse templates, we conduct baselines and R ETRO P ROMPT with thesame templates for each dataset.

Original: The specific templates we use for each dataset are in Appendix.
Revised: The specific templates we use for each dataset are in Appendix.

Original: Asfor few-shot and zero-shot experiments, we leverage different settings, respectively.
Revised: Asfor few-shot and zero-shot experiments, we leverage different settings, respectively.

Original: Few-shot Setting.
Revised: Few-shot Setting.

Original: We follow the few-shot setting of LM-BFF [11] to conduct 4-shot and 16-shotexperiments and evaluate the average performance with a fixed set of seeds, S seed , across five differentsampled D train for each task.
Revised: We follow the few-shot setting of LM-BFF [13] to conduct 4-shot and 16-shotexperiments and evaluate the average performance with a fixed set of seeds, S seed , across five differentsampled D train for each task.

Original: Note that our knowledge-store is constructed with the few-shot trainingset in this setting.
Revised: Note that our knowledge-store is constructed with the few-shot trainingset in this setting.

Original: Zero-shot Setting.
Revised: Zero-shot Setting 4 .

Original: We leverage vanilla RoBERTa large for all baselines (except LOTClass [38])to directly inference on the test set.
Revised: We leverage vanilla RoBERTa large for all baselines (except LOTClass [41])to directly inference on the test set.

Original: To take advantage of retrieval mechanism, R ETRO P ROMPTfollows LOTClass [38] to utilize unlabeled trainsets for retrieval.
Revised: To take advantage of retrieval mechanism, R ETRO P ROMPTfollows LOTClass [41] to utilize unlabeled trainsets for retrieval.

Original: Specifically, we take the vanilla
Revised: Specifically, we take the vanilla

Original: RoBERTa large to tag the pseudo labels on unlabeled trainset and create the open-book knowledge-storewith the unlabeled trainsets and pseudo labels.
Revised: RoBERTa large to tag the pseudo labels on unlabeled trainset and create the open-book knowledge-storewith the unlabeled trainsets and pseudo labels.

Original: Lastly, R ETRO P ROMPT make predictions on the testset based on the constructed datastore without tuning any of the model parameters .
Revised: Lastly, R ETRO P ROMPT make predictions on the testset based on the constructed datastore without tuning any of the model parameters .

Original: 4.3 Experimental Results
Revised: 4.3 Experimental Results

Original: Few-shot Results.
Revised: Few-shot Results.

Original: As shown in Table 1, we find R ETRO P ROMPT consistently outperforms baselinemethod LM-BFF and KnowPrompt, both in 4-shot and 16-shot experiments.
Revised: As shown in Table 1, we find R ETRO P ROMPT consistently outperforms baselinemethod LM-BFF and KnowPrompt, both in 4-shot and 16-shot experiments.

Original: Especially for information extraction tasks with multiple classes, discrete demonstrations cannot be applied to the inputdue to the limited input sequence length, while our neural demonstration can also work and achieves improvement on these multi-class datasets.
Revised: Especially for information extraction tasks with multiple classes, discrete demonstrations cannot be applied to the input due to the limited input sequence length, while our neural demonstration can also work and achievesimprovement on these multi-class datasets.

Original: Moreover, R ETRO P ROMPT obtain better performancecompared with KPT.
Revised: Moreover, R ETRO P ROMPT obtain better performancecompared with KPT.

Original: Compared with KPT with external knowledge, we only focus on referencing theinternal few-shot trainsets without visiting the external knowledge base.
Revised: Compared with KPT with external knowledge, we only focus on referencing theinternal few-shot trainsets without visiting the external knowledge base.

Original: Besides, we observe that
Revised: Besides, we observe that

Original: R ETRO P ROMPT has a relatively lower standard deviation than the baselines.
Revised: R ETRO P ROMPT has a relatively lower standard deviation than the baselines.

Original: The reason may lie thatthe retrieval mechanism can compensate for instabilities in parametric predictions.
Revised: The reason may lie thatthe retrieval mechanism can compensate for instabilities in parametric predictions.

Original: [Figure]
Revised: [Figure]

Original: Zero-shot Results.
Revised: Zero-shot Results.

Original: From Table 1, we also observe that R ETROP ROMPT achieves improvements in the zero-shot setting.
Revised: From Table 1, we also observe that R ETRO - [Figure] P ROMPT achieves improvements in the zero-shot setting.

Original: Anothernotable point is that R ETRO P ROMPT performs even better than KPTin the zero-shot setting, revealing that exploring own data to decoupleknowledge from memorization has more potential than leveragingexternal knowledge.
Revised: Anothernotable point is that R ETRO P ROMPT performs even better than KPTin the zero-shot setting, revealing that exploring own data to decoupleknowledge from memorization has more potential than leveragingexternal knowledge.

Original: Moreover, we achieve superior performance to
Revised: Moreover, we achieve superior performance to

Original: LOTClass even though we utilize the vanilla RoBERTa large withoutany training.fully-supervised Results.
Revised: LOTClass even though we utilize the vanilla RoBERTa large withoutany training. Fully-supervised Results.

Original: The experiments in  fully-supervised settings with long-tail distribution illustrate that R ETRO P ROMPT achievesimprovement compared with baselines.
Revised: As shown in Figure 3, the experimentsin fully-supervised settings with long-tail distribution illustrate that R ETRO P ROMPT achieves improvement compared with baselines.

Original: This indicates that our retrievalmechanism extends the LM’s ability to learn hard examples in the fully-supervised datasets.
Revised: Thisindicates that our retrieval mechanism extends the LM’s ability to learn hard examples in the fullysupervised datasets.

Original: 4.4 Model Generalization to New Domains
Revised: 4.4 Model Generalization to New Domains

Original: [Figure]
Revised: [Figure]

Original: [Table] 
Revised: [Table] [Figure] [Table]

Original: [Figure]
Revised: [Figure]

Original: [Table]
Revised: [Table]

Original: The scarce data may bring the overfitting problem for thelots of memory parameters of PLMs, even though promptlearning.
Revised: The scarce data may bring the overfitting problem for the lotsof memory parameters of PLMs, even though prompt learning.

Original: Thus, we conduct cross-domain experiments tovalidate the generalization of our R ETRO P ROMPT .
Revised: Thus, we conduct cross-domain experiments to validate thegeneralization of our R ETRO P ROMPT .

Original: Specifically, we utilize the model trained on the source datasetsand directly test on the other target datasets.
Revised: Specifically, we utilizethe model trained on the source datasets and directly test on theother target datasets.

Original: From Table 2,we can find that our method consistently outperformsbaselines.
Revised: From Table 2, we can find that our methodconsistently outperforms baselines.

Original: This finding illustrates that R ETRO P ROMPTachieves great model generalization to new domains.
Revised: This finding illustrates that R ETRO P ROMPT achieves great model generalization to newdomains.

Original: 4.5 Analysis of Memorization
Revised: 4.5 Analysis of Memorization

Original: It is necessary and interesting to further explore the memorization mechanism to help us better understand the utilityof retrieval for memorization in NLP.
Revised: It is necessary and interesting to further explore the memorization mechanism to help us betterunderstand the utility of retrieval for memorization in NLP.

Original: Definition of Memorization Measurement.
Revised: Definition of Memorization Measurement.

Original: Inspired by the idea of [9] in the computer visionarea, we define memorization measures as to how the classification varies when a training instance zis deleted from the trainset.
Revised: Inspired by the idea of [11] in the computer visionarea, we define memorization measures as to how the classification varies when a training instance zis deleted from the trainset.

Original: We follow [24, 56] to define and derive the memorization score for atraining instance z as follows:
Revised: We follow [27, 60] to define and derive the memorization score for atraining instance z as follows:

Original: [Equation]
Revised: [Equation]

Original: where ˆ θ ξ, − z denotes the parameters of the model trained with the instance z down-weighted by ξ , ˆ θ isthe parameters of the model trained with all instances and H ˆ θ = 1 n (cid:80) ni
Revised: where ˆ θ ξ, − z denotes the parameters of the model trained with the instance z down-weighted by ξ , ˆ θ isthe parameters of the model trained with all instances and H ˆ θ = 1 n (cid:80) ni

Original: =1 ∇ 2 θ L ( z i , ˆ θ ) .
Revised: =1 ∇ 2 θ L ( z i , ˆ θ ) .

Original: Thus S delate ( z )is the amount of change of P ( y | x ; θ ) when the instance z is down-weighted by a small amount ξ .
Revised: Thus S delate ( z )is the amount of change of P ( y | x ; θ ) when the instance z is down-weighted by a small amount ξ .

Original: Top-memorized Instances: Typical or Atypical?
Revised: Top-memorized Instances: Typical or Atypical?

Original: Since the SST-2 dataset provides the annotationsof phrase-level sentiment polarity labels, we adopt SST-2 to analyze the memorization by judging theatypical of an instance by checking the percentage of positive phrases.
Revised: Since the SST-2 dataset provides the annotationsof phrase-level sentiment polarity labels, we adopt SST-2 to analyze the memorization by judging theatypical of an instance by checking the percentage of positive phrases.

Original: We collect such statistics from
Revised: We collect such statistics from

Original: SST-2 and find that a typical positive instance has a relatively high percentage of positive phrases,and a typical negative instance should have a relatively low percentage of positive phrases.
Revised: SST-2 and find that a typical positive instance has a relatively high percentage of positive phrases,and a typical negative instance should have a relatively low percentage of positive phrases.

Original: Basedon the above observation, we apply the memorization score defined in Eq.
Revised: Basedon the above observation, we apply the memorization score defined in Eq.

Original: 8 to select Top-10% and
Revised: 8 to select Top-10% and

Original: Bottom-10% memorized instances from the trainset and collect the average percentage of positivephrases in these instances.
Revised: Bottom-10% memorized instances from the trainset and collect the average percentage of positivephrases in these instances.

Original: [Table]
Revised: [Table]

Original: As shown in Table 3,we can conclude following findings: (1) The PLM tends to give atypical samplesdeeper memory attention.
Revised: As shown in Table 3,we can conclude following findings: (1) The PLM tends to give atypical samplesdeeper memory attention.

Original: Specifically, no matter
Revised: Specifically, no matter

Original: LM-BFF or our method, the top-10% memorizednegative instances have a higher percentage of positive phrases than the average percentage of positive phrases of all negative instances.
Revised: LM-BFF or our method, the top-10% memorizednegative instances have a higher percentage of positive phrases than the average percentage of positive phrases of all negative instances.

Original: 2) LM-BFFhas lower memorization scores on hard samples thanfine-tuning.
Revised: 2) LM-BFFhas lower memorization scores on hard samples thanfine-tuning.

Original: We think it owns to prompt learningcan help PLMs recall what they learned from pretraining without strengthening memory for downstream data.
Revised: We think it owns to prompt learningcan help PLMs recall what they learned from pretraining without strengthening memory for downstream data.

Original: 3) R ETRO P ROMPT further has loweraverage memorization scores than fine-tuning and
Revised: 3) R ETRO P ROMPT further has loweraverage memorization scores than fine-tuning and

Original: LM-BFF, which illustrates that our method is less memory dependent.
Revised: LM-BFF, which illustrates that our method is less memory dependent.

Original: This result may be attributedto decoupling knowledge from memorization through retrieval to alleviating the rote of PLMs.
Revised: This result may be attributedto decoupling knowledge from memorization through retrieval to alleviating the rote of PLMs.

Original: [Table]
Revised: [Table]

Original: Case Analysis.
Revised: Case Analysis.

Original: As shown in Table 6, we manually list the top-ranked and bottom-ranked traininginstances of SST-2 according to our model.
Revised: As shown in Table 6, we manually list the top-ranked and bottom-ranked traininginstances of SST-2 according to our model.

Original: It reveals that the top-ranked memorized instances seemto show universal opinions indirectly.
Revised: It reveals that the top-ranked memorized instances seemto show universal opinions indirectly.

Original: Thus, we inspect them as atypical/hard for sentiment classification.
Revised: Thus, we inspect them as atypical/hard for sentiment classification.

Original: While those instances with 0 memorizationscores are straightforward to show their opinion forsentiment classification, representing the typical instance.
Revised: While those instances with 0 memorizationscores are straightforward to show their opinion forsentiment classification, representing the typical instance.

Original: Note that F ( p kNN ) is defined to representthe difficulty of the sample discriminated by k NN distribution.
Revised: Note that F ( p kNN ) is defined to representthe difficulty of the sample discriminated by k NN distribution.

Original: And the Table 6 also shows that F ( p kNN )indeed reflect atypicality of examples, which validate the effectiveness of the k NN guided training.
Revised: And the Table 6 also shows that F ( p kNN )indeed reflect atypicality of examples, which validate the effectiveness of the k NN guided training.

Original: 4.6 Ablation Study
Revised: 4.6 Ablation Study

Original: Component Ablation.
Revised: Component Ablation

Original: As shown in Table 4, the performance of component ablation experimentswith four variants has a clear drop, which proves the effectiveness of our retrieval component.
Revised: As shown in Table 4, the performance of component ablation experimentswith four variants has a clear drop, which proves the effectiveness of our retrieval component.

Original: Wealso find that neural demonstration and k NN-train have more improvement in the few-shot settingthan k NN-test.
Revised: Wealso find that neural demonstration and k NN-train have more improvement in the few-shot settingthan k NN-test.

Original: Note that k NN-test is similar to k NN-LM [23, 14] and the results reveals that simplyincorporate k NN in the test process of prompt learning has little influence in a few-shot setting.
Revised: Note that k NN-test is similar to k NN-LM [26, 17] and the results reveals that simplyincorporate k NN in the test process of prompt learning has little influence in a few-shot setting.

Original: [Table]
Revised: [Table]

Original: Key Representation and k NN Acquisition.
Revised: Key Representation and k NN Acquisition.

Original: We studythe effect of using different representations of the key in theknowledge-store.
Revised: We studythe effect of using different representations of the key in theknowledge-store.

Original: We experiment with two types of representations: (1) prompt-based representation, which is thedefault setting, and (2) [CLS] based representation of current
Revised: We experiment with two types of representations: (1) prompt-based representation, which is thedefault setting, and (2) [CLS] based representation of current

Original: LM.
Revised: LM.

Original: We also experiment with two types of calculation ofk NN distribution: (1) representation based similarity score(refer as rep-similar), which is the default setting, and (2)
Revised: We also experiment with two types of calculation ofk NN distribution: (1) representation based similarity score(refer as rep-similar), which is the default setting, and (2)

Original: BM25 based score , which calculates the correlation scorebetween the query and each key examples with BM25 [45]algorithm.
Revised: BM25 based score , which calculates the correlation scorebetween the query and each key examples with BM25 [48]algorithm.

Original: Results in Table 5 show that using prompt-based representations for key and representationbased similarity scores for k NN leads to the best performance.
Revised: Results in Table 5 show that using prompt-based representations for key and representationbased similarity scores for k NN leads to the best performance.

Original: It suggests that prompt learn betterrepresentations for context similarity and the representation similarity based k NN distribution isbetter than BM25 based scores.
Revised: It suggests that prompt learn betterrepresentations for context similarity and the representation similarity based k NN distribution isbetter than BM25 based scores.

Original: [Table]
Revised: [Table]

Original:  Related Work
Revised:  Related Work

Original: [Table]
Revised: [Table]

Original: Retrieval-enhanced PLMs.
Revised: Retrieval-enhanced PLMs.

Original: Our pipeline is partly inspired by discrete demonstration methodssuch as [11, 32, 46, 25, 26] that retrieves few training examples in a natural language prompt, whilewe propose neural demonstration for enhancing the input to alleviate the limitations of input length.
Revised: Our pipeline is partly inspired by discrete demonstration methodssuch as [13, 35, 28, 29] that retrieves few training examples in a natural language prompt, whilewe propose neural demonstration for enhancing the input to alleviate the limitations of input length.

Original: Another line researches of retrieval augmentation [12, 20, 29] retrieve useful information from aexternal knowledge corpus (e.g., Wikipedia) for a particular task (e.g., an open-domain question).
Revised: Another line researches of retrieval augmentation [15, 23, 32, 49, 3] retrieve useful information froma external knowledge corpus (e.g., Wikipedia) for a particular task (e.g., an open-domain question).

Original: Unlike these works, we focus on retrieving examples from the internal training data.
Revised: Unlike these works, we focus on retrieving examples from the internal training data.

Original: Besides, semiparametric methods [23, 14, 22, 21, 1, 39] have risen to leverage k -nearest neighbor classifier thatmakes the prediction based on representation similarities, to enhance pre-trained language models.
Revised: Besides, semiparametric methods [26, 17, 25, 24, 1, 42] have risen to leverage k -nearest neighbor classifier thatmakes the prediction based on representation similarities, to enhance pre-trained language models.

Original: However, unlike these models using nearest neighbors only for augmenting the process of prediction,we aim to develop a comprehensive retrieval mechanism for input, training and test process.
Revised: However, unlike these models using nearest neighbors only for augmenting the process of prediction,we aim to develop a comprehensive retrieval mechanism for input, training and test process.

Original: Prompt learning for PLMs.
Revised: Prompt learning for PLMs.

Original: With the birth of GPT-3 [3], prompt learning [33] has recentlyarisen to fill the gap between masked LM objective of PLMs and downstream fine-tuning objective.
Revised: With the birth of GPT-3 [4], prompt learning [36] has recentlyarisen to fill the gap between masked LM objective of PLMs and downstream fine-tuning objective.

Original: Prompt learning has achieves very impressive performance on various tasks [48, 49, 4, 37, 13, 5],especially under the setting of few-shot learning.
Revised: Prompt learning has achieves very impressive performance on various tasks [51, 52, 5, 40, 16, 6],especially under the setting of few-shot learning.

Original: Moreover, continuous prompts have also beenproposed [30, 27, 34] to reduce prompt engineering, which directly appends a series of learnablecontinuous embeddings as prompts into the input sequence.
Revised: Moreover, continuous prompts have also beenproposed [33, 30, 37] to reduce prompt engineering, which directly appends a series of learnablecontinuous embeddings as prompts into the input sequence.

Original: Our work is orthogonal to previousprompt learning approaches, which aim to optimize prompts, while we focus on the systematic studyof retrieving related examples from training data to enhance prompt learning.
Revised: Our work is orthogonal to previousprompt learning approaches, which aim to optimize prompts, while we focus on the systematic studyof retrieving related examples from training data to enhance prompt learning.

Original:  Conclusion and Future Work
Revised:  Conclusion and Future Work

Original: We propose R ETRO P ROMPT that decouples knowledge from memorization by introducing retrievalaugmentation to further improve the generalization ability of prompt learning on the input side andthe whole process of model training and prediction.
Revised: We propose R ETRO P ROMPT that decouples knowledge from memorization by introducing retrievalaugmentation to further improve the generalization ability of prompt learning on the input side andthe whole process of model training and prediction.

Original: R ETRO P ROMPT , is a straightforward yet effectiveretrieval method that combines both neural demonstrations, k NN guider for training and prediction.
Revised: R ETRO P ROMPT , is a straightforward yet effectiveretrieval method that combines both neural demonstrations, k NN guider for training and prediction.

Original: Our extensive results show that it outperforms other demonstration-enhanced prompt methods andknowledge-enhanced prompt methods in few-shot, zero-shot and fully-supervised settings.
Revised: Our extensive results show that it outperforms other demonstration-enhanced prompt methods andknowledge-enhanced prompt methods in few-shot, zero-shot and fully-supervised settings.

Original: Analyzingthe essence of memorization validates the effectiveness of decoupling knowledge from memorization.
Revised: Analyzingthe essence of memorization validates the effectiveness of decoupling knowledge from memorization.

Original: Interesting future directions include: 1) apply to other tasks, such as QA and NLG, 2) explore thenoise data mining for unsupervised learning, 3) further improve the retrieve efficiency for largedatasets, etc. 
Revised: Interesting future directions include: 1) apply to other tasks, such as QA and NLG, 2) explore thenoise data mining for unsupervised learning, 3) further improve the retrieve efficiency for largedatasets, etc. A Datasets and Templates

Revised: In this section, we introduce the information of datasets as shown in Table 7 and list the templates weuse in experiments as follows.

Revised: SST-2, MR, CR .

Revised: For the single sentence classification tasks, we follow the LM-BFF [13] to designthe templates:

Original: Checklist
Revised: T ( x ) = [CLS] x It was [MASK] . We set Verbalizer: (great/terrible) → (positive/negative) for SST-2 MR and CR.

Revised: For the Yahoo dataset,we assign the Verbalizer following the original labels.

Revised: MNLI, QNLI, QQP .

Revised: For the sentence pair classification tasks, we follow LM-BFF [13] to set

Revised: Verbalizer: (Yes/Maybe/No) →

Revised: (entailment/neutral/contradiction), and define the following templates:

Revised: [Table]

Revised: FewNERD, SemEval, TACRED .

Revised: FewNERD, SemEval and TACRED are datasets for informationextraction, which require inserting the entity into the template.

Revised: Therefore, we follow [8] and [6] todefine the template and verbalizers.

Revised: B Compared Baselines

Revised: In this subsection, we introduce the baselines we compare with and re-produce them under the samesettings with their open-source codes.

Revised: LM-BFF uses several other tricks, such as prompt ensemble, while KPT utilizes tremendous externalknowledge.

Revised: We do not use any of these tricks and external knowledge since we get the most out of thedata to decouple part of knowledge from parametric memorization.

Revised: Our R ETRO P ROMPT mechanismis orthogonal to other methodological improvements of prompt-tuning (such as continuous prompt in

Revised: P-tuning [37] and DART [58] ) and can be combined with other prompt-tuning methods in futurework.

Revised: Fine-tuning (FT).

Revised: The traditional fine-tuning method regard the hidden embedding of [CLS]token of the PLM as the representation of the sentence and then feeds them into a classification layerto make predictions.

Revised: LM-BFF.

Revised: LM-BFF [13] is a typical prompt-tuning method wrapping an input sentence into ahandcrafted template.

Revised: Here we re-produce LM-BFF based on their open-source codes 5 with thesame manual prompts as R ETRO P ROMPT for a fair comparison.

Revised: LM-BFF (+Demo).

Revised: This approach is the above LM-BFF [13] combined with the demonstration [4].

Revised: Different from R ETRO P ROMPT , it uses examples of natural language as demonstrations, which isrestricted by the input length of the language model.

Revised: Thus, LM-BFF (+demo) is not suitable formulti-class classification tasks.

Revised: KnowPrompt.

Revised: KnowPrompt [6] is a SOTA prompt-tuning method for relation extraction taskswith multiple classes.

Revised: We apply our R ETRO P ROMPT over KnowPrompt on information extractiontasks for comparison, aiming to verify the broad applicability of our method.

Revised: Incorporating Knowledge into Prompt (KPT).

Revised: KPT [20] focuses on incorporating external knowledge into the verbalizer by refining the expanded label word space to improve and stabilize prompttuning, which is a solid baseline for comparison.

Revised: We follow their public codes 6 to conduct experimentsin the same setting for a fair comparison.

Revised: LOTClass.

Revised: LOTClass [41] is the SOTA method in unsupervised text classification that utilizes the

Revised: PLM to extract the label-related words from the whole unlabeled training corpus.

Revised: Then it leveragesthe Masked Category Prediction task to train on the unlabeled corpus with pseudo labels.5 https://github.com/princeton-nlp/LM-BFFhttps://github.com/ShengdingHu/KnowledgeablePromptTuning

Revised: [Figure]

Revised: C Experimental Settings

Revised: We report the hyper-parameters in Table 8.

Revised: Most of the hyper-parameters are the default parametersof LM-BFF 7 .

Revised: [Table]

Revised: D Discussion of Limitation

Revised: Analysis of Efficiency We make the comparison between LM-BFF (man), LM-BFF (+demo) and

Revised: R ETRO P ROMPT in speed on the MR dataset for the 16-shot setting.

Revised: We observe that the speed of

Revised: R ETRO P ROMPT and LM-BFF (+demo) are approximately 1.12 and 20 times slower than LM-BFF(man) on the 16-shot MR dataset.

Revised: The slow inference of LM-BFF (+demo) is due to the fact thatthey sample from the top r % instances ( r = 50 ) for each class to use as demonstrations and vastlyincrease the length of the input, thus, increasing computational complexity significantly.

Revised: And thebottleneck of computational speed is general limitations of retrieval methods, and our method is noexception.

Revised: We will leave the engineering optimization about retrieval speed in our future work.

Revised: Analysis of memory usage Actually, our method adopt FAISS tools for retrieval.

Revised: FAISS is anexcellent open-source library for fast nearest neighbor retrieval in high-dimensional spaces, whichsupports searching only from RAM, which involves k-means clustering for improving memory usageefficiency.

Revised: Memory usage is negligible in the few-shot settings and acceptable in the full-data settings.

Revised: Our retrieval process is performed mainly on CPU, and we compare the utilization of CPU with andwithout retrieval in the SST-2 full setting as follows:

Revised: • The CPU utilization was 46.2% with the retrieval process and 2.5% without it

Revised: (Our CPU isIntel(R) Xeon(R)

Revised: Silver 4210R CPU @

Revised: 2.40GHz with 40 cores).• In terms of memory usage, adding retrieval requires about 2.5G more memory than not.

Revised: Oneway to reduce resource usage is to store the datastore on the disk in advance, then read andrelease it in the retrieval process.

Revised: E Tuning Retrieve Parameters

Revised: The final distribution of the label is affected by the hyperparameters of β , k and λ when conductingk NN-train and k NN-test.

Original: For all authors...
Revised: Thus, we provide insight into the effect of β , k and λ on the final results.β varies: Figure 4(a) shows the performance of the model when the β increases and reveals thatthe model performs worse as the β increases on a 16-shot CR dataset. This finding indicates that amoderate degree of k NN guiding training is essential since k NN can help the model attend to hardexamples, but excessive attendance of k NN-train also can bring the noise.λ varies: From Figure 4(b), we find that model achieves optimal results on a 16-shot MR datasetwhen λ is set to be 0.2 while attaining the best results on MR in the zero-shot setting when λ is set tobe 0.7. We think the model may require more reference when there is no data for training.k varies: As shown in Figure 4(c), the model performance in the 16-shot MR dataset fluctuates verylittle. In contrast, the result in the zero-shot MR dataset continues to improve as k increases until itconverges when reaching a threshold ( k = 256 ).

Original: (a) Do
Revised: It illustrates that the k -NN retrieval provides moreevidence for reference in zero-setting. F More Fully-supervised Results As shown in the Figure 5, we have complemented the performance of R ETRO P ROMPT and baselinein full-data settings. [Figure]

Original:  the main claims made in the abstract and introduction accurately reflect the paper’scontributions and scope?
Revised: For all authors... (a) Do the main claims made in the abstract and introduction accurately reflect the paper’scontributions and scope?

Original: [Yes]
Revised: Yes]

Original: (b) Did you describe the limitations of your work
Revised: (b) Did you describe the limitations of your work?

Original: [Yes] See Appendix
Revised: [Yes] See Appendix.

Original: (c) Did you discuss any potential negative societal impacts of your work?
Revised: (c) Did you discuss any potential negative societal impacts of your work?

Original: [Yes] See
Revised: [Yes] See

Original: Appendix.
Revised: Appendix.

Original: (d) Have you read the ethics review guidelines and ensured that your paper conforms to
Revised: (d) Have you read the ethics review guidelines and ensured that your paper conforms to

Original: them?
Revised: them?

Original: [Yes]
Revised: [Yes]2.

Original: If you are including theoretical results...
Revised: If you are including theoretical results...

Original: (a) Did you state the full set of assumptions of all theoretical results?
Revised: (a) Did you state the full set of assumptions of all theoretical results?

Original: [N/A]
Revised: [N/A](b)

Original: (b) Did you include complete proofs of all theoretical results?
Revised: Did you include complete proofs of all theoretical results?

Original: [N/A]
Revised: [N/A]3.

Original: If you ran experiments...
Revised: If you ran experiments...

Original: (a) Did you include the code, data, and instructions needed to reproduce the main experi-
Revised: (a) Did you include the code, data, and instructions needed to reproduce the main experi-

Original: mental results (either in the supplemental material or as a URL)?
Revised: mental results (either in the supplemental material or as a URL)?

Original: [Yes] We include the
Revised: [Yes] We include the

Original: source code and data in our supplemental 
Revised: source code and data in our supplemental material submission

Original: material

Original: submission, and we outline the data
Revised: and we outline the data

Original: generation procedure, the evaluation protocol the training regime, and everything 
Revised: generation procedure, the evaluation protocol, the training regime, and everything else

Original: else

Original: necessary for reproduction either in the main body of the paper or in the appendix
Revised: necessary for reproduction either in the main body of the paper or in the appendix.

Original: (b) Did you specify all the training details (e.g., data splits, hyperparameters,
Revised: (b) Did you specify all the training details (e.g., data splits, hyperparameters,

Original: how they
Revised: how they

Original: were chosen)?
Revised: were chosen)?

Original: [Yes] See Subsection 4.2 and Appendix.
Revised: [Yes] See Subsection 4.2 and Appendix.

Original: (c) Did you report error bars (e
Revised: (c) Did you report error bars (e

Original: g., with respect to the random seed after running experi-
Revised: g., with respect to the random seed after running experi-

Original: ments multiple times)?
Revised: ments multiple times)?

Original: [Yes] We list the standard deviation for few-shot setting.
Revised: [Yes] We list the standard deviation for few-shot setting.

Original: (d) Did you include the total amount of compute and the type of resources used (e.g., type of GPUs, internal cluster, or cloud provider)?
Revised: (d) Did you include the total amount of compute and the type of resources used (e.g., typeof GPUs, internal cluster, or cloud provider)?

Original: [Yes] We introduce type of resources in
Revised: [Yes] We introduce type of resources in

Original: Section 4.2.
Revised: Section 4.2.

Original: If you are using existing assets (e.g., code, data, models) or curating/releasing new assets...
Revised: If you are using existing assets (e g., code, data, models) or curating/releasing new assets...

Original: (a) If your work uses existing assets, did you cite the creators?
Revised: (a) If your work uses existing assets, did you cite the creators?

Original: [Yes]
Revised: [Yes]

Original: (b) Did you mention the license of the assets?
Revised: (b) Did you mention the license of the assets?

Original: [No] The code and the data are proprietary.
Revised: [No] The code and the data are proprietary.

Original: (c) Did you include any new assets either in the supplemental material or as a URL?
Revised: (c) Did you include any new assets either in the supplemental material or as a URL?

Original: [No] (d) Did you discuss whether and how consent was obtained from people whose data you’re using/curating?
Revised: [No](d) Did you discuss whether and how consent was obtained from people whose data you’reusing/curating?

Original: [No] The code and the data are proprietary.
Revised: [No] The code and the data are proprietary.

Original: (e) Did you discuss whether the data you are using/curating contains personally identifiable
Revised: (e) Did you discuss whether the data you are using/curating contains personally identifiable

Original: information or offensive content?
Revised: information or offensive content?

Original: [No]
Revised: [No]

Original: If you used crowdsourcing or conducted research with human subjects...
Revised: If you used crowdsourcing or conducted research with human subjects...

Original: [Table] (a) Did you include the full text of instructions given to participants and screenshots, if[Table]
Revised: (a) Did you include the full text of instructions given to participants and screenshots, if

Original: applicable?
Revised: applicable?

Original: [N/A]
Revised: [N/A]

Original: [Table] (b) Did you describe any potential participant risks, with links to Institutional Review[Table]
Revised: (b) Did you describe any potential participant risks, with links to Institutional Review

Original: Board ) approvals, if applicable?
Revised: Board (IRB) approvals, if applicable?

Original: [N/A]
Revised: [N/A]

Original: [Table] ) Did you include the estimated hourly wage paid to participants and the total amount[Table]
Revised: (c) Did you include the estimated hourly wage paid to participants and the total amount

Original: spent on participant compensation
Revised: spent on participant compensation?

Original: [N/A]
Revised: [N/A]

Original: [Table]