Original: Decoupling Knowledge from Memorization: Retrieval-augmented Prompt Learning
Revised: Decoupling Knowledge from Memorization: Retrieval-augmented Prompt Learning

Original: Abstract
Revised: Abstract

Original: Prompt learning approaches have made waves in natural language processing by inducing better few-shot performance while they still follow a parametric-based learning paradigm; the oblivion and rote memorization problems in learning may encounter unstable generalization issues.
Revised: Prompt learning approaches have made waves in natural language processing by inducing better few-shot performance while they still follow a parametric-based learning paradigm; the oblivion and rote memorization problems in learning may encounter unstable generalization issues.

Original: Specifically, vanilla prompt learning may struggle to utilize atypical instances by rote during fully-supervised training or overfit shallow patterns with low-shot data.
Revised: Specifically, vanilla prompt learning may struggle to utilize atypical instances by rote during fully-supervised training or overfit shallow patterns with low-shot data.

Original: To alleviate such limitations, we develop
Revised: To alleviate such limitations, we develop

Original: R ETRO P ROMPT with the motivation of decoupling knowledge from memorization to help the model strike a balance between generalization and memorization.
Revised: R ETRO P ROMPT with the motivation of decoupling knowledge from memorization to help the model strike a balance between generalization and memorization.

Original: In contrast with vanilla prompt learning, R ETRO P ROMPT constructs an open-book knowledge-store from training instances and implements a retrieval mechanism during the process of input, training and inference, thus equipping the model with the ability to retrieve related contexts from the training corpus as cues for enhancement.
Revised: In contrast with vanilla prompt learning, R ETRO P ROMPT constructs an open-book knowledge-store from training instances and implements a retrieval mechanism during the process of input, training and inference, thus equipping the model with the ability to retrieve related contexts from the training corpus as cues for enhancement.

Original: Extensive experiments demonstrate that R ETRO P ROMPT can obtain better performance in both few-shot and zero-shot settings.
Revised: Extensive experiments demonstrate that R ETRO P ROMPT can obtain better performance in both few-shot and zero-shot settings.

Original: Besides, we further illustrate that our proposed R ETRO P ROMPT can yield better generalization abilities with new datasets.
Revised: Besides, we further illustrate that our proposed R ETRO P ROMPT can yield better generalization abilities with new datasets.

Original: Detailed analysis of memorization indeed reveals R ETRO P ROMPT can reduce the reliance of language models on memorization; thus, improving generalization for downstream tasks 3 .
Revised: Detailed analysis of memorization indeed reveals R ETRO P ROMPT can reduce the reliance of language models on memorization; thus, improving generalization for downstream tasks 3 .

Original:  Introduction
Revised:  Introduction

Original: Large parametric language models [46, 7, 23, 32] have achieved dramatic empirical success in natural language processing (NLP).
Revised: Large parametric language models [45, 7, 22, 31] have achieved dramatic empirical success in natural language processing (NLP).

Original: Notably, pre-trained language models (PLMs) have learned a substantial amount of in-depth knowledge from data, and have archived tremendous promise in few-shot/zero-shot learning ability with the natural language prompts [13, 51, 58].
Revised: Notably, pre-trained language models (PLMs) have learned a substantial amount of in-depth knowledge from data, and have archived tremendous promise in few-shot/zero-shot learning ability with the natural language prompts [12, 50, 57].

Original: However, Recent studies [38, 40, 60] observe that prompt learning with PLMs usually generalizes unstably in an extremely low-resource setting or emerging domains.
Revised: However, Recent studies [37, 39, 59] observe that prompt learning with PLMs usually generalizes unstably in an extremely low-resource setting or emerging domains.

Original: One potential reason is that, it is non-trivial for parametric models to learn rare or hard patterns well with rote memorization , thus, resulting in inefficient generalizable performance.
Revised: One potential reason is that, it is non-trivial for parametric models to learn rare or hard patterns well with rote memorization , thus, resulting in inefficient generalizable performance.

Original: Intuitively, if we regard the whole training set as a book and the test phase as the examination , the current training-test procedure of prompt learning (based on batch data training) can be viewed as page-by-page memorization and closed-book examination [43].
Revised: Intuitively, if we regard the whole training set as a book and the test phase as the examination , the current training-test procedure of prompt learning (based on batch data training) can be viewed as page-by-page memorization and closed-book examination [42].

Original: During training, vanilla prompt learning may struggle to memorize atypical instances in a fully-supervised setting or overfit shallow patterns with low-shot data [64, 10].
Revised: During training, vanilla prompt learning may struggle to memorize atypical instances in a fully-supervised setting or overfit shallow patterns with low-shot data [62, 9].

Original: Specifically, recent studies[11, 12] have proposed a longtail theory, which notes that when the training set has a long-tail distribution and contain small “sub-populations” with atypical instances, then PLMs indeed predict on the test data through rote memorizing these atypical instances rather than learning the common patterns [64, 57].
Revised: Specifically, recent studies[10, 11] have proposed a longtail theory, which notes that when the training set has a long-tail distribution and contain small “sub-populations” with atypical instances, then PLMs indeed predict on the test data through rote memorizing these atypical instances rather than learning the common patterns [62, 56].

Original: The limitations of rote memorization remind us of the human learning process of “learn by analogy” and the proverb that “the palest ink is better than the best memory” .
Revised: The limitations of rote memorization remind us of the human learning process of “learn by analogy” and the proverb that “the palest ink is better than the best memory” .

Original: Note that humans can perform associative learning to recall relevant skills in deep memories for reinforcing each other, thus, owning the extraordinary abilities to solve few-shot and zero-shot tasks.
Revised: Note that humans can perform associative learning to recall relevant skills in deep memories for reinforcing each other, thus, owning the extraordinary abilities to solve few-shot and zero-shot tasks.

Original: Motivated by these, we endeavor to improve the generalization ability of prompt learning with retrieval and association.
Revised: Motivated by these, we endeavor to improve the generalization ability of prompt learning with retrieval and association.

Original: Our intuition is that the difficulty of resolving the above limitations can be substantially alleviated if we can decouple the knowledge from memorization by constructing an open-book knowledge-store from the training data; thus, referring to related knowledge could provide a strong enhancement signal to help the model strike a balance between generalization and memorization.
Revised: Our intuition is that the difficulty of resolving the above limitations can be substantially alleviated if we can decouple the knowledge from memorization by constructing an open-book knowledge-store from the training data; thus, referring to related knowledge could provide a strong enhancement signal to help the model strike a balance between generalization and memorization.

Original: [Figure]
Revised: [Figure]

Original: Specifically, we introduce a novel retrievalaugmented framework based on prompt learning ( R ETRO P ROMPT ) as shown in
Revised: Specifically, we introduce a novel retrievalaugmented framework based on prompt learning ( R ETRO P ROMPT ) as shown in

Original: Figure 1.
Revised: Figure 1.

Original: The open-book knowledge store ( K , V ) , defined as the set of key: promptbased example embeddings and value: corresponding label words constructed from the training data, are served as additional references for the model to decouple knowledge from pure memorization to some extent.
Revised: The open-book knowledge store ( K , V ) , defined as the set of key: promptbased example embeddings and value: corresponding label words constructed from the training data, are served as additional references for the model to decouple knowledge from pure memorization to some extent.

Original: Specifically, to integrate retrieved knowledge into the input, Firstly , we design to incorporate neural demonstrations into the input sequences as in-context augmentation, where the demonstration is retrieved from the knowledge-store.
Revised: Specifically, to integrate retrieved knowledge into the input, Firstly , we design to incorporate neural demonstrations into the input sequences as in-context augmentation, where the demonstration is retrieved from the knowledge-store.

Original: Then , we apply a non-parametric algorithm k NN over the input query and knowledge store, and regard k NN results as an indication of easy vs. hard instances.
Revised: Then , we apply a non-parametric algorithm k NN over the input query and knowledge store, and regard k NN results as an indication of easy vs. hard instances.

Original: Moreover, we automatically force the model to emphasize the hard instances identified by k NN by assigning a scaling during training.
Revised: Moreover, we automatically force the model to emphasize the hard instances identified by k NN by assigning a scaling during training.

Original: Lastly , the k NN results are further employed at the output of the PLM head to participate in masked prediction.
Revised: Lastly , the k NN results are further employed at the output of the PLM head to participate in masked prediction.

Original: The model conducts inference through linearly interpolating the non-parametric nearest neighbor distribution with the output of prompt learning, which regards the Top- k nearest reference instances as cues from ( K , V ) .
Revised: The model conducts inference through linearly interpolating the non-parametric nearest neighbor distribution with the output of prompt learning, which regards the Top- k nearest reference instances as cues from ( K , V ) .

Original: The considerable performance gains on nine tasks in few-shot and zero-shot settings demonstrate that our systemic retrieval mechanism helps the model generalize better with scarce data.
Revised: The considerable performance gains on nine tasks in few-shot and zero-shot settings demonstrate that our systemic retrieval mechanism helps the model generalize better with scarce data.

Original: Experiments in the fully-supervised setting with long-tail distribution illustrate that our R ETRO P ROMPT can deal with atypical instances more robustly.
Revised: Experiments in the fully-supervised setting with long-tail distribution illustrate that our R ETRO P ROMPT can deal with atypical instances more robustly.

Original: We further adopt self-influence [28] as our memorization scoring function to analyze the memorization process between fine-tuning, prompt learning and our
Revised: We further adopt self-influence [27] as our memorization scoring function to analyze the memorization process between fine-tuning, prompt learning and our

Original: R ETRO P ROMPT .
Revised: R ETRO P ROMPT .

Original: The final analysis results show that 1) the training samples having the highest memorization scores are mostly atypical, 2) R ETRO P ROMPT generalize better than fine-tuning and convention prompt-tuning with decoupling knowledge from memorization to alleviate the rote of
Revised: The final analysis results show that 1) the training samples having the highest memorization scores are mostly atypical, 2) R ETRO P ROMPT generalize better than fine-tuning and convention prompt-tuning with decoupling knowledge from memorization to alleviate the rote of

Original: PLMs.
Revised: PLMs.

Original: In a nutshell, our work may open up new avenues to improve the generalization of prompting
Revised: In a nutshell, our work may open up new avenues to improve the generalization of prompting

Original: PLMs by decoupling knowledge from memorization.
Revised: PLMs by decoupling knowledge from memorization.

Original:  Preliminaries of Prompt Learning
Revised:  Preliminaries of Prompt Learning

Original: Assuming that M , T respectively denotes the PLM and the template function for prompt tuning.
Revised: Assuming that M , T respectively denotes the PLM and the template function for prompt tuning.

Original: Formally, the text classification task takes a query sentence x
Revised: Formally, the text classification task takes a query sentence x

Original: = ( x 0 , x 1 , ..., x n ) as input.
Revised: = ( x 0 , x 1 , ..., x n ) as input.

Original: Then, classify it into the label y ∈ Y .
Revised: Then, classify it into the label y ∈ Y .

Original: While prompt learning converts the task into a MLM problem with cloze-style objectives.
Revised: While prompt learning converts the task into a MLM problem with cloze-style objectives.

Original: Specifically, the template function T inserts text pieces into x as ˆ x = T ( x ) , where ˆ x refers to the input of M with a [MASK] token.
Revised: Specifically, the template function T inserts text pieces into x as ˆ x = T ( x ) , where ˆ x refers to the input of M with a [MASK] token.

Original: For instances, when we have to classify the text x =“The movie makes absolutely no sense.”
Revised: For instances, when we have to classify the text x =“The movie makes absolutely no sense.”

Original: into label N EGATIVE (labeled as 0) or P OSITIVE (labeled as 1), we wrap it into ˆ x = [CLS] x It was [MASK][SEP] (1)
Revised: into label N EGATIVE (labeled as 0) or P OSITIVE (labeled as 1), we wrap it into ˆ x = [CLS] x It was [MASK][SEP] (1)

Original: [Figure]
Revised: [Figure]

Original: The verbalizer f : Y (cid:55)→ V is defined as a mapping from the label space Y to those words in the vocabulary, which constructs the label word set V .
Revised: The verbalizer f : Y (cid:55)→ V is defined as a mapping from the label space Y to those words in the vocabulary, which constructs the label word set V .

Original: The base component of M produces the sequence representation over ˆ x , and we choose the hidden vector at the [MASK] position as the contextual representation h ˆ x ∈ R d , where d is the dimension of hidden states.
Revised: The base component of M produces the sequence representation over ˆ x , and we choose the hidden vector at the [MASK] position as the contextual representation h ˆ x ∈ R d , where d is the dimension of hidden states.

Original: Then the MLM head of M can operate on h ˆ x to calculate each word v ’s probability in the vocabulary being filled in [MASK]
Revised: Then the MLM head of M can operate on h ˆ x to calculate each word v ’s probability in the vocabulary being filled in [MASK]

Original: P M ( [MASK] =
Revised: P M ( [MASK] =

Original: v | ˆ x ) .
Revised: v | ˆ x ) .

Original: We let V y to represent the subset of V which is connected with a unique label y , ∪ y ∈Y V y = V .
Revised: We let V y to represent the subset of V which is connected with a unique label y , ∪ y ∈Y V y = V .

Original: Finally, the probability distribution over the label y is calculated as:
Revised: Finally, the probability distribution over the label y is calculated as:

Original: [Equation]
Revised: [Equation]

Original: where g refers to the function converting the probability of label words to the probability of classes.
Revised: where g refers to the function converting the probability of label words to the probability of classes.

Original:  R ETRO P ROMPT : Retrieval-augmented Prompt Learning
Revised:  R ETRO P ROMPT : Retrieval-augmented Prompt Learning

Original: We introduce a simple and general retrieval-augmented framework for prompt learning, named
Revised: We introduce a simple and general retrieval-augmented framework for prompt learning, named

Original: R ETRO P ROMPT , whose basis is the dense retriever (§3.1) with an open-book knowledge-store to decouple knowledge from memorization.
Revised: R ETRO P ROMPT , whose basis is the dense retriever (§3.1) with an open-book knowledge-store to decouple knowledge from memorization.

Original: As shown in Figure 2, R ETRO P ROMPT consists of three components: retrieval of neural demonstration for enhancing input (§3.2), the k NN guided training (§3.3) and the k NN-based probability for cloze-style prediction (§3.4).
Revised: As shown in Figure 2, R ETRO P ROMPT consists of three components: retrieval of neural demonstration for enhancing input (§3.2), the k NN guided training (§3.3) and the k NN-based probability for cloze-style prediction (§3.4).

Original: 3.1 Dense Retriever
Revised: 3.1 Dense Retriever

Original: Open-book Knowledge-store
Revised: Open-book Knowledge-store

Original: The first step of our proposed framework is to build a knowledgestore for retrieval that can decouple from memorization and captures the semantics of the instance from the training set C .
Revised: The first step of our proposed framework is to build a knowledgestore for retrieval that can decouple from memorization and captures the semantics of the instance from the training set C .

Original: Specifically, we leverage the encoder to embed instance representation over the C to construct the knowledge-store.
Revised: Specifically, we leverage the encoder to embed instance representation over the C to construct the knowledge-store.

Original: Given the i -th example ( c i , y i ) in the training data
Revised: Given the i -th example ( c i , y i ) in the training data

Original: C , we obtain the key-value pair ( h ˆ c i , v i ) , in which ˆ c i
Revised: C , we obtain the key-value pair ( h ˆ c i , v i ) , in which ˆ c i

Original: =
Revised: =

Original: T ( c i ) , h
Revised: T ( c i ) , h

Original: ˆ c i ∈ R
Revised: ˆ c i ∈ R

Original: d is the embedding of the [MASK] token in the last layer of the PLM, and v i =
Revised: d is the embedding of the [MASK] token in the last layer of the PLM, and v i =

Original: f ( y i ) denotes the label word of the i -th example.
Revised: f ( y i ) denotes the label word of the i -th example.

Original: Compared with kNN-LM [27] that constructing with sliding generative corpus and tokens, our knowledge-store is more suitable for prompt learning.
Revised: Compared with kNN-LM [26] that constructing with sliding generative corpus and tokens, our knowledge-store is more suitable for prompt learning.

Original: We store all pairs ( h ˆ c , v ) in a key-value datastore ( K , V )
Revised: We store all pairs ( h ˆ c , v ) in a key-value datastore ( K , V )

Original: where
Revised: where

Original: h ˆ c serves as key and v as value as follows:
Revised: h ˆ c serves as key and v as value as follows:

Original: [Equation]
Revised: [Equation]

Original: The knowledge-store maye be flexible to edit, add or delete any instances and can be asynchronously updated during the training procedure.
Revised: The knowledge-store maye be flexible to edit, add or delete any instances and can be asynchronously updated during the training procedure.

Original: Note that our knowledge-store is constructed from few-shot trainsets in the corresponding few-shot settings rather than the whole available training data.
Revised: Note that our knowledge-store is constructed from few-shot trainsets in the corresponding few-shot settings rather than the whole available training data.

Original: Efficient Searching Considering that the size of the training data C can be enormous, we must ensure an efficient retrieval process.
Revised: Efficient Searching Considering that the size of the training data C can be enormous, we must ensure an efficient retrieval process.

Original: As shown in the above creation of open-book knowledge-store, we can build the matrix D ∈ R |C|× d as the index of training examples.
Revised: As shown in the above creation of open-book knowledge-store, we can build the matrix D ∈ R |C|× d as the index of training examples.

Original: Given a query set Q , we first encode each query example with template mapping function T ( · ) to get a set of prompt-based query vectors h ˆ q for retrieval augmentation on the fly.
Revised: Given a query set Q , we first encode each query example with template mapping function T ( · ) to get a set of prompt-based query vectors h ˆ q for retrieval augmentation on the fly.

Original: Then, we utilize query vectors to search for the closest examples over the index D via maximum inner product search (MIPS).
Revised: Then, we utilize query vectors to search for the closest examples over the index D via maximum inner product search (MIPS).

Original: For the retrieval process, we choose FAISS [22] to query the open-book knowledge-store efficiently.
Revised: For the retrieval process, we choose FAISS [21] to query the open-book knowledge-store efficiently.

Original: FAISS is an excellent open-sourced toolkit for fast nearest neighbor retrieval.
Revised: FAISS is an excellent open-sourced toolkit for fast nearest neighbor retrieval.

Original: Asynchronous Refresh of the Knowledge-store Since the neural demonstration may lead to the variable contextual representation of instance as the parameters of the PLM are continually updated, we thus propose to “refresh” the index of retrieval by asynchronously re-embedding and re-indexing all embeddings in an open-book knowledge-store every j training epochs 4 .
Revised: Asynchronous Refresh of the Knowledge-store Since the neural demonstration may lead to the variable contextual representation of instance as the parameters of the PLM are continually updated, we thus propose to “refresh” the index of retrieval by asynchronously re-embedding and re-indexing all embeddings in an open-book knowledge-store every j training epochs 4 .

Original: In § 4.6, we empirically demonstrate that this procedure results in performance improvement.
Revised: In § 4.6, we empirically demonstrate that this procedure results in performance improvement.

Original: 3.2 Retrieval of Neural Demonstration
Revised: 3.2 Retrieval of Neural Demonstration

Original: To enhance the PLMs with the ability to learn by analogy through the knowledge-store, we further propose neural demonstrations that can be concatenated with input instance at the embedding layer to improve the generalization ability of our R ETRO P ROMPT .
Revised: To enhance the PLMs with the ability to learn by analogy through the knowledge-store, we further propose neural demonstrations that can be concatenated with input instance at the embedding layer to improve the generalization ability of our R ETRO P ROMPT .

Original: For the t -th query instance q t , we first utilize prompt-based representation h ˆ q t to query the cached representations of open-book knowledgestore.
Revised: For the t -th query instance q t , we first utilize prompt-based representation h ˆ q t to query the cached representations of open-book knowledgestore.

Original: Then we retrieve m nearest neighbors {{ c (1)1 , ..., c (1) m } , ..., { c ( L ) 1 , ..., c ( L ) m }} of q t for each class, where the superscript L denotes the total number of the classes and the c ( l ) i is retrieved as the i -th nearest neighbor in the l -th class.
Revised: Then we retrieve m nearest neighbors {{ c (1)1 , ..., c (1) m } , ..., { c ( L ) 1 , ..., c ( L ) m }} of q t for each class, where the superscript L denotes the total number of the classes and the c ( l ) i is retrieved as the i -th nearest neighbor in the l -th class.

Original: After the model retrieves the Top- m candidates for each class, their corresponding representation h ( l ) ˆ c i and label word v ( l ) from knowledge-store will be incorporated into the encoder to act as a demonstration learning.
Revised: After the model retrieves the Top- m candidates for each class, their corresponding representation h ( l ) ˆ c i and label word v ( l ) from knowledge-store will be incorporated into the encoder to act as a demonstration learning.

Original: Since the h ( l ) ˆ c i is already vector, we intuitively aggregate the m neighbor vectors for each class according to their similarity and incorporate the demonstration into the input representation of ˆ x after the word embedding layer of the M as follows:
Revised: Since the h ( l ) ˆ c i is already vector, we intuitively aggregate the m neighbor vectors for each class according to their similarity and incorporate the demonstration into the input representation of ˆ x after the word embedding layer of the M as follows:

Original: [Equation]
Revised: [Equation]

Original: where e ( · ) represents the word embedding layer of M , ⊕ denotes the concatenation of input sequences, α ( l ) i is the softmax score for the i -th retrieval belonging to l -th class label to denote their relevance with ˆ q , and I is the sequence features for inputting the next layer of PLM.
Revised: where e ( · ) represents the word embedding layer of M , ⊕ denotes the concatenation of input sequences, α ( l ) i is the softmax score for the i -th retrieval belonging to l -th class label to denote their relevance with ˆ q , and I is the sequence features for inputting the next layer of PLM.

Original: As shown in the above equation, we encode demonstration representation with the weighted sum of the retrieval representation.
Revised: As shown in the above equation, we encode demonstration representation with the weighted sum of the retrieval representation.

Original: Thus, retrieval scores are directly used in the final representation, making the framework differentiable.
Revised: Thus, retrieval scores are directly used in the final representation, making the framework differentiable.

Original: To this end, we denote this style of demonstration as neural demonstration , significantly different from prior work of discrete demonstration [13].
Revised: To this end, we denote this style of demonstration as neural demonstration , significantly different from prior work of discrete demonstration [12].

Original: Neural vs. Discrete Demonstration Compared with prior discrete demonstrations described in [13, 36, 50, 29], retrieving weighted neural demonstrations from the knowledge-store to augment prompt learning has advantages in the following three major aspects: (1) neural demonstrations could be more tolerant of the model’s maximum input length than discrete demonstrations, while the discrete demonstration is usually not suitable for multi-class classification tasks due to the limitation of input length, such as relation extraction, etc.
Revised: Neural vs. Discrete Demonstration Compared with prior discrete demonstrations described in [12, 35, 49, 28], retrieving weighted neural demonstrations from the knowledge-store to augment prompt learning has advantages in the following three major aspects: (1) neural demonstrations could be more tolerant of the model’s maximum input length than discrete demonstrations, while the discrete demonstration is usually not suitable for multi-class classification tasks due to the limitation of input length, such as relation extraction, etc.

Original: (2) the model needs to deal with large retrieval tokens for discrete demonstration, making it time-consuming and computationally intensive to perform cross-attention operations due to the quadratic attention complexity.
Revised: (2) the model needs to deal with large retrieval tokens for discrete demonstration, making it time-consuming and computationally intensive to perform cross-attention operations due to the quadratic attention complexity.

Original: In contrast, dealing with much shorter instance representations as neural demonstrations unleashes the potential of cross-attention and accelerates the inference.
Revised: In contrast, dealing with much shorter instance representations as neural demonstrations unleashes the potential of cross-attention and accelerates the inference.

Original: (3) when sampling examples based on the similarity between instances, our cloze-style contextual representation is more informative and consistent than the contextual representation from [CLS] of Sentence-BERT [48] (adopted in LM-BFF).
Revised: (3) when sampling examples based on the similarity between instances, our cloze-style contextual representation is more informative and consistent than the contextual representation from [CLS] of Sentence-BERT [47] (adopted in LM-BFF).

Original: 3.3 Retrieve k NN for Guiding Training
Revised: 3.3 Retrieve k NN for Guiding Training

Original: Eager learners (e.g., PLMs) are optimized to learn a global function that maps from the text to semantic label space.
Revised: Eager learners (e.g., PLMs) are optimized to learn a global function that maps from the text to semantic label space.

Original: Lazy learners such as k -nearest neighbor classifiers, on the contrary, aims to approximating the neighborhoods around those test examples [2].
Revised: Lazy learners such as k -nearest neighbor classifiers, on the contrary, aims to approximating the neighborhoods around those test examples [2].

Original: Since k NN can easily predict for each encountered query instance based on pre-trained representation without an extra classifier, it is intuitively to leverage the k NN’s classification results as the prior external knowledge to guide the
Revised: Since k NN can easily predict for each encountered query instance based on pre-trained representation without an extra classifier, it is intuitively to leverage the k NN’s classification results as the prior external knowledge to guide the

Original: PLMs’ parameters attending to hard examples (hard samples usually refer to atypical samples) during the training process (also referred as k NN-train for the abbreviation).
Revised: PLMs’ parameters attending to hard examples (hard samples usually refer to atypical samples) during the training process (also referred as k NN-train for the abbreviation).

Original: Particularly, our intuition is to differentiate between easy and hard examples according to the prediction of k NN.
Revised: Particularly, our intuition is to differentiate between easy and hard examples according to the prediction of k NN.

Original: Given the t -th query instance q t , we leverage the h q t querying the open-book knowledge-store ( K , V ) to retrieve the k -nearest neighbors N of q t according to a similarity function d ( · , · ) , where d ( · , · ) typically adopt the inner product similarity.
Revised: Given the t -th query instance q t , we leverage the h q t querying the open-book knowledge-store ( K , V ) to retrieve the k -nearest neighbors N of q t according to a similarity function d ( · , · ) , where d ( · , · ) typically adopt the inner product similarity.

Original: Then, we compute the distribution over neighbors according to the softmax of their similarities and aggregate probability mass for each label word across its occurrences in the retrieved targets:
Revised: Then, we compute the distribution over neighbors according to the softmax of their similarities and aggregate probability mass for each label word across its occurrences in the retrieved targets:

Original: [Equation]
Revised: [Equation]

Original: Given the probability p k NN of the query instance q t being predicted as the gold class (also as the probability value of the gold class in the P k NN ), we propose to retrieve the k NN for guiding the training process of prompt learning.
Revised: Given the probability p k NN of the query instance q t being predicted as the gold class (also as the probability value of the gold class in the P k NN ), we propose to retrieve the k NN for guiding the training process of prompt learning.

Original: The k NN guider reweights the cross-entropy loss L CE by adjusting the relative loss for the correctly-classified or misclassified instances identified by k NN, respectively.
Revised: The k NN guider reweights the cross-entropy loss L CE by adjusting the relative loss for the correctly-classified or misclassified instances identified by k NN, respectively.

Original: Specifically, we apply the negative log-likelihood as the modulating factor F ( p k NN ) .
Revised: Specifically, we apply the negative log-likelihood as the modulating factor F ( p k NN ) .

Original: The final loss L is defined as:
Revised: The final loss L is defined as:

Original: [Equation]
Revised: [Equation]

Original: where β denotes a scalar to determine the proportion of each loss term.
Revised: where β denotes a scalar to determine the proportion of each loss term.

Original: Note that p k NN is computed using the leave-one-out distribution on the training set due to the fact that each example in the training set cannot retrieve itself.
Revised: Note that p k NN is computed using the leave-one-out distribution on the training set due to the fact that each example in the training set cannot retrieve itself.

Original: The motivation of modulating factor is inspired by Focal-loss [35], while we focus on exploit leveraging k-NN’s results for calibrating the training of LMs.
Revised: The motivation of modulating factor is inspired by Focal-loss [34], while we focus on exploit leveraging k-NN’s results for calibrating the training of LMs.

Original: 3.4 k NN based probability for Cloze-style Prediction
Revised: 3.4 k NN based probability for Cloze-style Prediction

Original: Apart from the neural demonstration on the input side and k NN guided training process (also referred as k NN-test for the abbreviation), we further present k NN based probability for Cloze-style prediction on the inference process, providing the PLM ability to retrieve nearest neighbors for decisions rather than making predictions only based on memorized parameters.
Revised: Apart from the neural demonstration on the input side and k NN guided training process (also referred as k NN-test for the abbreviation), we further present k NN based probability for Cloze-style prediction on the inference process, providing the PLM ability to retrieve nearest neighbors for decisions rather than making predictions only based on memorized parameters.

Original: Given the non-parametric k nearest neighbor distribution P k NN of the query instance q t being predicted as y , we follow [14, 27, 17] to reformulate the P ( y | q t ) by interpolating the P k NN with the already-trained base PLM’s MLM prediction P M using parameter λ to produce the final probability of the label:
Revised: Given the non-parametric k nearest neighbor distribution P k NN of the query instance q t being predicted as y , we follow [13, 26, 16] to reformulate the P ( y | q t ) by interpolating the P k NN with the already-trained base PLM’s MLM prediction P M using parameter λ to produce the final probability of the label:

Original: [Equation]
Revised: [Equation]

Original: Different from k NN-LM [27, 17] that mainly retrieve tokens to augment the language modeling, we focus on leveraging prompt-based kNN’s distribution for reference at test time, which can unlock the model prediction process as an open-book examination for prompt learning.
Revised: Different from k NN-LM [26, 16] that mainly retrieve tokens to augment the language modeling, we focus on leveraging prompt-based kNN’s distribution for reference at test time, which can unlock the model prediction process as an open-book examination for prompt learning.

Original:  Experiments 4.1 Datasets and Baselines
Revised:  Experiments 4.1 Datasets and Baselines

Original: Datasets We evaluate R ETRO P ROMPT on several types of natural language understanding tasks, including single sentence classification tasks (SST-2 [55], MR [44], and CR [20]) and sentence pair classification tasks (MNLI [59], QNLI [47], and QQP 5 ).
Revised: Datasets We evaluate R ETRO P ROMPT on several types of natural language understanding tasks, including single sentence classification tasks (SST-2 [54], MR [43], and CR [19]) and sentence pair classification tasks (MNLI [58], QNLI [46], and QQP 5 ).

Original: To further evaluate the effectiveness of the proposed approach with multi-class classification, we also conduct experiments on the information extraction tasks, including FewNERD [9], SemEval [-
-] Task 8 (SemEval) [18], and TACRED [63].
Revised: To further evaluate the effectiveness of the proposed approach with multi-class classification, we also conduct experiments on the information extraction tasks, including FewNERD [8], SemEval [-
-]2010 Task 8 (SemEval) [17], and TACRED [61].

Original: The detailed statistics of the datasets are shown in Appendix A.
Revised: The detailed statistics of the datasets are shown in Appendix A.

Original: Baselines We compare with LM-BFF [13] for single sentence and sentence pair classification tasks and adopt SOTA prompt learning model KnowPrompt [6] as the baseline for information extraction tasks.
Revised: Baselines We compare with LM-BFF [12] for single sentence and sentence pair classification tasks and adopt SOTA prompt learning model KnowPrompt [6] as the baseline for information extraction tasks.

Original: Note that the discrete demonstration method cannot be applied to multi-class classification tasks due to the input length limitations; thus, we leave out the experimental table about the results of KnPr (D-demo).
Revised: Note that the discrete demonstration method cannot be applied to multi-class classification tasks due to the input length limitations; thus, we leave out the experimental table about the results of KnPr (D-demo).

Original: We also compare our R ETRO P ROMPT with the knowledge-enhanced prompt learning
Revised: We also compare our R ETRO P ROMPT with the knowledge-enhanced prompt learning

Original: [Table]
Revised: [Table]

Original: [Table][Table] [Table] Ours
Revised: [Table]

Original: [Table][Table] [Table] [Table]
Revised: [Table]

Original: LOTClass ♣

Original: [Table][Table]
Revised: [Table]

Original: ♣
Revised: ♣

Original: [Table]
Revised: [Table]

Original: FT [Table] [Table] [Table]
Revised: ♣

Original: [Table]
Revised: [Table]

Original: FT
Revised: FT

Original: [Table] [Table]
Revised: [Table] [Table]

Original: Ours
Revised: Ours

Original: [Table]
Revised: [Table]

Original: method KPT [21] since KPT leverages the external knowledge base for enhancing prompt learning while we focus on utilizing internal trainsets as a knowledge-store.
Revised: method KPT [20] since KPT leverages the external knowledge base for enhancing prompt learning while we focus on utilizing internal trainsets as a knowledge-store.

Original: You can refer to Appendix B for  detailed introduction of baseline methods.
Revised: You can refer to Appendix B for the detailed introduction of baseline methods.

Original: 4.2 Evaluation protocols and details
Revised: 4.2 Evaluation protocols and details

Original: The experiments are implemented on 1 NVIDIA V100 and utilize Pytorch [45] as the base library.
Revised: The experiments are implemented on 1 NVIDIA V100 and utilize Pytorch [44] as the base library.

Original: We adopt RoBERTa large [39] as the PLM and employ AdamW as the optimizer for all experiments.
Revised: We adopt RoBERTa large [38] as the PLM and employ AdamW as the optimizer for all experiments.

Original: To mitigate the influence of diverse templates, we conduct baselines and R ETRO P ROMPT with the same templates for each dataset.
Revised: To mitigate the influence of diverse templates, we conduct baselines and R ETRO P ROMPT with the same templates for each dataset.

Original: We list the specific experimental settings and tuning retrieve parameters in Appendix C and D. As for few-shot and zero-shot experiments, we leverage different settings, respectively.
Revised: We list the specific experimental settings and tuning retrieve parameters in Appendix C and D. As for few-shot and zero-shot experiments, we leverage different settings, respectively.

Original: [Figure]
Revised: [Figure]

Original: Few-shot Setting.
Revised: Few-shot Setting.

Original: We follow the few-shot setting of LM-BFF [13] to conduct 4-shot and 16-shot experiments and evaluate the average performance with a fixed set of seeds, S seed , across several different sampled D train for each task.
Revised: We follow the few-shot setting of LM-BFF [12] to conduct 4-shot and 16-shot experiments and evaluate the average performance with a fixed set of seeds, S seed , across several different sampled D train for each task.

Original: Note that our knowledge-store is constructed with the few-shot training set in this setting.
Revised: Note that our knowledge-store is constructed with the few-shot training set in this setting.

Original: Zero-shot Setting 6 .
Revised: Zero-shot Setting 6 .

Original: We leverage vanilla RoBERTa large for all baselines (except LOTClass [42]) to directly inference on the test set.
Revised: We leverage vanilla RoBERTa large for all baselines (except LOTClass [41]) to directly inference on the test set.

Original: To take advantage of retrieval mechanism, R ETRO P ROMPT follows
Revised: To take advantage of retrieval mechanism, R ETRO P ROMPT follows

Original: LOTClass [42] to utilize unlabeled trainsets for retrieval.
Revised: LOTClass [41] to utilize unlabeled trainsets for retrieval.

Original: Specifically, we take the vanilla RoBERTa large to tag the pseudo labels on unlabeled trainset and create the open-book knowledge-store with the unlabeled trainsets and pseudo labels.
Revised: Specifically, we take the vanilla RoBERTa large to tag the pseudo labels on unlabeled trainset and create the open-book knowledge-store with the unlabeled trainsets and pseudo labels.

Original: Lastly, R ETRO P ROMPT make predictions on the test set based on the constructed datastore without tuning any of the model parameters .
Revised: Lastly, R ETRO P ROMPT make predictions on the test set based on the constructed datastore without tuning any of the model parameters .

Original: 4.3 Experimental Results
Revised: 4.3 Experimental Results

Original: Few-shot Results.
Revised: Few-shot Results.

Original: As shown in Table 1, we find R ETRO P ROMPT consistently outperforms baseline method LM-BFF and KnowPrompt, both in 4-shot and 16-shot experiments.
Revised: As shown in Table 1, we find R ETRO P ROMPT consistently outperforms baseline method LM-BFF and KnowPrompt, both in 4-shot and 16-shot experiments.

Original: Especially for information extraction tasks with multiple classes, discrete demonstrations cannot be applied to the input due to the limited input sequence length, while our neural demonstration can also work and achieves improvement on these multi-class datasets.
Revised: Especially for information extraction tasks with multiple classes, discrete demonstrations cannot be applied to the input due to the limited input sequence length, while our neural demonstration can also work and achieves improvement on these multi-class datasets.

Original: Moreover, R ETRO P ROMPT obtain better performance compared with KPT.
Revised: Moreover, R ETRO P ROMPT obtain better performance compared with KPT.

Original: Compared with KPT with external knowledge, we only focus on referencing the internal few-shot trainsets without visiting the external knowledge base.
Revised: Compared with KPT with external knowledge, we only focus on referencing the internal few-shot trainsets without visiting the external knowledge base.

Original: Besides, we observe that
Revised: Besides, we observe that

Original: R ETRO P ROMPT has a relatively lower standard deviation than the baselines.
Revised: R ETRO P ROMPT has a relatively lower standard deviation than the baselines.

Original: The reason may lie that the retrieval mechanism can compensate for instabilities in parametric predictions.
Revised: The reason may lie that the retrieval mechanism can compensate for instabilities in parametric predictions.

Original: [Table]
Revised: [Table]

Original: Zero-shot Results.
Revised: Zero-shot Results.

Original: From Table 1, we also observe that R ETRO P ROMPT achieves improvements in the zero-shot setting.
Revised: From Table 1, we also observe that R ETRO P ROMPT achieves improvements in the zero-shot setting.

Original: Another notable point is that
Revised: Another notable point is that

Original: R ETRO P ROMPT performs even better than KPT in the zero-shot setting, revealing that exploring own data to decouple knowledge from memorization has more potential than leveraging external knowledge.
Revised: R ETRO P ROMPT performs even better than KPT in the zero-shot setting, revealing that exploring own data to decouple knowledge from memorization has more potential than leveraging external knowledge.

Original: Moreover, we achieve superior performance to LOTClass even though we utilize the vanilla RoBERTa large without any training.
Revised: Moreover, we achieve superior performance to LOTClass even though we utilize the vanilla RoBERTa large without any training.

Original: Fully-supervised Results.
Revised: Fully-supervised Results.

Original: As shown in Figure 3, the experiments in fully-supervised settings with long-tail distribution illustrate that R ETRO P ROMPT achieves improvement compared with baselines.
Revised: As shown in Figure 3, the experiments in fully-supervised settings with long-tail distribution illustrate that R ETRO P ROMPT achieves improvement compared with baselines.

Original: This indicates that our retrieval mechanism extends the
Revised: This indicates that our retrieval mechanism extends the

Original: LM’s ability to learn hard examples in the fullysupervised datasets.
Revised: LM’s ability to learn hard examples in the fullysupervised datasets.

Original: 4.4 Model Generalization to New Domains
Revised: 4.4 Model Generalization to New Domains

Original: The scarce data may bring the overfitting problem for the lots of memory parameters of PLMs, even though prompt learning.
Revised: The scarce data may bring the overfitting problem for the lots of memory parameters of PLMs, even though prompt learning.

Original: Thus, we conduct cross-domain experiments to validate the generalization of our R ETRO P ROMPT .
Revised: Thus, we conduct cross-domain experiments to validate the generalization of our R ETRO P ROMPT .

Original: Specifically, we utilize the model trained on the source datasets and directly test on the other target datasets.
Revised: Specifically, we utilize the model trained on the source datasets and directly test on the other target datasets.

Original: From Table 2, we can find that our method consistently outperforms baselines.
Revised: From Table 2, we can find that our method consistently outperforms baselines.

Original: This finding illustrates that R ETRO P ROMPT achieves great model generalization to new domains.
Revised: This finding illustrates that R ETRO P ROMPT achieves great model generalization to new domains.

Original: 4.5 Analysis of Memorization
Revised: 4.5 Analysis of Memorization

Original: It is necessary and interesting to further explore the memorization mechanism to help us better understand the utility of retrieval for memorization in NLP.
Revised: It is necessary and interesting to further explore the memorization mechanism to help us better understand the utility of retrieval for memorization in NLP.

Original: Definition of Memorization Measurement.
Revised: Definition of Memorization Measurement.

Original: Inspired by the idea of [11] in the computer vision area, we define memorization measures as to how the classification varies when a training instance z is deleted from the trainset.
Revised: Inspired by the idea of [10] in the computer vision area, we define memorization measures as to how the classification varies when a training instance z is deleted from the trainset.

Original: We follow [28, 64] to define and derive the memorization score for a training instance z as follows:
Revised: We follow [27, 62] to define and derive the memorization score for a training instance z as follows:

Original: [Equation]
Revised: [Equation]

Original: where ˆ θ ξ, − z denotes the parameters trained with the instance z down-weighted by ξ , ˆ θ refers to the parameters of the model trained with all instances and H ˆ θ = 1 n (cid:80) ni
Revised: where ˆ θ ξ, − z denotes the parameters trained with the instance z down-weighted by ξ , ˆ θ refers to the parameters of the model trained with all instances and H ˆ θ = 1 n (cid:80) ni

Original: =1 ∇ 2 θ L ( z i , ˆ θ ) .
Revised: =1 ∇ 2 θ L ( z i , ˆ θ ) .

Original: Thus S delate ( z ) refers to the amount of change of P ( y | x ; θ ) when the instance z is down-weighted by ξ .
Revised: Thus S delate ( z ) refers to the amount of change of P ( y | x ; θ ) when the instance z is down-weighted by ξ .

Original: Top-memorized Instances: Typical or Atypical?
Revised: Top-memorized Instances: Typical or Atypical?

Original: Since the SST-2 dataset provides the annotations of phrase-level sentiment polarity labels, we adopt SST-2 to analyze the memorization by judging the atypical of an instance by checking the percentage of positive phrases.
Revised: Since the SST-2 dataset provides the annotations of phrase-level sentiment polarity labels, we adopt SST-2 to analyze the memorization by judging the atypical of an instance by checking the percentage of positive phrases.

Original: We achieve such statistics from SST-2 and observe that a typical positive instance has a relatively high percentage of positive phrases, and a typical negative instance should have a relatively low percentage of positive phrases.
Revised: We achieve such statistics from SST-2 and observe that a typical positive instance has a relatively high percentage of positive phrases, and a typical negative instance should have a relatively low percentage of positive phrases.

Original: Based on the above observation, we apply the memorization score defined in Eq.
Revised: Based on the above observation, we apply the memorization score defined in Eq.

Original: 8 to select Top-10% and Bottom-10% memorized instances from the trainset and collect the average percentage of positive phrases in these instances.
Revised: 8 to select Top-10% and Bottom-10% memorized instances from the trainset and collect the average percentage of positive phrases in these instances.

Original: As shown in Table 3,we can conclude following findings: (1) The PLM tends to give atypical samples deeper memory attention.
Revised: As shown in Table 3,we can conclude following findings: (1) The PLM tends to give atypical samples deeper memory attention.

Original: Specifically, no matter LM-BFF or our method, the top-
Revised: Specifically, no matter LM-BFF or our method, the top-

Original: [Table]
Revised: [Table]

Original: 10% memorized negative instances have a higher percentage of positive phrases than the average percentage of positive phrases of all negative instances.
Revised: 10% memorized negative instances have a higher percentage of positive phrases than the average percentage of positive phrases of all negative instances.

Original: 2) LM-BFF has lower memorization scores on hard samples than fine-tuning.
Revised: 2) LM-BFF has lower memorization scores on hard samples than fine-tuning.

Original: We think it owns to prompt learning can help PLMs recall what they learned from pre-training without strengthening memory for downstream data. 3)
Revised: We think it owns to prompt learning can help PLMs recall what they learned from pre-training without strengthening memory for downstream data. 3)

Original: R ETRO P ROMPT further has lower average memorization scores than fine-tuning and LM-BFF, which illustrates that our method is less memory dependent.
Revised: R ETRO P ROMPT further has lower average memorization scores than fine-tuning and LM-BFF, which illustrates that our method is less memory dependent.

Original: This result may be attributed to decoupling knowledge from memorization through retrieval to alleviating the rote of PLMs.
Revised: This result may be attributed to decoupling knowledge from memorization through retrieval to alleviating the rote of PLMs.

Original: [Table]
Revised: [Table]

Original: Case Analysis.
Revised: Case Analysis.

Original: As shown in Table 6, we manually list the bottom-ranked and top-ranked training instances of SST-2 according to our model.
Revised: As shown in Table 6, we manually list the bottom-ranked and top-ranked training instances of SST-2 according to our model.

Original: It reveals that the top-ranked memorized instances seem to show universal opinions indirectly.
Revised: It reveals that the top-ranked memorized instances seem to show universal opinions indirectly.

Original: Thus, we inspect them as atypical/hard for sentiment classification.
Revised: Thus, we inspect them as atypical/hard for sentiment classification.

Original: While those instances with 0 memorization scores are straightforward to show their opinion for sentiment classification, representing the typical instance.
Revised: While those instances with 0 memorization scores are straightforward to show their opinion for sentiment classification, representing the typical instance.

Original: Note that F ( p kNN ) is defined to represent the difficulty of the sample discriminated by k NN distribution.
Revised: Note that F ( p kNN ) is defined to represent the difficulty of the sample discriminated by k NN distribution.

Original: And the Table 6 also shows that F ( p kNN ) indeed reflect atypicality of examples, which validates the effectiveness of the k NN guided training.
Revised: And the Table 6 also shows that F ( p kNN ) indeed reflect atypicality of examples, which validates the effectiveness of the k NN guided training.

Original: 4.6 Ablation Study
Revised: 4.6 Ablation Study

Original: Component Ablation.
Revised: Component Ablation.

Original: As shown in Table 4, the performance of component ablation experiments with four variants has a clear drop, which validates the power of our retrieval component.
Revised: As shown in Table 4, the performance of component ablation experiments with four variants has a clear drop, which validates the power of our retrieval component.

Original: We also find that neural demonstration and k NN-train have more improvement in the few-shot setting than k NN-test.
Revised: We also find that neural demonstration and k NN-train have more improvement in the few-shot setting than k NN-test.

Original: Note that k NN-test is similar to k NN-LM [27, 17] and the results reveals that simply incorporate k NN in the test process of prompt learning has little influence in a few-shot setting.
Revised: Note that k NN-test is similar to k NN-LM [26, 16] and the results reveals that simply incorporate k NN in the test process of prompt learning has little influence in a few-shot setting.

Original: [Table]
Revised: [Table]

Original: Key Representation and k NN Acquisition.
Revised: Key Representation and k NN Acquisition.

Original: We study the effect of using different representations of the key in the knowledge-store.
Revised: We study the effect of using different representations of the key in the knowledge-store.

Original: We experiment with two types of representations: (1) prompt-based representation, which is the default setting, and (2) [CLS] based representation of current
Revised: We experiment with two types of representations: (1) prompt-based representation, which is the default setting, and (2) [CLS] based representation of current

Original: LM.
Revised: LM.

Original: We also experiment with two types of calculation of k NN distribution: (1) representation
Revised: We also experiment with two types of calculation of k NN distribution: (1) representation

Original: [Table]
Revised: [Table]

Original: similarity score (refer as rep-similar), which is the default setting, and (2)
Revised: similarity score (refer as rep-similar), which is the default setting, and (2)

Original: BM25 based score , which calculates the correlation score between the query and each key examples with BM25 [49] algorithm.
Revised: BM25 based score , which calculates the correlation score between the query and each key examples with BM25 [48] algorithm.

Original: Results in Table 5 show that using prompt-based representations for key and representation based similarity scores for k NN leads to the best performance.
Revised: Results in Table 5 show that using prompt-based representations for key and representation based similarity scores for k NN leads to the best performance.

Original: It suggests that prompt learn better representations for context similarity and the representation similarity based k NN distribution is better than BM25 based scores.
Revised: It suggests that prompt learn better representations for context similarity and the representation similarity based k NN distribution is better than BM25 based scores.

Original: [Table]
Revised: [Table]

Original:  Related Work
Revised:  Related Work

Original: Retrieval-enhanced PLMs.
Revised: Retrieval-enhanced PLMs.

Original: Our pipeline is partly inspired by discrete demonstration methods such as [13, 36, 29, 30] that retrieves few training examples in a natural language prompt, while we propose neural demonstration for enhancing the input to alleviate the limitations of input length.
Revised: Our pipeline is partly inspired by discrete demonstration methods such as [12, 35, 28, 29] that retrieves few training examples in a natural language prompt, while we propose neural demonstration for enhancing the input to alleviate the limitations of input length.

Original: Another line researches of retrieval augmentation [15, 24, 33, 50, 3] retrieve useful information from a external knowledge corpus (e.g., Wikipedia) for a particular task (e.g., an open-domain question).
Revised: Another line researches of retrieval augmentation [14, 23, 32, 49, 3] retrieve useful information from a external knowledge corpus (e.g., Wikipedia) for a particular task (e.g., an open-domain question).

Original: Unlike these works, we focus on retrieving examples from the internal training data.
Revised: Unlike these works, we focus on retrieving examples from the internal training data.

Original: Besides, semiparametric methods [27, 17, 26, 25,  1, 43] have risen to leverage k -nearest neighbor classifier, a classic non-parametric algorithm that makes the prediction based on representation similarities, to enhance pre-trained language models in various tasks However, unlike these models using nearest neighbors only for augmenting the process of prediction, we aim to develop a comprehensive retrieval mechanism for input, training and test process.
Revised: Besides, semiparametric methods [26, 16, 25, 24, 1, 42] have risen to leverage k -nearest neighbor classifier, a classic non-parametric algorithm that makes the prediction based on representation similarities, to enhance pre-trained language models in various tasks However, unlike these models using nearest neighbors only for augmenting the process of prediction, we aim to develop a comprehensive retrieval mechanism for input, training and test process.

Original: Prompt learning for PLMs.
Revised: Prompt learning for PLMs.

Original: With the birth of GPT-3 [4], prompt learning [37] has recently arisen to fill the gap between masked LM objective of PLMs and downstream fine-tuning objective.
Revised: With the birth of GPT-3 [4], prompt learning [36] has recently arisen to fill the gap between masked LM objective of PLMs and downstream fine-tuning objective.

Original: Prompt learning has achieves very impressive performance on various tasks, such as text classification [52, 54], named entity recognition [5, 41], relation extraction [16, 6], event extraction [19, 61], machine translation [56] and language generation [53], especially under the setting of few-shot learning.
Revised: Prompt learning has achieves very impressive performance on various tasks, such as text classification [51, 53], named entity recognition [5, 40], relation extraction [15, 6], event extraction [18, 60], machine translation [55] and language generation [52], especially under the setting of few-shot learning.

Original: Moreover, continuous prompts have also been proposed [34, 31, 38] to reduce prompt engineering, which directly appends a series of learnable continuous embeddings as prompts into the input sequence.
Revised: Moreover, continuous prompts have also been proposed [33, 30, 37] to reduce prompt engineering, which directly appends a series of learnable continuous embeddings as prompts into the input sequence.

Original: Our work is orthogonal to previous prompt learning approaches, which aim to optimize prompts, while we focus on the systematic study of retrieving related examples from training data to enhance prompt learning.
Revised: Our work is orthogonal to previous prompt learning approaches, which aim to optimize prompts, while we focus on the systematic study of retrieving related examples from training data to enhance prompt learning.

Original:  Conclusion and Future Work
Revised:  Conclusion and Future Work

Original: We propose R ETRO P ROMPT that decouples knowledge from memorization by introducing retrieval augmentation to further improve the generalization ability of prompt learning on the input side and the whole process of model training and prediction.
Revised: We propose R ETRO P ROMPT that decouples knowledge from memorization by introducing retrieval augmentation to further improve the generalization ability of prompt learning on the input side and the whole process of model training and prediction.

Original: R ETRO P ROMPT , is a straightforward yet effective retrieval method that combines both neural demonstrations, k NN guider for training and prediction.
Revised: R ETRO P ROMPT , is a straightforward yet effective retrieval method that combines both neural demonstrations, k NN guider for training and prediction.

Original: Our extensive results show that it outperforms other demonstration-enhanced prompt methods and knowledge-enhanced prompt methods in few-shot, zero-shot and fully-supervised settings.
Revised: Our extensive results show that it outperforms other demonstration-enhanced prompt methods and knowledge-enhanced prompt methods in few-shot, zero-shot and fully-supervised settings.

Original: Analyzing the essence of memorization validates the effectiveness of decoupling knowledge from memorization.
Revised: Analyzing the essence of memorization validates the effectiveness of decoupling knowledge from memorization.

Original: Interesting future directions include: 1) apply to other tasks, such as QA and NLG, 2) explore the noise data mining for unsupervised learning, 3) further improve the retrieve efficiency for large datasets, etc.
Revised: Interesting future directions include: 1) apply to other tasks, such as QA and NLG, 2) explore the noise data mining for unsupervised learning, 3) further improve the retrieve efficiency for large datasets, etc.

Original: Acknowledgments
Revised: Acknowledgments

Original: We want to express gratitude to the anonymous reviewers for their kind comments.
Revised: We want to express gratitude to the anonymous reviewers for their kind comments.

Original: This work was supported by National Natural Science Foundation of China (No.62206246, 91846204 and
Revised: This work was supported by National Natural Science Foundation of China (No.62206246, 91846204 and

Original: U19B2027), Zhejiang Provincial Natural Science Foundation of China (No. LGG22F030011),
Revised: U19B2027), Zhejiang Provincial Natural Science Foundation of China (No. LGG22F030011),

Original: Ningbo Natural Science Foundation (2021J190), and Yongjiang Talent Introduction Programme (2021A-156-G).
Revised: Ningbo Natural Science Foundation (2021J190), and Yongjiang Talent Introduction Programme (2021A-156-G).

Original: Checklist
Revised: Checklist

Original: For all authors... (a) Do the main claims made in the abstract and introduction accurately reflect the paper’s contributions and scope?
Revised: For all authors... (a) Do the main claims made in the abstract and introduction accurately reflect the paper’s contributions and scope?

Original: [Yes] (b) Did you describe the limitations of your work?
Revised: [Yes] (b) Did you describe the limitations of your work?

Original: [Yes] See Appendix.
Revised: [Yes] See Appendix.

Original: (c) Did you discuss any potential negative societal impacts of your work?
Revised: (c) Did you discuss any potential negative societal impacts of your work?

Original: [Yes] See Appendix.
Revised: [Yes] See Appendix.

Original: (d) Have you read the ethics review guidelines and ensured that your paper conforms to them?
Revised: (d) Have you read the ethics review guidelines and ensured that your paper conforms to them?

Original: [Yes] 
Revised: [Yes] 2.

Original: If you are including theoretical results... (a) Did you state the full set of assumptions of all theoretical results?
Revised: If you are including theoretical results... (a) Did you state the full set of assumptions of all theoretical results?

Original: [N/A] (b) Did you include complete proofs of all theoretical results?
Revised: [N/A] (b) Did you include complete proofs of all theoretical results?

Original: [N/A] 3.
Revised: [N/A] 3.

Original: If you ran experiments... (a) Did you include the code, data, and instructions needed to reproduce the main experimental results (either in the supplemental material or as a URL)?
Revised: If you ran experiments... (a) Did you include the code, data, and instructions needed to reproduce the main experimental results (either in the supplemental material or as a URL)?

Original: [Yes] We include the source code and data in our supplemental material submission, and we outline the data generation procedure, the evaluation protocol, the training regime, and everything else necessary for reproduction either in the main body of the paper or in the appendix.
Revised: [Yes] We include the source code and data in our supplemental material submission, and we outline the data generation procedure, the evaluation protocol, the training regime, and everything else necessary for reproduction either in the main body of the paper or in the appendix.

Original: (b) Did you specify all the training details (e.g., data splits, hyperparameters, how they were chosen)?
Revised: (b) Did you specify all the training details (e.g., data splits, hyperparameters, how they were chosen)?

Original: [Yes] See Subsection 4.2 and Appendix.
Revised: [Yes] See Subsection 4.2 and Appendix.

Original: (c) Did you report error bars (e.g., with respect to the random seed after running experiments multiple times)?
Revised: (c) Did you report error bars (e.g., with respect to the random seed after running experiments multiple times)?

Original: [Yes] We list the standard deviation for few-shot setting.
Revised: [Yes] We list the standard deviation for few-shot setting.

Original: (d) Did you include the total amount of compute and the type of resources used (e.g., type of GPUs, internal cluster, or cloud provider)?
Revised: (d) Did you include the total amount of compute and the type of resources used (e.g., type of GPUs, internal cluster, or cloud provider)?

Original: [Yes] We introduce type of resources in Section 4.2. 4.
Revised: [Yes] We introduce type of resources in Section 4.2. 4.

Original: If you are using existing assets (e.g., code, data, models) or curating/releasing new assets... (a) If your work uses existing assets, did you cite the creators?
Revised: If you are using existing assets (e.g., code, data, models) or curating/releasing new assets... (a) If your work uses existing assets, did you cite the creators?

Original: [Yes] (b) Did you mention the license of the assets?
Revised: [Yes] (b) Did you mention the license of the assets?

Original: [No] The code and the data are proprietary.
Revised: [No] The code and the data are proprietary.

Original: (c) Did you include any new assets either in the supplemental material or as a URL?
Revised: (c) Did you include any new assets either in the supplemental material or as a URL?

Original: [No] (d) Did you discuss whether
Revised: [No] (d) Did you discuss whether

Original: and how consent was obtained from people whose data you’re using/curating?
Revised: and how consent was obtained from people whose data you’re using/curating?

Original: [No] The code and the data are proprietary.
Revised: [No] The code and the data are proprietary.

Original: (e) Did you discuss whether the data you are using/curating contains personally identifiable information or offensive content?
Revised: (e) Did you discuss whether the data you are using/curating contains personally identifiable information or offensive content?

Original: [No] 5.
Revised: [No] 5.

Original: If you used crowdsourcing or conducted research with human subjects... (a) Did you include the full text of instructions given to participants and screenshots, if applicable?
Revised: If you used crowdsourcing or conducted research with human subjects... (a) Did you include the full text of instructions given to participants and screenshots, if applicable?

Original: [N/A] (b) Did you describe any potential participant risks, with links to Institutional Review Board (IRB) approvals, if applicable?
Revised: [N/A] (b) Did you describe any potential participant risks, with links to Institutional Review Board (IRB) approvals, if applicable?

Original: [N/A] (c) Did you include the estimated hourly wage paid to participants and the total amount spent on participant compensation?[N/A] A Datasets and Templates
Revised: [N/A] (c) Did you include the estimated hourly wage paid to participants and the total amount spent on participant compensation?

Original: In this section, we introduce the datasets as shown in Table 7 and list the templates we use in experiments as follows.

Original: [Table]

Original: SST-2, MR, CR .

Original: For the single sentence classification tasks, we follow the LM-BFF [13] to design the templates:

Original: T ( x ) = [CLS] x It was [MASK] . We set Verbalizer: (great/terrible) → (positive/negative) for SST-2 MR and CR.
Revised: [N/A

Original: For the Yahoo dataset, we assign the Verbalizer following the original labels.

Original: MNLI, QNLI, QQP .

Original: For the sentence pair classification tasks, we follow LM-BFF [13] to set

Original: Verbalizer: (Yes/Maybe/No) →

Original: (entailment/neutral/contradiction), and define the following templates:

Original: T ( x 1 , x 2 ) =

Original: [CLS] x 1 ?

Original: [MASK] , x

Original: FewNERD, SemEval, TACRED .

Original: FewNERD, SemEval and TACRED are datasets for information extraction, which require inserting the entity into the template.

Original: Therefore, we follow [8] and [6] to define the template and verbalizers.

Original: B Compared Baselines

Original: In this subsection, we introduce the baselines we compare with and re-produce them under the same settings with their open-source codes.

Original: LM-BFF uses several other tricks, such as prompt ensemble, while KPT utilizes tremendous external knowledge.

Original: We do not use any of these tricks and external knowledge since we get the most out of the data to decouple part of knowledge from parametric memorization.

Original: Our R ETRO P ROMPT mechanism is orthogonal to other methodological improvements of prompt-tuning (such as continuous prompt in

Original: P-tuning [38] and DART [62] ) and can be combined with other prompt-tuning methods in future work.

Original: Fine-tuning (FT).

Original: The traditional fine-tuning method regard the hidden embedding of [CLS] token of the PLM as the representation of the sentence and then feeds them into a classification layer to make predictions.

Original: LM-BFF.

Original: LM-BFF [13] is a typical prompt-tuning method wrapping an input sentence into a handcrafted template.

Original: Here we re-produce LM-BFF based on their open-source codes 7 with the same manual prompts as R ETRO P ROMPT for a fair comparison.

Original: LM-BFF (+Demo).

Original: This approach is the above LM-BFF [13] combined with the demonstration [4].

Original: Different from R ETRO P ROMPT , it uses examples of natural language as demonstrations, which is restricted by the input length of the language model.

Original: Thus, LM-BFF (+demo) is not suitable for multi-class classification tasks.

Original: KnowPrompt.

Original: KnowPrompt [6] is a SOTA prompt-tuning method for relation extraction tasks with multiple classes.

Original: We apply our R ETRO P ROMPT over KnowPrompt on information extraction tasks for comparison, aiming to verify the broad applicability of our method.

Original: Incorporating Knowledge into Prompt (KPT).

Original: KPT [21] focuses on incorporating external knowledge into the verbalizer by refining the expanded label word space to improve and stabilize prompttuning, which is a solid baseline for comparison.

Original: We follow their public codes 8 to conduct experiments in the same setting for a fair comparison.

Original: LOTClass.

Original: LOTClass [42] is the SOTA method in unsupervised text classification that utilizes the

Original: PLM to extract the label-related words from the whole unlabeled training corpus.

Original: Then it leverages the Masked Category Prediction task to train on the unlabeled corpus with pseudo labels.

Original: C Experimental Settings

Original: We report the hyper-parameters in Table 8.

Original: Most of the hyper-parameters are the default parameters of LM-BFF 9 .

Original: D Tuning Retrieve Parameters

Original: The final distribution of the label is affected by the hyperparameters of β , k and λ when conducting k NN-train and k NN-test.

Original: Thus, we provide insight into the effect of β , k and λ on the final results.

Original: [Figure]

Original: β varies: Figure 4(a) shows the performance of the model when the β increases and reveals that the model performs worse as the β increases on a 16-shot CR dataset.

Original: This finding indicates that a moderate degree of k NN guiding training is essential since k NN can help the model attend to hard examples, but excessive attendance of k NN-train also can bring the noise.

Original: λ varies: From Figure 4(b), we observe that model achieves optimal results on a 16-shot MR dataset when λ is set to be 0.2 while attaining the best results on MR in the zero-shot setting when λ is set to be 0.7.

Original: We think the model may require more reference when there is no data for training.

Original: k varies: As shown in Figure 4(c), the model performance in the 16-shot MR dataset fluctuates very little.

Original: In contrast, the result in the zero-shot MR dataset continues to improve as k increases until it converges when reaching a threshold ( k = 256 ).

Original: It illustrates that the k -NN retrieval provides more evidence for reference in zero-setting.

Original: E Discussion of Limitation

Original: Analysis of Efficiency We make the comparison between LM-BFF (man), LM-BFF (+demo) and

Original: R ETRO P ROMPT in speed on the MR dataset for the 16-shot setting.

Original: We observe that the speed of

Original: R ETRO P ROMPT and LM-BFF (+demo) are approximately 1.12 and 20 times slower than LM-BFF (man) on the 16-shot MR dataset.

Original: The slow inference of LM-BFF (+demo) is due to the fact that they sample from the top r % instances ( r = 50 ) for each class to use as demonstrations and vastly increase the length of the input, thus, increasing computational complexity significantly.

Original: And the bottleneck of computational speed is general limitations of retrieval methods, and our method is no exception.

Original: We will leave the engineering optimization about retrieval speed in our future work.

Original: Analysis of memory usage Actually, our method adopt FAISS tools for retrieval.

Original: FAISS is an excellent open-source library for fast nearest neighbor retrieval in high-dimensional spaces, which supports searching only from RAM, which involves k-means clustering for improving memory usage efficiency.

Original: Memory usage is negligible in the few-shot settings and acceptable in the full-data settings.

Original: Our retrieval process is performed mainly on CPU, and we compare the utilization of CPU with and without retrieval in the SST-2 full setting as follows:

Original: • The CPU utilization was 46.2% with the retrieval process and 2.5% without it (Our CPU is Intel(R)

Original: Xeon(R)

Original: Silver 4210R CPU @

Original: 2.40GHz with 40 cores).

Original: • In terms of memory usage, adding retrieval requires about 2.5G more memory than not.

Original: One way to reduce resource usage is to store the datastore on the disk in advance, then read and release it in the retrieval process.