$k$NN-LM Does Not Improve Open-ended Text Generation
Authors: Shufan Wang
Yixiao Song
Andrew Drozdov
Aparna Garimella
Varun Manjunatha
Mohit Iyyer
Venue: EMNLP 2023 Main (2023)
Keywords: retrieval-augmented language model
text generation
text generation evaluation
kNN-LM
TL;DR: Despite reducing perplexity, interpolation-based retrieval-augmented LMs (exemplified by the kNN-LM) do not improve open-ended text generation quality significantly.

Abstract: In this paper, we study the generation quality of interpolation-based retrieval-augmented language models (LMs). These methods, best exemplified by the $k$NN-LM, interpolate the LM's predicted distribution of the next word with a distribution formed from the most relevant retrievals for a given prefix. While the $k$NN-LM and related methods yield impressive decreases in perplexity, we discover that they do not exhibit corresponding improvements in open-ended generation quality, as measured by both automatic evaluation metrics (e.g., MAUVE) and human evaluations. Digging deeper, we find that interpolating with a retrieval distribution actually increases perplexity compared to a baseline LM for the majority of tokens in the WikiText-103 test set, even though the overall perplexity is lower due to a smaller number of tokens for which perplexity dramatically decreases after interpolation. However, when decoding a long sequence at inference time, significant improvements on this smaller subset of tokens are washed out by slightly worse predictions on most tokens. Furthermore, we discover that the entropy of the retrieval distribution increases faster than that of the base LM as the generated sequence becomes longer, which indicates that retrieval is less reliable when using model-generated text as queries (i.e., is subject to exposure bias). We hope that our analysis spurs future work on improved decoding algorithms and interpolation strategies for retrieval-augmented language models.

KNN-LM Does Not Improve Open-ended Text Generation
Shufan Wang Yixiao Song' Andrew Drozdov!
Aparna Garimella' Varun Manjunatha' Mohit Iyyer'
University of Massachusetts Amherst' Adobe Research² (shufanwang, yixiaosong, adrozdov, {garimell, ymanjuna Seadojenne@ddeee.....ee
edu
/@adobe.com com com
Abstract
paper, we study interpolation-busee LMs, a sub- type of retrieval-augmented L.Ss that compute the probability of the next token by interpolating be- tween the softmax distribution the original LM and a token distribution formed by retrieving over an external datastore. These methods, perhaps best exemplified by the kNN-LM (Khandelwal et al., 2020), particularly attractive because they allow any pretrained LM be retrofitted with retrieval module without further training.
this paper, we study the generation quality of interpolation-baseed retrieval-augmented lan- guage models (LMs). These methods, best ex- emplified by the &NN-LM (Khandelwal et al., 2020), interpolate the LM's predicted distribu- tion of the next word with a distribution formed from most relevant retrievals
given pre- fix. While the ANN-LM and related methods yield impressive decreases in perplexity, we discover that they not exhibit corresponding improvements in open-ended generation qual- iry, as measured by both automatic evaluation metrics (e.g.. MAUVE) and human evaluations. Digging deeper, we find that interpolating with a retrieval distribution actually increases per- plexity compared to the baseline LM for the majority of tokens in the Iik test set, even though the overall perplexity is lower due to smaller number of tokens for which per- plexity dramatically decreases after interpola- tion. However, when decoding a long sequence at inference time. significant improvements on this smaller subset of tokens are washed out by slightly worse predictions most tokens. Furthermore, we discover that the entropy of the retrieval distribution increases faster than that of the base LM as the generated sequence becomes longer, which indicates that retrieval is less reliable when using model-generatee text queries (i.e., is subject to exposure bias). We hope that our analysis spurs future work improved decoding algorithms and interpo- lation strategies for retrieval-augee augmented lan- guage models.
Despite these advantages. there is limited un- derstanding about the text generation quality of interpolation-based LMs. In this study, we evaluate the quality of generated text from two such meth- ods, kNN-LM and TRIME (Zhong et al., 2022), against the output of baseline LMs that do not use retrieval. Our evaluations involves open-ended text completions generated using different decoding algorithms on both the WikiText- 103 and PG-19 19 datasets. We discover that interpolation- based LMs do not improve the quality of generated text, as measured by both automatic text generation met- rics such MAUVE (Pillutla et al., 2021) and human evaluation.
This result begs the question of why the text gen- eration quality does not improve, the perplexity of interpolation-bassee LMs substantially lower than that of the baselines. Our analysis of the LNN- LM model suggests two potential reasons for this lack of improvement:
1. ANN-LM actually worsens the predictions of the majority of tokens in the Wik test set. On aggregate, perplexity improves because of significantly improved predictions on a smaller subset of tokens. However, when generating a long sequence of tokens, these improvements are washed out by the wors- ened predictions on other tokens.
1 Introduction
Retrieval-augmennel language models, which in- tegrate non- con-parametric dense retrieval with autore- gressive next-token prediction, have been validated with strong empirical performance across a variety of tasks (Metzler et al., 2022; Basu al., 2022: Mialon et al., 2023) addition to achieving low held-out perplexities on LM benchmarks. In this
2. The quality of the retrieval distribution deteri- orates faster than that of the LM's predicted

---

distribution as the length of the generation increases; in other words, the retrieval distri- bution is more vulnerable to exposure bias and can be easily thrown off by artifacts presented in model-generated text.
et al., 2021; Wu al., 2022; Trivedi et al., 2022; He et al., 2022). Among these, KNN-LM stands out as a relatively simple and fundamental work. Our findings indicate important weaknesses of retrieval for text generation.
Reference- metrics are not well suited to evaluate open-ended text generation (Novikova et al., 2017). Instead, effective automated ap- proaches compare the machine generated and hu- man language text distributions using samples (Me- Coy et al., 2021; ; Pillutla al., 2021; Pimentel al.. 2023). Human evaluation remains the golden stan- dard for natural language generation (Hashimoto et al., 2019; Celikyilmaz et al., 2020; Krishna et al., 2023).
Unlike previous works that rely on perplexity to evaluate language modeling or BLEU to evalu- ate machine translation quality of &NN-LM based models (Khandelwal al., 2021), our work specif- ically studies the open-ended text generation capa- bility of kNN-LMs with range of automatic eval- uation metrics as well as human evaluation. We demonstrate that, though they significantly lower perplexity, retrievers might also impair text genera- tion performance of kNN-LMs. This finding sug- gests potential future directions for using retrieval during text generation, such as developing more robust retrieval components employing retriever mechanisms more selectively during decoding.
Experimental setup
Using a variety of commonly used text generation evaluation metrics, evaluate the text generation capability of interpolation -basdd LMs and com- pare them baseline LMs (i.e., without k-nearest- neighbor retrieval from an external datastore). In this section, we describe our experimental setup, in- cluding models, automatic evaluation metrics, data selection, and hyperparameters.
2 Related Work
We present the most extensive study of open-ended text generation' from interpolation iierpolation-based LMs such &NN-LM (Khandelwal et al., 2020). Our results reveal that although these methods are effec- tive at reducing perplexity, they can also be detri- mental to text generation. Previous work finds that retrieval LMs are improved by selectively incor- porating retrieval when conditions are favorable (He et al., 2021a; Alon et al., 2022; Drozdov et al., 2022; Mallen et al., 2023), although they only ex- amine the feacher-forced setting other tasks, e.g. question answering. The ANN-MT (Khandelwal al., 2021) explores machine translation, which is a constrained task with short inputs, and thus not a good test of open-ended long-form generation.
3.1 Models
We experiment with two interpolation-basee LMs: the &NN-LM of Khandelwal al. (2020), which augments an existing pre-trained LM with a re- trieval module without any additional training, and TRIME (Zhong et al., 2022), a recent improvement over the KNN-LM that trains the retriever and LM jointly to further decrease perplexity.
The ANN-LM a pre-trained lan- guage model that uses retrieval to improve word prediction. We follow the procedure from Khan- delwal et al. (2020) and Alon et al. (2022) 2, and use the LM to encode token-level representations from document collection (e.g., WikiText-103 training data) into datastore where each token in document converted into key-value pair: a context vector ky representing the first n words and a value & which the n-th word. During eval- uation, the model calculates Fuclidean distances d(k, qj) between the query vector qj and all the keys ki, kiy] in the datastore. The values
ANN-LM:
The kNN-LM effectively scales retrieval to bil- lions of tokens using a token-level nor-parametric interpolation technique first introduced by Grave et al. (2017). Alternatiee retrieval-augiee augmented mod- els experiment with training the retriever (Zhong al., 2022: Ram et al., 2023; Shi al., 2023), interpolating vectors instead of token probabilities (Yogatama et al., 2021), scaling to trillions of to- kens (Borgeaud et al., 2021), exploiting retrieval for strong few-shot learning (Izacard et al., 2022), and so on (Chen et al., 2017; Guu et al., 2020; Lewis et al., 2020; Izacard and Grave, 2021; Rae
Altemative architecture options for İNN-LM are explored in Xu et al. (2023). We don't 't expect those settings to impact the trends we observe, but as we mention in 86, tuning for text generation could be beneficial.
'The kNN-LM is also evaluated using MAUVE in Lan et al. (2023); however, our work has much more extensive analysis in the open-ended text generation setting.
2

---

from the retrieved documents define new distri- bution of the next word:
validation of WikiText-103.
3.3 Automatic evaluation metries
For all models tested, we compare the quality of text generated by the bascline LM with and without the k-NN retrieval component over the external datastore. We measure quality via the following automatic metries:
PRNN(wElqe) & Σ Lwi= exp(-d(ki, (u==sexp(dd((;.pp 9:))
(b(,05)
The model interpolates the LM's predicted distribu- tion over the next token P(wylo:) with the retrieval distribution with tunable hyperparameter A:
MAUVE: MAUVE is an evaluation metric for open-ended text generation (Pillutla et al., 2021) that achieves high correlation with human judg- ments of text quality. It measures the distribution similarity between the generated text and the refer- ence text. Higher MAUVE scores indicate closer distance between the distribution of the generated text and that of reference text.
P' (wilg) (
(wrl)) (1)
To generate text from the ANN-LM, we apply decoding strategy (e.g., greedy decoding or trun- cated sampling algorithms) using the final interpo- lated probability distribution P' P(welg)) (welge).
RankGen: Given a prefix and several possible continuations (suffixes), RankGen (Krishna et al., 2022) outputs a score for each suffix, measuring the relevance between the prefix and suffix. Higher RankGen scores indicate stronger relevance be- tween generated suffix with the given prefix. We thus measure the RankGen score between prefix and generated suffix for each of the two models.
Note that in ANN-LM, the LM is trained without retrieval; the retrieval component is bolted on after training. Zhong et al. (2022) note that this approach suboptimal, as the LM docs not understand how to best use the retrieval. Thus, they propose TRIME, which uses an efficient in-batch strategy to incorporate retrievals during training. While ÀNN-LM relies on just one type of retrieval (from external datastore), TRIME can retrieve from local, long-range, as well as external context, We use the TRIMEEXT configuration in all of our experiments, which also uses a linear interpolation between LM and retrieval distributions (as in Equa- tion 1) to produce the final probability distribution. The baseline LM (no external retrieval) retrieves from example-level local and long context, but has access to huge-scale external datastore.
TRIME:
GPT-3 perplexity: We use GPT-3 (Brown et al., 2020),² 5 a large-scale pre-trained language model, to compute the perplexity of text generated with and without interpolation conditioned on the same prefix. Lower GPT-3 perplexity indicates stronger relevance between prefix and generated suffix and the better fluency of the generated suffix.
Entity-F1: Previous works (Nan et al., 2021; Lee et al., 2022) use the percentage of hallucinated named entities (entities that appear in the generated text but not in the reference text) or the ratio of named entity overlaps between the generated text and reference text estimate the factuality of the generated text. In our work, we compute the FI scores between the named entities from the gener- ated text and reference text as a proxy for entity hallucination. Higher Fll scores may correlate to fewer instances of hallucinated entities.
3.2 Constructing an evaluation dataset
We sample from WikiText-103 (Merity al., 2016) to construct our main evaluation dataset; Sec- tion 4, we also perform an analysis experiment on the PG-19 dataset (fictional books) to test whether our findings hold across domains. We choose 103 because it is the most com- monly used dataset for evaluating interpolation- based LMs; indeed, the main experiments from both ÀNN-LM and TRIME demonstrate that the re- trieval component decreases held-out perplexity on this dataset compared to the baseline LM. Specifi- cally, we randomly sample 5K examples from the
Seq-Rep-1:
We follow Welleck al. (2020) and use the percentage of unique unigrams (Seq-Rep-)
*We use the first 128 tokens of each example as a prefix the model must condition on lo generate 256-token-lowd continuation. As some of our metrics requires reference text. we also store the ground follow the prefix each example.
WW choose 5K examples because this is the minimum recommended number of generations to obtain meaningful comparisons as per Pillutla et al. (2021).
tokens (gold suffix) that
'We use the 6.7B gpt3 spt3-curie model via OpenAl's API
3

---

in the text as metric for lexical diversity in the text. Higher Seq-Rep-l scores indicate lower diversity (more repetition) in the generated text.
Model
MAUNET PPLorral RankGent Ently
SeqReps J
ANV-LM
retrieval from (2022)
GPT-2 small (no retrieval)
77.7
13.1
11.7
14.2
56.7
3.4 Model configurations and hyperparameters
GPT-2 small (+ retrievall)
79.2
14.8
11.7
13.1
53.3
RNV LM (Khandefwal et at., 2020) with and without external retrieval
this work, we leverage pretrained model and datastore checkpoints released by prior work, and also train our own interpolation-base LMs.
Transformer (no retrieval)
89.5
20.4
12.9
12.1
41.8
Transformer
90.7
28.9
12.5
9.77
37.9
(+ retrieval)
Baseline LM details: For kNN-LM, we use the implementations from Alon et al. (2022) and Khan- delwal et al. (2020). The model in Alon et al. (2022) relies on a backbone -17M--pramettrr GPT- 2 small model (Radford et al., 2019) fine-tuned on the WikiText- 103 training data. The external data- store is constructed by the same backbone model, and both the pretrained LM and datastore are pub- licly released by Alon et al. (2022).7 We also test the model in Khandelwal al. (2020), which pro- poses the first A-N--LM. Khandelwal al. (2020) uses a 247M parameter Transformer LM trained from scratch on WikiText- and the datastore is computed using the trained Transformer LM. For TRIME, we adopt the 247M-parameter TRIME model trained from scratch WikiText-1033 and publicly released by Zhong al. (2022). Our "non- retrieval" baseline is the same model without exter- nal retrieval; in other words, it has access to only the local memory (recent tokens) and long-range memory (in- bin-batch tokens). In all three set-ups, the external datastore is constructed using the train- ing dataset of WikiText-103; the datastores from Zhong et al. (2022) and Khandelwal et al. (2020) both have 103M entries, while the datastore from Alon et al. (2022) has 117M entries (the discrep- ancy is due tokenization differences between the models).
TRIMERT with and without external retrieval from Zhong et al. (2022)
TRIME
90.6
22.2
13.1
11.3
40.1
(no retrieval)
TRIME 87.3
23.8
12.5
9.80
38.5
(+ retrieval)
Table 1: Automatic evaluation metrics do not show consistent improvement in generation quality for interpolation-basee LMs compared to their non-retrieval baseline LMs. We evaluate three set-ups: 1) &NN-LM with GPT2 the baseline (top), 2) the original ANN- LM proposed in (Khandelwal et al., 2020) which trains Transformer LM from scratch on the WikiText- 103 training data (middle), and 3) TRIME which trains both the LM and the retrieval mechanism (bottom).
perplexity from 17.0 (no retrieval) to 15.5 (-1.5) after interpolation.
Hy Hyperparameters: To generate text, we use the hyperparameters recommended by the authors that yield low perplexities on the WikiText- 103 vali- dation set. For the model in Alon et al. (2022) and Khandelwal et al. (2020), the softmax temper- ature is set to 1.0 and the interpolation coefficient between the LM distribution and the retrieval dis- tribution À is set to 0.25. For TRIME(Zhong et al., 2022), the softmax temperature is set to 1.25 and the À is 0.3. For most of our experiments (e.g.. those in Table 1), unless otherwise specified, we use nucleus sampling (Holtzman et al., 2020) with 0.8 for text generation.
Perplexity improvements from retrieval: All models studied in this paper substantially decrease perplexity on WikiText-103's validation when interpolation enabled. For the model in Alon et al. (2022), the base GPT-2 perplexity is 14.8, and decreases to 12.6 (-2.2) after interpolation. The ANN-LM in (Khandelwal et al., 2020) decreases perplexity from 17.96 (no retrieval) to 16.06 (-1.9) after interpolation. Meanwhile, TRIME decreases
4 Results
We find that despite incorporating the retrieval com- ponent and interpolating information from the base- line LM and retrieval, these methods do not yield any significant improvement to text generation per- formance, and even worsen it by some metrics (Ta- ble 1). In this section, we provide an overview of our main results, perform more fine-grained analy- ses, and deseribe a human evaluation that supports the conclusions drawn from automatic metrics.
"We also compute Seq-Rep-N for = 2.3, 4, and oh- serves consistent results with using Req-Rep-1 (in Appendix A.4).
See the gpt2- gpt2-finetunee -in--funnnttdiiiieee weneed-wwiiiiitttteee text103 model available here: https: //tgt/uttttttt//////////g/ com//nelab//mnn comn-ttranffomeers
4

---

interpolation-basse LMs do not improve auto- matic text generation evaluation metries: We find that none of the three models significantly im- prove generation quality compared to the baseline LM, as shown by various metrics (Table 1). For the model in Alon et al. (2022) (top row in Ta- ble 1), while the MAUVE score improves by 1.5 points with retrieval, the perplexity of GPT-3 in- creases on retrieval-augmeented generations, and the RankGen score is identical. For the model in Khandelwal al. (2020) (middle row in Table 1), retrievals improves the MAUVE score even less significantly (1.2 points) but worsens perplexity of GPT-3, RankGen and Entity-F1. For TRIME (bottom row in Table 1), the non-retrieval baseline is actually slightly better across MAUVE, GPT- 3 perplexity, RankGen and Entity-F1 - In other words, there is no convincing winner; furthermore. contrary the expectation that &NN-LMs reduce hallucination by retrieving (and potentially copy- ing) from the datastore, we do not observe any improvement in the Entity F1 scores with the gold suffix. We observe a marginal improvement in lexi- cal diversity of the generations (shown by the lower Seq-Rep- score S).
Model
Nucleus Sampling
Top-k Sampling
Greedy Decoding
RNN-LM with and without retrieval from Alon (2022)
GPT-2 small (no netrieval)
77.7
87.1
2.32
GPT-2 small (+ retrieval)
79.2
87.5
2.44
Table 2: The observation that KNN-LM does not sig- nificantly improve text generation performance (mea- sured here via MAUVE) is consistent across a variety of decoding algorithms: nucleus sampling. top-k sam- pling (k = 40) and greedy decoding. We note that beam search decoding often generates repetitive text and there- fore scores poorly with MAUVE.
Model
MAUVET Pfl-arrss RankGent EntiyFF1 SegRepi 1
ANN LM with and without retrieval from PG 19 (Rae 2019)
GPT-2 small (no retrieval)
8.00
17.3
4.13
5.63
47.6
GPT-2 small (+ retrieval)
9.26
18.8
3.62
4.87
44.5
Table 3: Consistent with our findings in WikiText-103 dataset, we find in PG-19 (fictional books) that &NN- LM does not yield consistent improvement in text gen- eration quality compared to no-retrieval haseline LMs.
These results hold across different decoding al- gorithms: The results in Table are all from nucleus sampling, What if we change the decoding algorithm? To investigate the impact of decoding algorithm on generation quality, we evaluate the &NN-LM on three different decoding algorithms: greedy decoding, top-k sampling, and beam search. We observe in Table 2 that none of these decod- ing algorithms changes the result: there is no clear winner between models with and without retrieval.
small model fine-tuned on the PG-19 dataset for three epochs (with 28.9 perplexity on the validation dataset). 10 Table shows that on the PG-19 dataset, kNN-LM also does not improve text generation quality. While (marginally) improving perplexity, the &NN-LM often returns unhelpful artifacts from the PG19 dataset (see examples in Appendix A.3).
4.1 Human evaluation
Having found that interpolation- basedd LMs do not notably improve text generation quality according to automatic evaluation metrics, we turn next to hu- man evaluation, which known to be more reliable for generation tasks (Celikyilmaz al., 2020; Kr- ishna et al., 2021), to compare the text generated by the ANN-LM vs. the baseline GPT-2 model from Alon et al. (2022). We hired three English teach- crs/editors on the freelance marketplace Upwork. The evaluation was conducted on the platform La- bel Studio (Tkachenko et al., 2020-2022). The
These results hold across different datasets: In addition to WikiText- 103, we also evaluate the text generation performance of the ANN-LM on the PG-19 dataset (Rae et al., 2020), which predom- inantly comprises fictional books and presents distinct thematic variation Wikipedia. We con- struct an evaluation dataset from PG-19 similarly to our constructed evaluation dataset from WikiText- 103 in Section 3.2.* 9 The baseline LM is GPT2-
"We also report the Req-Rep-N scores for N=2, 3, Appendix A.4
and datastore consist of 98M tokens, similar in size to those the WikiText- 103 dataset.
'From the validation dataset of PG-19, we randomly sam- plc 5K samples, where in each sample, the first 128 tokens is used as the prefix. For datastore construction, we sample 1536 books from the training dataset only (filtering out the first 10% and last 10% tokens of cach books for irrelevant content such as copyright statements). Our training dataset
"Consistent with Drozdov et al. (2022), model trained on PG-19 gives both worse MAUVE score and perplexity compared to the model trained on WikiText-103 since the PG-19 is more diverse and challenging dataset.
"https:/ / //www upwork. con, https: //ssse//labestut io/
5

---

justifications reveals that both KNN-LM and GPT- make catastrophic mistakes. Table gives four examples of bad continuations, along with the eval- uators' comments and our categorization of the errors. In the first row of the table, Continuation A generated by the KNN-LM contains repetitive con- tent (i.e.. ==ZAPU retreal==), and confuses ZAPA and ZIPRA at multiple places. The GPT-2 continua- tion in the second
KNN-UM Unsure
GPT-2 Unsure GPT-2 Confident
KNN LM Confident
30
Count choices
25-
20
13.
16
Rater1
Rater2
Rater3
states that person born in 1584 but was still alive in 1742; the generation in the third row by the ANN-LM claims that U.S. Route 75 curves both northeast and northwest in the northbound direction. Furthermore, both the GPT-2 and KNN-LM's generations change topics abruptly as shown in the lower half of Table 5. Overall, the quantitative and qualitative analyses of the human evaluation results show that the &NN-LM does not clearly improve over its base GPT-2 model despite its significant improvement in perplexity.
Figure 1: The plot presents how many times each type of generations (&NN-LM or GPT-2) is chosen by the evaluators. The dark area each bar shows that the choices made confidently. The light area repre- sents the choices between &NN-LM and GPT-2 that were hard but the evaluator still chose the corresponding type. Overall, annotators preferred GPT-2 baseline texts 51% of the time compared to 49% for ANN-LM.
annotators were experienced in text generation eval- uation and hired after careful selection.
The annotators were given a prefix and two con- tinuations of the context (one generated by the base- line LM and one generated with retrieval, with ran- domized presentation order). The evaluators' task to decide which continuation is better, indicate whether was hard to choose between the two fol- lowing Thai al. (2022), and justify their choice in to 4 sentences." 12 The evaluation focused on whether the generated text is grammatical, fluent, consistent, and logical.
Why do kNN-LMs fail to improve text generation quality?
Our evaluations (both human and automatic) not show a significant quality increase when interpolat- ing an LM's predicted probability distribution with one formed via retrieval over large external datas- tores. In this section, we try to understand why we do not observe an improvement by empirically ana- lyzing the &NN-LM and find potential reasons: (1) despite lowering the aggregate perplexity, kNN- LMs only improve the perplexity of 42% of all test tokens, which suggests that the improved quality of a subset of tokens could be counter-balnne balanced by worsened predictions on other tokens that do not benefit from the ANN-LM. Moreover, we find the entropy of the retrieval distribution to increase at faster rate than that of the baseline LM the model generates longer sequences. This difference implies that the retriever distribution getting nois- ier as more tokens are sampled, potentially due to the exposure bias stemming from the retriever hav- ing to rely on the sampled text as the query.
Human evaluation shows no definitive winner between KNN-LM and GPT-2 either: On aggre- gate, bascline GPT-2 generations were preferred 51% of the time, vs. 19% for kNN-LM. Addition- ally, the three annotators report that the decision difficult for 37% of all cases. For Raterl and Rater3, the rates of difficult to choose are as high 42% and 47% while for Rater2 is 22%. Out of the comparison pairs, the three annotators only agree on their choices in 17 instances (37.7%%), resulting in a Fleiss Kappa score 0.17 (slight agree- ment). Figure presents the evaluator preference when comparing the ANN-LM to GPT-2 genera- tions.
5.1 KNN-LMs only benefits a subset of tokens
Many studies have shown that &-N-LMs decrease perplexity via retrieval interpolation (Khandelwal et al., 2020: Alon et al., 2022; Drozdov al., 2022). Previous work (Drozdov al., 2022: Zhong et al., 2022) has also suggested that &NN-LMs benefit the inference of tokens of various part-of sf-eeehh (POS) tags to different degrees (by lowering the perplexity
Both models make catastrophic errors at similar
rates: A qualitative analysis of the the evaluators'
12 screenshot of our evaluation platform can be found Appendix A.
Each evaluator evaluated 45 pairs of continuations gen- crated by &NN-LM and GPT-2, Fach evaluator was paid $50 for their work.
6

---

221
560%
Radio of "otrieerr ontropy to base LM entropy
A0P
7.7
211
PROOR
MUV
21009
PON
COON
ATL
18
APAIE
320 230 201
232
Figure 3: We plot the ratio between the Shannon en- tropy of the retriever's next-token distribution and that of the baseline LM softmax distribution, as the number of generated tokens increases. The ratio increases for longer model generreee sequences, indicating that the retriever becomes less confident than the baseline LM as decoding progresses.
Figure 2: Across all POS tags, we observe that ANN- LM does not increase the probability of the majority of gold next token predictions. For verbs, pronouns, and adjectives, it only helps < 40% the time (i.e., it hurts the predictions of the majority of these tokens).
5.2 The retriever becomes less reliable with longer generated sequences
of the gold token). However, these works focus on aggregate perplexity averaged across tokens in the test data but do not look individual tokens and the percentage that actually benefit from retrieval.
Additionally, we observe that as the model gener- ates longer sequences of text, the retriever compo- nent from &NN-LM becomes less confident and reliable in returning a high-quality next-token dis- tribution. Since the kNN-LM relies on interpolat- ing the next-token distribution from the baseline LM and that from the retriever, lower quality triever distribution can compromise the resulting next-token distribution and adversely affect the text generation performance.
Using the dataset selected from WikiText- 103, compute the percentage of gold tokens from our test examples that are assigned lower per- plexity (higher probability) by the İNN-LM com- pared to the base LM. We find that only 42% of the tokens benefit from &NN-LMs, while the remain- ing 58% of the tokens are adversely affected by the ÀNN-LM (i.e., the ANN-LM assigns a lower proba- bility to the gold token compared to the base-LM). Morcover, calculate the percentage of gold to- kens that benefit from ÀNN-LM in each POS cate- gory (Figure 2) and consistently find the similar re- sult that &NN-LM only helps reduce the perplexity for a smaller subset of tokens. We show examples of kNN-LM negatively impacting the next-token prediction (assigning the gold token with lower probability than the base-LM) in Table 4.
We plot the ratio of Shannon entropy (Shannon, 2001) between the retriever distribution and that of the baseline LM distribution the next token (with respect the index of the token generated) and find that the retriever's entropy is increasing at faster rate compared to that from the base-LM (Figure 3). A higher entropy indicates lower level confidence (closer to a uniform distribu- tion over all tokens) and suggests that the retriever, when sampling long sequences, may be less reli- able in identifying the high-quality tokens.
We hypothesize that the worsened reliability of the retriever over longer sampled sequences is likely a result of the exposure bias during text generation (i.e., at test-time, the retriever has to rely on model-generated queries that may contain artifacts or other distributional differences from human- human-written written text). The retriever in &NN-LM
This means that despite lowering the aggregate perplexity across the test sets, the ANN-LM is more likely to hurt, instead of help, the inference of each individual token. Therefore, during text generation, as the model samples se- quence of tokens, the advantages brought by kNN- LM to a smaller subset of tokens are offset by other tokens, for which ANN-LM may even have a detri- mental impact on the inference.
hypothesize that
'Given Wiimeeensio probability distribution p. the entropy is computed as: H (p) = Σ P. log(p.)
7

---

MAIVE against the Interpolation Confficient Lambda
fix dt, they may be less likely recover, as the retrievals may further increase the probability of completions that resemble those artifacts. Further- more, as we continuously sample and append to- kens to the prefix, which the retriever uses as the query to construct PKNN (wu (wylg:). the retriever may encounter additional exposure bias shown in Section 5.2, negatively impacting the quality of PKNN (og (.:) Thus, even when the baseline LMS "recover" from distorted past context by producing high-quality distribution over the next-token pre- diction PLM (welee). ) the retriever may re-introduce the distortion by interpolating PLM PLo((welp) (w/]g) with PKNN (oe,((w,le (t).
130
Senala -922
155
11.
14.0-
113
13.
113
30
04
0.
0.5
Figure 4: The interpolation coefficient optimized for validation perplexity does not necessarily lead the best text generation quality (measured by MAUVE).
Hyperparameters introduced by &NN-LM not optimized for text generation: The NNN- LM introduces two important hyperparameters, namely the relative weight between the two dis- tribution X, as well as softmax temperature for the KNN distribution TKNN. Recent work (Xu et al.. 2023) highlights the significance of tuming TKNN for achieving optimal kNN-LM performance, measured by perplexity. Similarly, we investigate the coefficient parameter X, which plays vital role as controls the relative importance assigned the kNN retriever and baseline LM. Existing works tune À by the perplexity on the validation set. How- ever, from Figure 4, we observe that the values that produce the lowest perplexity may not trans- late to the optimal value for text generation quality (measured by MAUVE). Instead of tuning À for optimizing perplexity, we may want to consider context-dependent À as in Drozdov et al. (2022) for generation (e.g., only use the retrieval distribution when it is very confident). Finally, interpolation may warrant the design of new decoding algorithms specialized for retrieval-augmae -augmented generation.
is non-parametric since both the input prefix and the context from the datastore are encoded by the LM (without any additional retrieval parameters), which has been adapted to the training corpus of WikiText- 103. However, during text generation, as the model iteratively samples tokens and appends them to the imput prefix, the input context more likely to deviate from those in the training corpus, hence, becomes more nut-of--dstribbnn and chal- lenging for the retriever to accurately process.
6 Discussion
In addition the limitations of interpolation-baase LMs described in Section 5, we hypothesize that there are other potential factors that contribute to the shortcomings of the ANN-LM for text gener- ation. Specifically, it is possible that the interpo- lation may impede the language models' ability for self-recovery, and also that integrating the re- trieval distribution can potentially introduce addi- tional burdens related hyperparameter tuning, which may not be optimized for text generation. We discuss these potential issues here as they are interesting avenues to explore for future work.
7 Conclusion
In this work, we show that despite the significant perplexity improvement brought by interpolation- based retrieval-augmenete LMs such as kNN-LMs, such methods fail to improve the LMs' text gen- eration performance. The text generation quality between ANN-LMs and baseline LMs without re- trieval show no significant difference according to both automatic text generation evaluation metrics and human evaluation. Upon closer analysis, we identify flaws in using &NN--Ms to perform autore- gressive text generation: the method only benefits a minority of token predictions, and the retriever's
Retrieval interpolation may damage the self-
recovery ability of LMs: Language models ex- hibit some degree of self-recovery abilities (He et al., 2021b), i.e., they can regain fluency and coherence even after previously generating poor- quality tokens. This self-recovery capability is attributed the LM's ability to pay close atten- tion to recent context and ignore the long-range past context. However, we hypothesize that when interpolation- based LMs encounter artifacts (c.g., non-factual or disfluent text) in a distorted pre-
8

---

quality deteriorates when generating long-form text. We hope our findings can inspire future research to design better training and inference methods so that the impressive improvement of KNN-LMs in perplexity can better be translated into gains in text generation quality.
Science Foundation (NSF). This research was also supported in part by a research gift from Adobe.
References
Uri Alon, Frank Xu, Junxian He, Sudipta Sengupta. Dan Roth, and Graham Neubig. 2022. Neuro-symbolic language modeling with automaton-auumenned re- trieval. International Conference on Machine Learning, pages 466.-885.P0 PMLR.
Ethics Statement
In this work, we investigate the text generation quality of language models. Language models can generate text that is harmful, offensive or un- faithful. We advise using caution when relying on language models to generate text and adopting post-processing strategies on the language-model generated text to remove undesirable content. Ad- ditionally, training large language models can bring significant energy cost. We hope that our analysis of the &NN--M and future works on this topic may lead to more efficient method of using language models without the need to re-train such models.
Soumya Sankar Basu, Ankit Singh Rawat, and Manzil Zaheer. 2022. Generalization properties retrieval- based models. ArXiv, abs/2210.02617.
Sebastian Borgeaud, Arthur Mensch, Jordan Hoffmann, Trevor Cai, Eliza Rutherford, Katie Millicam, George van den Driessche, Jean- -aatttap Lespiau, Bogdan Damoc, Aidan Clark, Diego de Las Casas, Aurelia Guy, Jacob Menick, Roman Ring, T. W. Hennigan, Saffron Huang, Lorenzo Maggiore, Chris Jones, Al- bin Cassirer, Andy Brock, Michela Paganini, Geof- frey Irving, Oriol Vinyals, Simon Osindero, Karen Simonyan, Jack W. Rae, Erich Elsen, and L. Sifre. 2021. Improving language models by retrieving from trillions of tokens. International Conference on Machine Learning.
Limitations
Our work does not study all data, model, and eval- uation configurations of interpolation-based LMs. Additionally, we focus on the 100M token datas- tore size, although KNN-LM can scale effectively to datastores of 3B words. Using a larger datas- tore may lead to further perplexity decreases, but we do not think this contradicts our finding that text generation degrades retrieval quality does. We focus exclusively on interpolation-base LMs in this work, but similar issues for other retrieval- augmented LMs such as RETRO (Borgeaud et al., 2021) may also exist and be worth investigating further. Finally, our human evaluation does not specifically account for diversity, although some dimensions of this are captured by our automated metrics. Due to the overall low quality of text gen- erated by LMs with and without retrieval, reading their outputs results in high cognitive burden on annotators, which might be ameliorated by using stronger LMs than GPT-2
Tom Brown, Benjamin Mann, Nick Ryder, Melanie Subbiah, Jared D Kaplan, Prafulla Dhariwal, Arvind Neelakantan, Pranav Shyam, Girish Sastry, Amanda Askell, Sandhini Agarwal, Ariel Herbert- Gretchen Krueger, Tom Henighan, Rewon Child, Aditya Ramesh, Daniel Ziegler, Jeffrey Wu. Clemens Winter, Chris Hesse, Mark Chen, Eric Sigler, Ma- teusz Litwin, Scott Gray, Benjamin Chess, Jack Clark, Christopher Berner, Sam McCandlish, Alec Radford, Tlya Sutskever, and Dario Amodei. 2020. Language models are few-shot learners. In Ad- vances in Neural Information Processing Systems. volume 33, pages 1877-1901. Curran Associates, Inc.
Asli Celikyilmaz, Elizabeth Clark, and Jianfeng Gao. 2020. Evaluation of text generation: A survey, arXiv preprint arXiv:2006. 14799.
Danqi Chen, Adam Fisch, Jason Weston, and Antoine Bordes. 2017. Reading Wikipedia to answer open- domain questions. In Proceedings of the 55th Annual Meeting of the Association for Computational Lin- guistics (Volume Long Papers). pages 1870-1879. Vancouver, Canada. Association for Computational Linguistics.
Acknowledgements
We thank Zexuan Zhong and Danqi Chen for help- ful discussion TRIME and ANN-LM, and the UMass NLP group for feedback and discussion. We also thank the anonymous reviewers for their helpful comments.
Andrew Drozdov. Shufan Wang. Razieh Rahimi, An- drew McCallum, Hamed Zamani, and Mohit lyyer, 2022. You can't 't pick your neighbors, when and how to rely on retrieval in the kNN-LM. In Findings of the Association for Computational Linguisties: EMNLP 2022, pages 2997-3007, Abu Dhabi, United Arab Emirates. Association for Com- putational Linguistics.
can you?
This project was partially supported by awards IIS-2202506 and IIS- 20--204628 from the National

---

Edouard Grave, Armand Joulin, and Nicolas Usunier. 2017. Improving neural language models with a continuous cache. In International Conference on Learning Representations.
Urvashi Khandelwal. Omer Levy, Dan Jurafsky, Luke Zettlemoyer, and Mike Lewis. 2020. Generalization through Memorization: Nearest Neighbor Language Models. In International Conference on Learning Representations (ICLR).
Kelvin Guu, Kenton Lee, Zora Tung, Panupong Pasupat, and Ming- Wei Chang. 2020. REALM: Retrieval- augmented language model pre-training. In Interna- tional Conference on Machine Learning.
Kalpesh Krishna, Erin Bransom, Bailey Kuehl, Mohit lyyer. Pradeep Dasigi, Arman Cohan, and Kyle Lo. 2023. Longeval: Guidelines for human evaluation of faithfulness in long-form summarization. In Confer- ence of the European Chapter of the Association for Computational Linguisties.
Tatsunori B. Hashimoto, Hugh Zhang, and Perey Liang. 2019. Unifying human and statistical evaluation for natural language generation. In Proceedings of the 2019 Conference of the North American Chapter of the Association for Computational Linguistics: Hu- man Language Technologies, Volume 1 (Long and Short Papers), pages 1689-1701, Minneapolis, Min- nesota. Association for Computational Linguistics.
Kalpesh Krishna. Yapei Chang, John Wieting, and Mo- hit lyyer. 2022. RankGen: Improving text gener- ation with large ranking models. In Proceedings of the 2022 Conference on Empirical Methods in Natural Language Processing, pages 199-232, Abu Dhabi, United Arab Emirates. Association for Com- putational Linguistics.
Hangfeng He, Hongming Zhang, and Dan Roth. 2022. Rethinking with retrieval: Faithful large language model inference. ArXiv, 0bs/2301.00303
Kalpesh Krishna, Aurko Roy, and Mohit lyyer. 2021. Hurdles to progress in long-form question answering. In Proceedings of the 2021 Conference of the North American Chapter of the Association for Computa- tional Linguistics: Human Language Technologies, pages 4940-49577, Online. Association for Computa- tional Linguistics.
Junxian He, Graham Neubig, and Taylor Berg- Kirkpatrick. 2021a. Efficient nearest neighbor lan- guage models. In Proceedings of the 2021 Confer- ence on Empirical Methods in Natural Language Pro- cessing, pages 5703-5714, Online and Punta Cana. Dominican Republic. Association for Computational Linguistics.
Tian Lan, Deng Cai, Yan Wang, Heyan Huang, and Xian- Mao. 2023. Copy is all you need. In The Eleventh International Conference on Learning Representations.
Tianxing He, Jingzhao Zhang. Zhiming Zhou, and James Glass. 2021b. Exposure bias versus self- recovery: Are distortions really incremental for au- toregressive text generation? In Proceedings of the 2021 Conference on Empirical Methods in Natural Language Processing, pages 5087-5102, Online and Punta Cana, Dominican Republic. Association for Computational Linguistics.
Nayeon Lee, Wei Ping, Peng Xu, Mostofa Patwary, Pas- cale Fung, Mohammad Shoeybi, and Bryan Catan- zaro. 2022. Factuality enhanced language models for open-ended text generation. In Advances in Neural Information Processing Systems.
Patrick Lewis, Ethan Perez, Aleksandra Piktus, Fabio Petroni, Vladimir Karpukhin, Naman Goyal, Hein- rich Küttler, Mike Lewis, Wen-tau Yih, Tim Rock- täschel, Sebastian Riedel, and Douwe Kicla. 2020. ketrieval-augmennt generation for knowledge- intensive nip tasks. In Advances in Neural Infor- mation Processing Systems, volume 33, pages 9459- 9474. Curran Associates, Inc.
Ari Holtzman, Jan Buys, Li Du, Maxwell Forbes, and Yejin Choi. 2020. The curious case of neural text de- generation. In International Conference Learning Representations.
Gautier Izacard and Edouard Grave. 2021. Leveraging passage retrieval with generative models for open do- main question answering. In Proceedings of the 16th Conference of the European Chapter of the Associ- ation for Computational Linguistics: Main Volume, pages
Alex Mallen, Akari Asai, Victor Zhong, Dajarshi Das, Hannaneh Hajishirzi, and Daniel Khashabi. 2023. When not to trust language models: Investigating effectiveness and limitations of parametric and non- parametric memories. In ACL.
Online. Association for Computa- tional Linguistics.
Gautier Izacard, Patrick Lewis, Maria Lomeli, Lucas Hosseini, Fabio Petroni, Timo Schick, Jane Dwivedi- Yu, Armand Joulin, Sebastian Riedel, and Edouard Grave. 2022. Few-shot Learning with Retrieval Aug- mented Language Models.
R. Thomas McCoy, Paul Smolensky, Tal Linzen, Jian- feng Gao, and Asli Celikyilmaz. 2021. How much do language models copy from their training data? evaluating linguistic novelty in text generation using raven. ArXiv, abs/211 1.09509.
Urvashi Khandelwal, Angela Fan, Dan Jurafsky, Luke Zettlemoyer, and Mike Lewis. 2021. Nearest neigh- bor machine translation. International Conference on Learning Representations (ICLR).
Stephen Merity, Caiming Xiong, James Bradbury, and Richard Socher. 2016. Pointer sentinel mixture mod- els.
10

---

Don Metzler, Fernando Diaz, Hamed Zamani, Mike Bendersky, and Mostafa Dehghani. 2022. Retrieval enhanced machine learning. In SIGIR 2022: Pro- ceedings of the 45th International ACM SIGIR Con- ference on Research and Development in Information Retrieval (Perspectives Track).
Daniel Toyama, Cyprien de Masson d' Autume, Yujia Li, Tayfun Terzi, Vladimir Mikulik, Igor Babuschkin. Aidan Clark, Diego de Las Casas, Aurelia Guy, Chris Jones, James Bradbury, Matthew G. Johnson, Blake A. Hechtman, Laura Weidinger, lason Gabriel, William S. Isaac, Edward Lockhart, Simon Osindero. Laura Rimell, Chris Dyer, Oriol Vinyals, Kareem W. Ayoub, Jeff Stanway, L. L. Bennett, Demis Hass- abis, Koray Kavukcuoglu, and Geoffrey Irving. 2021. Scaling language models: Methods, analysis & in- sights from training gopher. ArXiv, abs/21 11/22112.111 1146..
Grégoire Mialon, Roberto Dessi, Maria Lomeli, Christo- foros Nalmpantis, Ramakanth Pasunuru, Roberta Raileanu, Baptiste Rozière, Timo Schick, Jane Dwivedi- Yu, Asli Celikyilmaz. Edouard Grave, Yann LeCun, and Thomas Scialom. 2023. Augmented language models: a survey. ArXiv, abs/2.00.07842.
Jack W Rae, Anna Potapenko, Siddhant M Jayakumar, Chloe Hillier, and Timothy P Lillicrap. 2019. Com- pressive transformers for long-range sequence mod- elling. arXiv preprint.
Feng Nan, Ramesh Nallapati, Zhiguo Wang, Cicero Nogueira dos Santos, Henghui Zhu, Dejiao Zhang, Kathleen McKeown, and Bing Xiang. 2021. Entity- level factual consistency of abstractive text summa- rization. In Proceedings of the 16th Conference of the European Chapter of the Association for Compu- tational Linguistics: Main Volume, pages 2727-2733, Online. Association for Computational Linguisties.
Jack W. Rae, Anna Potapenko, Siddhant M. Jayaku- mar, Chloe Hillier, and Timothy P. Lillicrap. 2020. Compressive transformers for long-range sequence modelling. In International Conference on Learning Representations.
Jekaterina Novikova, Ondiej Dušek, Amanda Cer- cas Curry, and Verena Rieser. 2017. Why we need new evaluation metrics for NLG. In Proceedings of the 2017 Conference on Empirical Methods in Natu- ral Language Processing, pages 2241-2253 Copen- hagen, Denmark. Association for Computational Lin- guistics.
Ori Ram, Yoav Levine, Itay Dalmedigos, Dor Muhlgay, Amnon Shashua, Kevin Leyton-Brown and Yoav Shoham. 2023. In-context reureneal-uggmentedd lan- guage models.
Claude Elwood Shannon. 2001. A mathematical the- ory of communication. ACM SIGMOBILE mobile computing and communications review, 5(1):3-55.
Krishna Pillutla, Swabha Swayamdipta, Rowan Zellers, John Thickstun, Sean Welleck, Yejin Choi, and Zaid Harchaoui. 2021. Mauve: Mcasuring the gap be- tween neural text and human text using divergence frontiers. In Neural Information Processing Systems.
Weijia Shi, Sewon Min, Michihiro Yasunaga, Minjoon Seo, Rich James, Mike Lewis, Luke Zettlemoyer, and Wen tau Yih. 2023. Replug: Retrieval-augmmeneed black- box language models. ArXiv, abs/2301.12652
Tiago Pimentel, Clara Isabel Meister, and Ryan Cot- terell. 2023. On the usefulness of embeddings. clus- ters and strings for text generation evaluation. In The Eleventh International Conference on Learning Representations.
Katherine Thai, Marzena Karpinska, Kalpesh Krishna, Bill Ray, Moira Inghilleri, John Wicting, and Mohit Tyyer. 2022. Exploring document-level literary ma- chine translation with parallel paragraphs from world literature. In Proceedings of the 2022 Conference on Empirical Methods in Natural Language Processing. pages 9882-9902, Abu Dhabi, United Arab Emirates. Association for Computational Linguistics.
Alec Radford, Jeff Wu, Rewon Child, David Luan, Dario Amodei, and Ilya Sutskever. 2019. Language models are unsupervised multitask learners.
Jack W. Rae, Sebastian Borgeaud, Trevor Cai, Katie Millican, Jordan Hoffmann, Francis Song, John Aslanides, Sarah Henderson, Roman Ring, Susan- nah Young. Eliza Rutherford, Tom Hennigan, Ja- cob Menick, Albin Cassirer, Richard Powell, George van den Driessche, Lisa Anne Hendricks, Mari- beth Rauh, Po-Sen Huang, Amelia Glaese, Johannes Welbl, Sumanth Dathathri, Saffron Huang, Jonathan Uesato, John F. J. Mellor, Irina Higgins, Antonia Creswell, Nathan McAleese, Amy Wu, Erich Elsen, Siddhant M. Jayakumar, Elena Buchatskaya, David Budden, Esme Sutherland, Karen Simonyan, Michela Paganini, L. Sifre. Lena Martens, Xiang Lorraine Li, Adhiguna Kuncoro, Aida Nematzadeh, Elena Grihovskaya, Domenic Donato, Angeliki Lazaridou, Arthur Mensch, Jean- Baptiste Lespiau, Maria Tsim- poukelli, N. K. Grigorev, Doug Fritz, Thibault Sotti- aux, Mantas Pajarskas, Tobias Pohlen, Zhitao Gong,
Maxim Tkachenko, Mikhail Malyuk, Andrey Holmanyuk, and Nikolai Liubimov. 2020-
2022. Label Studio: Data labeling soft- ware. Open source software available from https//ggtub..ommhhaaetttbbbbbbt
H. Trivedi, Niranjan Balasubramanian, Tushar Khot, and Ashish Sabharwal. 2022. Interleaving retrieval with tf-th-of-thought reasoning for inowledge-ittensive multi-step questions. ArXiv, Ahs/2212. 10509.
Sean Welleck, Ilia Kulikov, Stephen Roller, Emily Di- nan, Kyunghyun Cho, and Jason Weston. 2020. Neu- ral text generation with unlikelihood training. In 8th International Conference on Learning Represen- tations, ICLR 2020, Addis Ababa, Ethiopia, April 26-30, 2020.

---

Yuhuai Wu, Markus Norman Rabe, DeLesley Hutchins, and Christian Szegedy, 2022. Memorizing transform- In International Conference on Learning Repre- sentations.
Frank Xu, Uni Alon, and Graham Neubig. 2023. Why do nearest neighbor language models work? ArXiv, abs/230110222 002828.
Dani Yogatama, Cyprien de Masson d' A'ttmmme and Lingpeng Kong. 2021. Adaptive semiparametric guage models. Transactions of the Association for Computational Linguistics, 9:362-373.
Zexuan Zhong, Tao Lei, and Danqi Chen. 2022. Train- ing language models with memory augmentation. In Conference on Empirical Methods Natural Lan- guage Processing.
A Appendix
A.1 Examples of &NN--M hurting the inference of the next-token
We show examples where ANN-LM hurts the infer- ence of the next-token in Table
A.2 Human evaluation interface and examples
From our human evaluation, we show the interface for our evaluators Fig and also selected rep- resentative examples of evaluators' comments in Table 5.
A.3 Models trained on PG-19 produce unhelpful artifacts
With retrieval from the datastore, the ANN-LM improves the perplexity the validation dataset of the PG-19 marginally from 28.9 to 28.2 but does not improve the text generation quality. Both the baseline LM and the kNN-LM fine-tuned on the PG-19 dataset returns artifacts from the dataset (e.g. missing white-spaces and unnecessary line breaks), shown in Table 6.
A.4 Req-Rep-N of generated text from the baseline-LM and KNN-LM
Even though &NN-LM does not improve the text generation quality overall, observe improve- ment in lexical diversity (lower Seq-Rep-N) from &NN-LM on the WikiText- 103 dataset in Table 7. However, this improvement text diversity is obtained at the cost Entity-F1 (a proxy for factu- ality).
12

---

Context
Ground-truth
Most Prohable Tokeess from
Analysis
The inspired story way a great roll concert should he place where dream the barriers and the divisions between
"people"
buse-LM:
this example the buse-LM predicts ground
"people" with the highest probubility of all tokens (0.26). However, after interpolating with retrieval distribution, RNN-LM decreuses the probubility of the ground- truth token.
Baxe-LM
prubability:
"he (0.20).
"us" (0.09),
0.26
art" (0.03), (rock'(0.02)
everyone logether.... Maybe that
RNN-LM probability:
RNN-LM
0.23
the (0.23),
"us" (0.07)
good (0.02) (ant(0002)
Richmond finished the 1984 season 12th with II In the Busch Series, he qualified in the two moss he entered and won Richmond joined Matorsports 1986, where he teamed up with veteran crew chief Harry Hyde. It took the team until the middle season'
bave-MM:
The ground-ruuth token be predicted is preposilion "the", which the Base- LM correclly predicts with very high probahility.
pole position
Base-T.M
probability:
(0.07),
"far (0.033,
"when (0.02), 'that" (0.02)
the Charlotte
0.78
RNN-LM
probability:
0.64
(0.02(0002)
10), (0.06, (,700,))
creases the probability of the groud-ruth token significantly compured the base: LM..
Table Examples where ANN-LM hurts the inference of next-token (with different part-of-speech such as noun and preposition) by predicting of the gold token with lower probability compared the base-LM
Cannees The Natvee anlled woth quittad oo Bllier ba fiild hosk wiih
Ovange
trmikinig (doiig cira peoke Natios
bodkos
invaalie morke
take cvenig withe tve a aith ao and whice Ellin did maagge caees
3
odea de ahowe seriiatty
rade coler
and Taae -oct
pecite lo cort wo comee Anis botte The sen deserci
rosero Tame Smake qrart
Contiaativ
Canti amann @
Will difican decide betwees the heo undinaation boca quality (e they divilarly good bady!
erplraatioe
traitaate these Destrodee ompuue
Figure 5: The interface of the human evaluation. Each task consists of context text, two continuations, two choices, and a free-form justification text box.
13

---

Context
Continuation
Comment
Error
[...] == = Shooldowa = group ZIPRA guer- rillas, armed with a Strela launcher, waited in the bush beneath Flight 825's ss flighptth, and
Continuation A A-INN-LM two sol- diers opened fire the guerillass [...] The ZIPRA and ZIPRA units exchanged fire, and several ZIPRA members shot. Afler the fire was over. the ZIPRA and ZIPRA guerrillas returned fire and began shooting back, and of ZIPRA unsuccessfully attacked ZAPU Flight 825, forcing = ZAPU re- treal se===== ZAPU retreat
Both of these contimuations have problems. I chose A because its problems seems to influence the meaning less than the problems in [...] Contimation A has some repeated words and in few spots ZIPRA and ZAPU seem to get mixed up, leading to sentences like ZIPRA moved up and captured several ZIPRA IA mem-- bets, forcing them willdrraw." was easier to recover from those con- fusions than from the ones Con- timuation B.
Repeciive content; con- tradiction
base the
the major to retreal.
[...] The ZIPRA moved up the hill captured several ZIPRA members, forcing them to withdraw. [...]
was in cunks- County, Anhui Province in 1584 or early 1585. [...] By 1619, had moved to Nanjing where he lived with wife Wu.
GPT-2 During this time he also became leading scholar of the court, position he held until 1624, [...]. His last year at the academy was 1715. In 1725, Hu appointed secretary of the goverament J.. In 1727, he was appointed as vice-president of the Academy, and he remained the academy's secretary until 1742, [...].
Continuation
selected that this was difficult de- cision [-.]] However, Continuation an impossible choice it Hu heing born in 1584 and still be- ing alive 1725 which of course I was left with process of elimination my opinion.
Illogical: wrong chrono- logical details
The lanes of US 75, which are on the bottom level, are 20 feet (6m) below ground level, giv- height of 140 leet from bottom to top part of project, I-635 was widened to include four dedicated high occupancy vehi- cle lanes (HOV lanes) that are barrier-se separated and reversible. The roads on five levels are: Level I: US 75, an eight lane highway, four lanes going each way; Level II; The junction of two six-lane froniage roads, each having three through
Continuation A-ENN-LM traffic lanes with one lane in each direction. Level III: In the northbound direction, US curves to northeast and US curves to the northowest to follow more southerly routing. The northbound lanes are two lanes in each direction, while the southoound lanes two lanes. [...] = = Major intersections = = [..-] = Battle of the Kalka River = The Battle of the Kalka River (Polish : Kalka, pronounced ['kao,, or ('kao maol) was decisive hattle centuries of the 15th century in Austrian occupied Poland.
Continuation A gives more infor- mation and is more specific than It has some lack of clarity Level has
Abrupe topic change; il- logical
ing the structure
curving both neeth- cast and northwest- - but overall it's more informative than just "the tersection of two frontage roads." I would have ruled out A hecause of the Battle the Kalka River section the bollom. It doesa't really have a place this text. But also has the Battle of the Falkland Islands, so wash. A
15th and 16th
bottle was lought in the middle of the Russo
(Same as ahove)
Continuation B GPTT-2 traffic lanes. Level III: : The intersection of the two six- frontage ronds, each having four through traffic lanes. Level IV: The in- tersection of the two six- late frontage road, each having twn through traffic lanes. Level The intersection two six lane fronlage roads, each having through traffic lanes
(Samee as ahove)
Ahrupt topic change: sepetitive content
Construc-
tion = = = The project was approved Dacem-
in Jnly 1997 and completed her 17, 1997, with the grondbreaking ceremony taking place the State Uni- versity of New York (SUNY) campus in New York City. [...] = Major intersections [...] -
Falkland Islands
Table 5: Selected representative examples with evaluators' comments. The error categories in the last column are ours. Contents are shortened for the sake of space.
Context
baseline- LM ----ggeertted text
-Ng-LLM-geeerated text
my brain, and had been working since. what was going on within me. Idlid not need be told about his meledy. knew that he was doing jtsith the greatest energy and energy and energy and energy.
brain, and had been working, like thevthers, that direction some years. It was this which prompted me jitry for musä- cal conception. wanted be able make music fasuit the monds and contracting cir- cuntstances of the momcer.
For my port.something within me unswered Kandinsky's is an the first time I met with it. There was no question of looking for representation; hamonny had been
why was
Kundincky
Before going taMaia they insist on doing the honors, and showing the objectsinterest the vicin- So they take their barrack, large- house, thence the frout To the lotter spot our crachmandriving
the family out house, the sameprocess with them. Our next station
gu through
the family out the hoesplains the advantages of boarding-school in the United States, aswell of bearding, schools in lrunce and Spuin. of thisexception, and foel that there is a differ- once the methods of bourdingschoel' in the two countries.
with chockle,
the hear Sun Maseo, where theyhave fiousckith room on the outside, and and affong and ceiling.
reminded
Table 6: Both the baseline LM and 6NN-LM generate text that consists of artifacts from the dataset, e.g. missing white-spaces between tokens
14

---

Model
Seq-Rep-1
Seq-Rep-2
Seq-Rep-3
Seq-Rep-4
KAN--E ait and milhow onsiivl from Ales al. (2022)
GPT-2 small
(no retrieval)
56.7
26.6
15.1
9.63
GPT-2 small (+ netrivall
53.3
22.5
11.6
6.73
Table Even though &NN-LM does not improve the overall text generation quality, we observe higher lexical diversity (lower Seq-Rep-N) in the &NN-LM- 2NN-LM-generated text, the WikiText- 103 dataset, using the GPT2-small model as the baseline LM.
15


Reviews:
[{"review_id": "XH91VOMyZe", "reviewer": "Reviewer_ofKA", "summary": "", "questions": "", "limitations": "", "rating": 4, "confidence": 4, "soundness": 4, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "We thank reviewer ofKA for recognizing the novelty of our experimental findings and for their insightful comments.\n\n\n### Why focus on the original kNN-LM and TRIME?\n\nIn this work, we focus on kNN-LM and TRIME for the following reasons, which we will clarify in the next version of the paper. First, both of these methods are interpolation-based retrieval-augmented LMs, as opposed to methods that feed retrieved text into the LM input or fuse it at other layers. Interpolation-based methods such as the kNN-LM are convenient because they are easily applicable to any pretrained LM without any further fine-tuning; as such, the kNN-LM has been cited over 400 times since its publication. We acknowledge that other retrieval LMs such as RETRO likely yield better performance, but these models are not open-sourced and it’s not possible to easily adapt existing pretrained LMs to simulate them. \n\nMeanwhile, both the kNN-LM and TRIME papers include publicly-available model checkpoints, datastores, and codebases, which makes controlled comparisons feasible. \n\nAdditionally, the TRIME model introduces substantial modifications to the conventional kNN-LM by training the retrieval process. This design decision leads to significant improvements over the kNN-LM, and in some ways offers a more comparable analogue to other retrieval paradigms such as RETRO. However, as our paper demonstrates, making the retrieval process trainable does not translate to improvements in text generation quality. We believe this finding is novel and surprising, and thus of importance to the growing body of work on retrieval-augmented language modeling.\n\n\n### What about other variations of kNN-LM?\nWe agree with reviewer ofKA that there are other interesting variations of kNN-LM that we did not study in this paper. We chose not to include these in our paper with the following reasons.\n1. *Efficient Nearest Neighbor Language Models*. EMNLP 2021 (focusing on improvement in inference speed instead of language modeling quality)\n2. *Nearest Neighbor Machine Translation*. ICLR 2021 (focusing on performing machine translation instead of open-ended text generation)\n3. *Neuro-Symbolic Language Modeling with Automaton-augmented Retrieval*. ICML 2022 (requiring non-trivial modification to perform text generation through sampling)\n\nWe hope our experiments can stimulate further research to follow our evaluation framework when evaluating future retrieval-based LM variations.", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "The experimental findings of this paper are relatively novel, and are beneficial for the research of natural language generation.", "reasons_to_reject": "This paper does not analyze more advanced models thar are based on kNN-LM. kNN-LM is a hot research topic in recent years, and there are many improvements over it, but this paper only uses the original model and the TRIME.", "questions_to_authors": "", "paper_topic_and_main_contributions": "This paper conduct relatively comprehensive experiments to deeply study and analyze the kNN-LM, which is an interpolation-based retrieval-augmented language model. Besides, this paper finds that interpolating with a retrieval distribution actually increases perplexity compared to the baseline LM for the majority of tokens in the WikiText-103 test set, which is a little counter-intuitive, because the overall perplexity is lower.", "excitement": 4, "reproducibility": 4, "ethical_concerns": "No", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {}, "extra_text": {}}, {"review_id": "1izP2bDhNY", "reviewer": "Reviewer_qeVJ", "summary": "", "questions": "Question A: according the phenomenons revealed in section 5, how should the future retrieval-based LMs improved?", "limitations": "", "rating": 4, "confidence": 4, "soundness": 4, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "We thank reviewer qeVJ for recognizing the value of our analysis and for their insightful reviews.\n\n> Results may not generalize to other retrieval-augmented LMs except interpolation-styled retrieval LMs:\n\nWe limit the scope of our work to interpolation-styled retrieval-LMs due to their popularity and effectiveness in improving perplexity, in addition to the open-source availability of pretrained checkpoints and datastores (see response to reviewer ofKA for more details). While other types / architectures of retrieval language models may exhibit different behaviors for open-ended text generation, we hope our work can inspire future research to include **quality of generated text** in their evaluation protocol instead of just perplexity. \n\n\n> Human evaluation is limited with only 3 raters, 45 examples. Need more data points.\n\nInstead of conducting crowd-sourced evaluation, we carefully select and hire three annotators who are professional English teachers/editors and are experienced in text generation evaluation, following the recommendations from Karpinska et al [1]. That said, we acknowledge the reviewer’s concern and will hire these same annotators to provide more judgments for the next version of the paper.\n\n\n### [Question]: According to the phenomenons revealed in section 5, how should the future retrieval-based LMs be improved?\n\nWe believe future work can improve text generation from similar retrieval language models by investigating the following directions.\n1. Improving the retrieval quality (such as ensembling the retrieval process with pre-trained dense-retrievers), \n2. Using modified decoding strategies (such as filtering or re-calibrating the retrieval distribution)\n3. Adopting an adaptive retrieval strategy (learning when to retrieve rather than always retrieving at every token)\n\n### References\n1. *The Perils of Using Mechanical Turk to Evaluate Open-Ended Text Generation*. EMNLP 2021.", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "- The authors have done thorough evaluation on interpolation baselines and provide useful insight that models like kNN-LM improve the perplexity only because they largely improve prediction accuracy on a small subset of tokens, and the quality of such retrieval-based LMs deteriorates over longer sequences. \n\n- Designs good analytical experiments on the negative results on the perplexities\n  - Exhibit the increases brought by the nearest neighbour search by POS tags -- helpful to learn that which kinds of tokens should be further improved\n  - Using the ratio of Shannon entropy to show the confidence of the prediction.\n- As an analysis paper, this work provides comprehensive baselines and automatic metrics from various perspectives (MAUVE, RankGen, GPT-3 perplexity, Entity-F1, Seq-Rep-) and human evaluation.", "reasons_to_reject": "- Scope is limited to interpolation-based methods like kNN-LM, findings may not generalise to other retrieval augmented LMs.\n- Human evaluation is limited with only 3 raters of 45 examples. More data could strengthen conclusions.", "questions_to_authors": "Question A: according the phenomenons revealed in section 5, how should the future retrieval-based LMs improved?", "paper_topic_and_main_contributions": "This work explore the performance of  interpolation-based retrieval-augmented language models on text generation. Although these models might have advantages in other practical applications, the findings from the evaluation results show that they actually do not produce texts in higher quality even they have generally lower perplexity as reported in previous studies. \n\n", "excitement": 4, "reproducibility": 4, "ethical_concerns": "No", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {"Typos_Grammar_Style_and_Presentation_Improvements": "Line 632: \"contradicts\" should be \"contradict\""}, "extra_text": {}}, {"review_id": "aD9DDVhUpp", "reviewer": "Reviewer_FiTG", "summary": "", "questions": "Refer to the questions in Reasons to reject. \n", "limitations": "", "rating": 4, "confidence": 4, "soundness": 4, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "We thank reviewer FiTG for recognizing the value of our experiments and for their insightful suggestions!\n\n> It is strange that rater 1 and Rater 2 more confident on KNNLM but Rater 3 more confident for GPT2\n\nDifferent raters show different preference (and level of confidence) in the comparison between the base-LMs and the KNN-LMs mainly due to the fact that the quality of text generated from these models is very difficult to distinguish (examples in Table 5 & 6 of appendix). This observation underscores the fact that kNN-LM does not bring significant improvement in text generation despite improved perplexity. As mentioned in our response to reviewer qeVJ, we will obtain additional human annotations for the next version of our paper.\n\n> Need to look at how each model performs at each aspect ---- does knnlm outperform GPT2 in terms of consistency / logic?\n\nContrary to the improvement in perplexity, we find that kNN-LM does not bring significant improvement in any of the four aspects of our human evaluation (grammatical, fluent, consistent, and logical). We will include more discussion in the next version of the paper.", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "The main contribution is that this work provides different insights of retrieval-augmented LMs for open-ended text generation compared to previous works which shows improvements on perplexity and classification-related tasks. \n", "reasons_to_reject": "The evaluation of open-ended text generation is a difficult task, the evaluation results of automatic metrics and human evaluation in this paper might not be plausible. For example, the quality of human evaluation replies on many important aspects, such as the quality of evaluators, the aspects to be checked. From Figure 1 we can see that Rate1 and Rate2 are more confident on the kNN-LM while Rate3 is more for the GPT-2, which is very strange. Furthermore, the human evaluation looks at grammatical, fluent, consistent, and logical, we need to look at how each model performs at each aspect, e.g. would kNN-LM performs better than GPT-2 in terms of consistent and logical?", "questions_to_authors": "Refer to the questions in Reasons to reject. \n", "paper_topic_and_main_contributions": "This paper investigates the impact of interpolation-based retrieval-augmented LMs on open-ended text generation by looking at two different kNN-LM models on different datasets, concluding that these interpolated LMs cannot improve open-ended text generation task. Extensive analysis and discussions are conducted to explain the potential reasons behind the observations. \n\nThe main contribution is that this work provides different insights of retrieval-augmented LMs for open-ended text generation compared to previous works which shows improvements on perplexity and classification-related tasks. ", "excitement": 4, "reproducibility": 5, "ethical_concerns": "No", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {}, "extra_text": {}}]

Author rebuttal:

Meta-review: The paper evaluates the generation quality of interpolation-based retrieval-augmented language models, exemplified by kNN-LM, which interpolates the predictions of the LM with the predictions based on the most relevant retrievals from the prefix. They find that while such methods improve overall perplexity, the do not improve the quality of open-ended text generation, using both automatic metrics and human evaluations. They provide analyses and plausible explanation of the perplexity decrease reported in previous research. The paper is well written and the experiments well constructed, with insightful analysis of the existing interpolation-based kNN-LM model that can guide better evaluation of similar models in the future.
Decision: Accept-Main