Title: Lift Yourself Up: Retrieval-augmented Text Generation with Self-Memory
Venue: NeurIPS 2023
Accepted: yes
Keywords: natural language processing

retrieval-augmented text generation

self memory
Abstract: With direct access to human-written reference as memory, retrieval-augmented generation has achieved much progress in a wide range of text generation tasks. Since better memory would typically prompt better generation (we define this as primal problem). The traditional approach for memory retrieval involves selecting memory that exhibits the highest similarity to the input. However, this method is constrained by the quality of the fixed corpus from which memory is retrieved. In this paper, by exploring the duality of the primal problem: better generation also prompts better memory, we propose a novel framework, selfmem, which addresses this limitation by iteratively employing a retrieval-augmented generator to create an unbounded memory pool and using a memory selector to choose one output as memory for the subsequent generation round. This enables the model to leverage its own output, referred to as self-memory, for improved generation. We evaluate the effectiveness of selfmem on three distinct text generation tasks: neural machine translation, abstractive text summarization, and dialogue generation, under two generation paradigms: fine-tuned small model and few-shot LLM. Our approach achieves state-of-the-art results in four directions in JRC-Acquis translation dataset, 50.3 ROUGE-1 in XSum, and 62.9 ROUGE-1 in BigPatent, demonstrating the potential of self-memory in enhancing retrieval-augmented generation models. Furthermore, we conduct thorough analyses of each component in the selfmem framework to identify current system bottlenecks and provide insights for future research.

Lift Yourself Up: Retrieval-augmented Text Generation with Self-Memory

Abstract

With direct access to human-written reference as memory, retrieval-augmented generation has achieved much progress in a wide range of text generation tasks. Since better memory would typically prompt better generation (we define this as primal problem ). The traditional approach for memory retrieval involves selecting memory that exhibits the highest similarity to the input. However, this method is constrained by the quality of the fixed corpus from which memory is retrieved. In this paper, by exploring the duality of the primal problem: better generation also prompts better memory, we propose a novel framework, Selfmem , which addresses this limitation by iteratively employing a retrieval-augmented generator to create an unbounded memory pool and using a memory selector to choose one output as memory for the subsequent generation round. This enables the model to leverage its own output, referred to as self-memory, for improved generation. We evaluate the effectiveness of Selfmem on three distinct text generation tasks: neural machine translation, abstractive text summarization, and dialogue generation, under two generation paradigms: fine-tuned small model and few-shot LLM. Our approach achieves state-of-the-art results in four directions in JRC-Acquis translation dataset, 50.3 ROUGE-1 in XSum , and 62.9 ROUGE-1 in BigPatent , demonstrating the potential of self-memory in enhancing retrieval-augmented generation models. Furthermore, we conduct thorough analyses of each component in the Selfmem framework to identify current system bottlenecks and provide insights for future research 1 .

1 Introduction

In recent years, retrieval-augmented text generation has attracted growing interest across various fields, including neural machine translation[28, 17, 2], dialogue response generation[81, 6, 46], and language modeling[36, 77, 19]. This innovative generation paradigm initially equips a fine-tuned small model or a large language model (LLM) with access to an external database (typically the training corpus) using information retrieval techniques. Subsequently, the generation process is conducted based on both the input text and the retrieved memory.

In this paradigm, the guiding principle for memory retrieval is to find the memory that exhibits the highest similarity to the current input [36, 96, 49]. This aligns with the human intuition that a more similar demonstration sample typically offers more hints. As demonstrated in Figure 1, for a retrieval-augmented translation model, the memory similarity alone exhibits a strong correlation with the final translation quality, regardless of other factors that may influence translation quality (e.g.,

polysemy, morphology, and coreference). We define this as the primal problem : better memory prompts better generation . Consequently, numerous studies have focused on how to retrieve better memory, ranging from sparse retrieval to dense retrieval [10, 63], from a fixed retriever to a learnable retriever [41, 8], and from sentence-level memory to more fine-grained token-level memory [36, 35].

However, a fundamental limitation exists in all previous works: the memory is retrieved from a fixed corpus and is constrained by the corpus's quality. Due to the finite retrieval space, bounded memory significantly restricts the potential of memory-augmented generation models [97]. In this paper, we explore the duality of the primal problem , which posits that better generation also prompts better memory . We propose a novel framework called Selfmem , which iteratively employs a retrieval-augmented generator to create an unbounded memory pool and uses a memory selector to choose one output as memory for the subsequent generation round. By combining the primal and dual problem , a retrievalaugmented generation model can elevate itself using its own output, referred to as self-memory. The key insight behind Selfmem is that the text more closely resembling the data distribution during inference is not the training data [87], but the model's own output.

Selfmem consists of two complementary components:

a retrieval-augmented generator and a memory selector. The generator operates under two distinct paradigms: fine-tuning a small model or few-shot prompting an LLM. For the former, we train the generator with labeled data and retrieved memory, while for the latter, we employ a fixed black-box LLM exclusively for inference alongside retrieved in-context learning samples. We then use the generator's output to train a memory selector based on a specific performance metric. By simply replacing the retrieved memory with unbounded generated memory, we achieve higher-quality generation output ( primal problem ), which subsequently serves as memory for the next round after being refined by the memory selector ( dual problem ).

To evaluate the efficacy of the Selfmem , we carry out comprehensive experiments in three distinct text generation tasks: neural machine translation, abstractive text summarization, and dialogue generation. We witness substantial enhancements over robust baselines, attaining state-of-the-art outcomes in JRC-Acquis (four directions), XSum (50.3 ROUGE-1), and BigPatent (62.9 ROUGE-1). To gain deeper insights into the Selfmem , we meticulously investigate each crucial component and pinpoint the existing system bottleneck to guide future research endeavors.

2 Related Work

2.1 Retrieval-augmented Text Generation

Since the world is not a snapshot once the training corpus is collected, we can never expect an ever-large model to capture everything in its parameters, even for LLMs like GPT-4 [62]. Therefore, it is crucial to equip these models with an external memory bank to store additional knowledge or useful demonstration examples for solving various NLP tasks[41, 78, 95].

In the translation domain, retrieval techniques have long been employed by the localization industry to enhance human translators' productivity and consistency even before the advent of machine translation [94]. Early works on machine translation primarily focused on utilizing memory for statistical machine translation (SMT) systems [80, 50]. For neural machine translation (NMT), [28] were the first to use search engines to retrieve memory from the training set and incorporate it with an external memory network. Subsequent research explored various aspects of retrievalaugmented NMT, such as memory encoding methods [92, 93, 31], joint training of retrievers and generators with monolingual data [8], memory granularity [35], and memory diversity [17]. For few-shot LLM generation, strategies for in-context example selection have been proposed to improve translation quality [2]. Furthermore, in-context machine translation has been shown to be effective for on-the-fly adaptation [79]. For dialogue response generation tasks, employing exemplar/template

0.0

0.2

0.4

0.6

0.8

1.0

Memory Similarity

0.2

0.3

0.4

0.5

0.6

0.7

0.8

0.9

Hypothesis BLEU

retrieval as an intermediate step has proven advantageous for generating informative responses [89, 91, 6, 7]. In-context learning example retrieval also aids in controllable dialogue [46]. Other applications include abstractive summarization [64, 14, 18, 15], code generation [30], paraphrase generation [34, 83], language modeling [36, 105], counterfactual data generation [24], open domain question answering [12, 33] and semantic parsing [99].

2.2 Neural Text Reranking

By alleviating the discrepancy between training and inference (i.e., exposure bias) and directly optimizing desired metrics, two-stage reranking methods have facilitated significant progress in various text generation tasks. In machine translation, pioneering works by [75] and [61] introduced and popularized discriminative reranking for SMT. In the context of NMT, research has focused on two primary reranking approaches: generative reranking [56, 32, 88] and discriminative reranking [39, 71, 23]. For syntactic parsing, [21] were the first to employ a two-stage reranking method to select outputs from a base parser, while [11] introduced a maximum entropy reranker. In text summarization, RefSum [53] proposed a second-stage summarization framework to address train-test distribution mismatches. SimCLS [54] used pairwise Learning To Rank (LTR) to select candidates with the highest matching scores. SummaReranker [68] adopted a multi-task mixture-of-experts framework to leverage different metrics capturing various aspects of generated candidates. BRIO [55] reused the base model for a second round of fine-tuning with both cross-entropy loss and a candidate-level ranking loss. JGR [76] employed an alternate training paradigm to train the generator and reranker.

Akey limitation of these reranking methods is that they only represent a one-way process, wherein the selected candidates become the system's final output. In contrast, our framework innovatively utilizes the chosen candidates as memory for the subsequent generation round of a retrieval-augmented generator, which can produce better candidates with enhanced memory.

3 Methods

In this section, we begin with a motivating experiment on generation as memory (§ 3.1). Then, we introduce Selfmem , a framework comprising a retrieval-augmented generator (§ 3.2) and a memory selector (§ 3.3). The complete framework and algorithm are illustrated in Figure 2 and Algorithm 1.

3.1 Generation as Memory

The primary motivation behind our framework stems from the observation that the memory, which is more similar in distribution to the data during inference, is not the training data (38.89 BLEU, as shown in the first row of Table 1). Instead, it is the model's own output (58.58 BLEU) within the unbounded generation space. One interesting exploration involves directly utilizing the generated output as memory in relation to the primal problem : better memory prompts better generation.

We conduct experiments on the JRC-Acquis En → De dataset. The first row in Table 1 represents conventional retrieval-augmented training with retrieved memory and achieves a 58.58 BLEU score. However, directly incorporating beam output of this trained model as memory (Beam) back into the generation model does not yield any improvements (row 2), despite its higher similarity to the reference compared to the retrieved ones. We hypothesize two potential reasons for this: (1) the retrieval-augmented generator may not generalize effectively in this context due to the

memory distribution shift (from 38.89 to 58.58), and (2) the beam memory does not offer any information gain compared to the retrieved one, even it exhibits more overlap with the references.

To investigate the first hypothesis, we conduct experiments under the oracle and random scenarios by using the reference as memory (Reference) and randomly sampled sentences as memory (Random). The result is shown in Table 1 and it illustrates that a retrieval-augmented generator (trained with

Target Distribution

NLL Loss

KL Loss

Y

N

Y

1

...

Y

X

Y

X

candidates

source

target

training

memory

...

...

...

...

(a) Retrieval-augmented Generator

(b) Memory Selector

Retrieval

Predicted Distribution

M

Primal

Dual

retrieved memory) has already learned to discriminate between different memories in both oracle and random scenarios, without updating the model weights.

To evaluate the second conjecture, we first define the token sets of the reference, retrieved memory, and beam memory as R , M , and B , respectively. The overlap token set, denoted by O , is defined as the tokens that overlap with the references in the beam memory but not in the retrieved memory, which is represented as R∩B-R∩M . O is considered as the additional information provided by the beam memory. Inspired by the confidence analysis of NMT model [58], we compute the set confidence score, ψ ( · ) , as follows:

where p ( y i | x, y <i ) is defined by the generation model. ψ ( · ) measures the confidence with which the generation model generates the tokens. The value of ψ ( R ) is 0.58, while that of O is 0.76, indicating that the generator is relatively confident in generating tokens in O , and therefore does not need to resort to external memory [38]. Beam search ranks generated candidates based on p ( y | x ) , where the selected memory falls within the confidence region of the generator and consequently provides no information gain. This observation motivates us to select memory according to metrics other than p ( y | x ) in the memory selector (§3.3).

3.2 Retrieval-augmented Generator

Given a text pair ( x, y ) , where x = { x 1 , ..., x | x | } is the source, y = { y 1 , ..., y | y | } is the target. They could be (document, summary) in summarization, (context, response) in dialogue generation or (source, target) in machine translation. The retrieval-augmented generation would first use x to retrieve memory m from datastore D . Then the generator G ξ ( x, m ) , parameterized by ξ , would take both x and m as input to generate the target sentence y . In this paper, following standard practice, we choose the training set as D = { ( x i , y i ) } | D | i =1 . For LLM as G ξ , we use the standard in-context learning format to give ( x, y ) as demonstration example. For tunable generator G ξ , we only keep the target side of top-1 retrieval results as memory and we consider two commonly used architectures: Joint-Encoder [29, 87, 41] and Dual-Encoder [92, 8, 17].

Joint-Encoder This architecture is the standard encoder-decoder-based model [3, 84]. The input is the concatenation of x and m . The encoder would first map the input into the hidden states H :

And the decoder would incorporate H by attention mechanism and generate tokens in an autoregressive manner:

Dual-Encoder Instead of treating x and m as a long sequence, this architecture has two encoders, one for x and the other for m . Their outputs are sequentially attended by the decoder with dual cross attention as in [17]:

We use Transformer [84] as the building block for both architectures and optimize G ξ with NLL loss:

3.3 Memory Selector

The role of memory selector S θ ( x, c ) , parameterized by θ , is to select one candidate c from the candidate pool C generated by G ξ based on a specific metric ∆( · , · ) . The chosen candidate c is then utilized as memory m for the subsequent generation round of G ξ . As discussed in §3.1, using p G ξ ( y | x ) as the metric ∆( · , · ) would result in falling into the confidence region of G ξ , leading to no information gain. Moreover, a larger value of p G ξ ( y | x ) does not necessarily guarantee improved generation quality [59]. Consequently, we define ∆( · , · ) as model-free metrics that are widely employed for assessing generation quality, such as BLEU for Neural Machine Translation (NMT) and ROUGE for Summarization. Our memory selector takes the concatenation of the source x and candidate c i as input, and produces a multinomial distribution p S θ ( ·| x ) over C .

In this paper, we focus on the role of the memory selector, S θ ( x, c ) , which is parameterized by θ . The objective of this selector is to choose a single candidate c from the candidate pool C , generated by G ξ , based on a specific metric, ∆( · , · ) .

In accordance with [39], the training goal for S θ is to minimize the discrepancy between the S θ 's predictions and the scores determined by ∆( · , · ) . This divergence is quantified using the KullbackLeibler (KL) divergence.

τ is the temperature to control the smoothness of the distribution. At inference, the output of the S θ is arg max c i ∈ C p S θ ( c i | x ) .

3.4 Combine Generator and Selector

We define two generation modes for G ξ . The first mode, referred to as the hypothesis mode , generates a single output for each input, which is utilized for system evaluation. The second mode, known as the candidate mode , produces N outputs for a given input, and is employed for training S θ as well as memory selection. By integrating two modes together, we present the complete framework of our proposed model, Selfmem , as illustrated in Algorithm 1.

4 Experimental Setup

4.1 Dataset

We assess the performance of Selfmem on three generation tasks, utilizing a total of seven datasets. Translation. We evaluate our framework on JRC-Acquis datasets [82], a collection of parallel

Algorithm 1 Selfmem Framework

Require: a dataset D , a retriever R , a memory selection metric ∆( · , · ) , a retrieval-augmented generator G ξ , and a memory selector S θ

1: retrieve memory M in D with R

2: train G ξ with D and M (if not LLM)

3: use G ξ to generate candidate pool C with M in candidate mode

4: train S θ on C with ∆( · , · )

5: while not converged in the validation set do

6: S θ selects memory from C as M

7: G ξ generates candidate pool C with M in candidate mode

8: end while

9: G ξ generates the final hypothesis with M in hypothesis mode

legislative text of European Union Law. It is the benchmark dataset used in translation memoryaugmented NMT task [28, 92, 8, 17]. We choose 4 translation directions, namely, Spanish ↔ English (Es ↔ En), German ↔ English (De ↔ En). Summarization. We evaluate on 2 summarization datasets: 1) XSum [60], extreme summarization, a single-document summarization dataset with highly abstractive articles from British Broadcasting Corporation. 2) BigPatent [73], consisting of 1.3 million records of U.S. patent documents along with human-written abstractive summaries. Dialogue. We experiment on DailyDialog [44], which contains multi-turn dialogs on daily life topics and is used by [13, 4, 103]. The detailed statistics for these datasets can be found in the Appendix A.

4.2 Implementation Details

We utilize the BM25 algorithm [70] for retrieval purposes. For all tasks, the candidate generation method consists of beam search with a beam width of 50. The number of iterations is determined by the performance on the validation set. For translation , we follow the approach of [93, 8, 17], employing a randomly initialized Transformerbase architecture as G ξ for trainable small model and XGLM [48] for LLM in-context learning. Evaluation metrics include BLEU, TER, and chrF++ obtained from SACREBLEU[66]. The memory selector S θ utilizes an XLM-R base [22] as backbone, with BLEU serving as ∆( · , · ) . For summarization , we initialize G ξ with BARTbase[40] for BigPatent and employ BRIO [55] for XSum . The evaluation metric comprises ROUGE (R1/2/L) [47]. For dialogue generation , BARTbase serves as the backbone for G ξ . Our dialogue system is evaluated using BLEU (B-1/2) and Distinct (D-1/2) scores [43]. For both dialogue and summarization tasks, we adhere to the methods of [54, 26], adopting RoBERTabase [52] as the backbone for S θ . The linear combination of B-1/2 is chosen as ∆( · , · ) for Dialogue Generation, while R-1/2/L is used for Summarization, following [76]. For further implementation details, please refer to the Appendix B and Appendix C for evaluation metrics.

5 Experimental Results

5.1 Machine Translation

We select four translation directions and experiment with two generation paradigms: trainable small models and few-shot prompted LLMs [85, 20]. For trainable models, we explore two architectures (joint and dual, as detailed in §3.2). The baselines comprise two types of translation systems: one being the vanilla sequence-to-sequence model [3, 84] without memory augmentation, and the other consisting of retrieval-augmented translation models focusing on memory encoding [28, 92], memory construction [101], memory retrieval [8], and memory diversity [17]. Based on the experimental results 2 shown in Table 2, Selfmem significantly enhances the performance of G ξ across four translation datasets and two different architectures. This is noteworthy, given that the parameters of the G ξ remain fixed, with the only variable being the input memory. This finding is consistent with the primal problem which posits that improved memory typically leads to better generation results.

The dual problem is revealed in Table 3. Self-memory, which essentially represents the model's own output, exhibits greater similarity with the ground truth and serves as a more effective memory for generating the final output. This observation highlights a key distinction between Selfmem and previous reranking works [39, 68]. Reranking aims to select candidates of higher quality than the beam output, whereas in Selfmem , the chosen candidates serve as memory for the retrieval-augmented generator and do not necessarily need to surpass the quality of the beam hypotheses.

In Table 4, we present the results of LLM with self-memory. We employ XGLM [48] as our backbone generator, with three different sizes ranging from 1.7B to 7.5B. We utilize the recommended prompt as described in [48]. We select three in-context learning examples and report the average scores from three separate runs, taking into account the sensitivity of example selection in ICL [49]. From the table, we first observe a general trend where few-shot translation performance improves as the

size of the model increases. Furthermore, we find that more similar translation demonstrations significantly enhance performance across all model sizes (from random, kNN to Self). This suggests that demonstration examples in in-context learning not only act as triggers for model ability but also adhere to the primal problem , where better demonstration example leads to better generation. Also, by comparing the results in Table 2 and Table 4, we can conclude that the cross-lingual LLM with designed examples still falls short of the supervised baselines in this task.

5.2 Summarization

In this paper, we compare the performance of our trainable model with those of REINA [87], PEGASUS [100], and BART [40]. The results are presented in Table5. Initially, it can be observed that memory has varying impacts on different datasets. The enhancement brought by memory in the BigPatent dataset is significantly larger than that in the XSum dataset. This can be attributed to the inherent characteristics of the BigPatent dataset, which consists of official patent documents that exhibit considerable similarity. Consequently, this greatly improves the summarization quality in accordance with the primal problem . Furthermore, we discovered that self-memory substantially enhances the performance of both BRIO (+1.2 R1) and BART (+18.5 R1), achieving state-of-the-art results on both datasets. We selected these baselines for a fair comparison, as they share the same base generator. Due to space constraints, additional comparisons and the confidence region of the SOTA model can be found in the Appendix E.

5.3 Dialogue Generation

As demonstrated in Table 6, the self-memory significantly enhances the performance of the retrievalaugmented generator for dialogue generation tasks. By optimizing memory using BLEU as ∆( · , · ) , the self-memory improves the B-1,2 score over retrieved memory by 3.08 B-1 and 0.6 B-2 on BARTjoint. Intriguingly, although Selfmem surpasses the baselines in terms of B-1/2, it falls behind in D-1 and D-2, which can be attributed to the trade-off between BLEU score and Distinct score when evaluating a dialogue system [104]. To address this issue, we opt for D-1,2 as ∆( · , · ) when optimizing S θ , denoted as BARTjoint † (D). The results in Table 6 highlight the remarkable flexibility of Selfmem by directly optimizing memory to achieve the desired attributes for diverse and informative dialogue.

6 Further Analysis

To gain a deeper insight into Selfmem , we first examine the impact of each key component, namely G ξ and S θ . Subsequently, we perform a detailed token-level analysis of the generated output concerning their frequency in the training set. Experiments are conducted on the JRC-Acquis En → De dataset. We also include latency analysis and human evaluation on Appendix F and G.

Tuning S θ We explored various S θ by direct selection from the candidate pool based on gold rankings. As shown in Figure 3a, both architectures with enhanced S θ significantly outperform the current SOTA performance (60.11 BLEU). Moreover, we assessed the candidate pool quality during this iterative process using an oracle S θ , as displayed in Figure 3b. A clear pattern emerges

1

2

3

4

Iteration

60

62

64

66

Hypothesis BLEU

Transformer-joint

rank1

rank2

rank3

rank4

rank5

rank6

1

2

3

4

Iteration

60

62

64

66

Transformer-dual

rank1

rank2

rank3

rank4

rank5

rank6

(a) Hypothesis

1

2

3

4

5

Iteration

45

50

55

60

65

70

Candidates BLEU

(b) Candidates

in this boxplot, revealing improvements in the oracle , quartile , average , and minimum scores of the candidate pool. These two experiments jointly clarify the Selfmem 's underlying intuition: a retrieval-augmented generator profits from superior memory, which can be chosen from its own unbounded output, and subsequently, the generator with improved memory produces a higher-quality candidate pool for the next selection round. Consequently, the model lift itself up.

Tuning G ξ As discussed in §3.1, we demonstrated that a trained retrieval-augmented generator, with fixed parameters, possesses the ability to distinguish between "good" and "bad" memory. This observation not only justifies our decision to maintain a fixed generator within our framework but also implies that the G ξ is not the current bottleneck of the Selfmem .

Frequency Analysis We conduct a comprehensive tokenlevel analysis by computing the 1-gram F1 scores for generated translations and subsequently categorizing the tokens based on their frequency in the training set. The results are depicted in Figure 4. A noticeable pattern emerges, suggesting that the more frequently a model encounters a token during training,

The Least

20%

40%

60%

80%

The Most

Frequency Rank

0.65

0.70

0.75

0.80

0.85

0.90

0.95

1.00

F1

Self

Retrieval

None

the higher the accuracy of the generated output [102]. Moreover, our findings indicate that retrievalaugmented models, particularly those incorporating self-memory augmentation, exhibit superior performance in handling long-tail inputs which are challenges for parametric models [67, 57].

7 Conclusion

For the first time, we investigate the fundamental limitation of bounded memory in the current retrieval-augmented literature. We combine the primal and dual problems together and propose Selfmem , a general framework for retrieval-augmented text generation by uplifting generation model with its own output. We conduct comprehensive experiments across various text generation tasks and different generation paradigms, including trainable small model and few-shot prompted LLM. We surpass strong baselines and improve the state-of-the-art performance in serval datasets. We also meticulously investigate each crucial component and pinpoint the existing system bottleneck to guide future research endeavors.

Limitations

We discuss the limitations of our framework as follows:

(1) Although Selfmem greatly improves the generation quality compared with other retrievalaugmented generation models, it requires more computational resources with respect to the memory selection process. For large dataset with long context (e.g., BigPatent ), it would become a more crucial problem considering the quadratic time complexity of transformer architecture.

(2) This paper proposes a general idea for the retrieval-augmented generation. But we only experiment with transformer-based architecture for both generator and memory selector and the architecture of generator and memory selector keeps the same across all text generation tasks. We believe the task-specific design for the model architecture, training objective and generation methods in different text generation scenarios would further improve the performance.

References

Armen Aghajanyan, Anchit Gupta, Akshat Shrivastava, Xilun Chen, Luke Zettlemoyer, and Sonal Gupta. Muppet: Massive multi-task representations with pre-finetuning. In Proc. of EMNLP , 2021.

Sweta Agrawal, Chunting Zhou, Mike Lewis, Luke Zettlemoyer, and Marjan Ghazvininejad. In-context examples selection for machine translation. CoRR , 2022.

Dzmitry Bahdanau, Kyunghyun Cho, and Yoshua Bengio. Neural machine translation by jointly learning to align and translate. In Proc. of ICLR , 2015.

Siqi Bao, Huang He, Fan Wang, Hua Wu, and Haifeng Wang. PLATO: pre-trained dialogue generation model with discrete latent variable. In Proc. of ACL , 2020.

Siqi Bao, Huang He, Fan Wang, Hua Wu, Haifeng Wang, Wenquan Wu, Zhen Guo, Zhibin Liu, and Xinchao Xu. PLATO-2: Towards building an open-domain chatbot via curriculum learning. In Proc. of ACL Findings , 2021.

Deng Cai, Yan Wang, Wei Bi, Zhaopeng Tu, Xiaojiang Liu, Wai Lam, and Shuming Shi. Skeleton-to-response: Dialogue generation guided by retrieval memory. In Proc. of NAACL , 2019.

Deng Cai, Yan Wang, Wei Bi, Zhaopeng Tu, Xiaojiang Liu, and Shuming Shi. Retrieval-guided dialogue response generation via a matching-to-generation framework. In Proc. of EMNLP , 2019.

Deng Cai, Yan Wang, Huayang Li, Wai Lam, and Lemao Liu. Neural machine translation with monolingual translation memory. In Proc. of ACL , 2021.

Chris Callison-Burch, Miles Osborne, and Philipp Koehn. Re-evaluating the role of Bleu in machine translation research. In Proc. of EACL , 2006.

Qian Cao and Deyi Xiong. Encoding gated translation memory into neural machine translation. In Proc. of EMNLP , 2018.

Eugene Charniak and Mark Johnson. Coarse-to-fine n-best parsing and maxent discriminative reranking. In Proc. of ACL , 2005.

Danqi Chen, Adam Fisch, Jason Weston, and Antoine Bordes. Reading wikipedia to answer open-domain questions. In Proceedings of the 55th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers) , pages 1870-1879, 2017.

Wei Chen, Yeyun Gong, Song Wang, Bolun Yao, Weizhen Qi, Zhongyu Wei, Xiaowu Hu, Bartuer Zhou, Yi Mao, Weizhu Chen, Biao Cheng, and Nan Duan. Dialogved: A pre-trained latent variable encoder-decoder model for dialog response generation. In Proc. of ACL , 2022.

Xiuying Chen, Hind Alamro, Mingzhe Li, Shen Gao, Rui Yan, Xin Gao, and Xiangliang Zhang. Target-aware abstractive related work generation with contrastive learning. In Proc. of SIGIR , 2022.

Xiuying Chen, Mingzhe Li, Shen Gao, Xin Cheng, Qiang Yang, Qishen Zhang, Xin Gao, and Xiangliang Zhang. A topic-aware summarization framework with different modal side information. Proc. of SIGIR , 2023.

Xiuying Chen, Mingzhe Li, Xin Gao, and Xiangliang Zhang. Towards improving faithfulness in abstractive summarization. In Proc. of NeurIPS , 2022.

Xin Cheng, Shen Gao, Lemao Liu, Dongyan Zhao, and Rui Yan. Neural machine translation with contrastive translation memories. In Proc. of EMNLP , 2022.

Xin Cheng, Shen Gao, Yuchi Zhang, Yongliang Wang, Xiuying Chen, Mingzhe Li, Dongyan Zhao, and Rui Yan. Towards personalized review summarization by modeling historical reviews from customer and product separately. arXiv preprint arXiv:2301.11682 , 2023.

Xin Cheng, Yankai Lin, Xiuying Chen, Dongyan Zhao, and Rui Yan. Decouple knowledge from paramters for plug-and-play language modeling. In Findings of the Association for Computational Linguistics: ACL 2023 , pages 14288-14308, Toronto, Canada, July 2023. Association for Computational Linguistics.

Xin Cheng, Xun Wang, Tao Ge, Si-Qing Chen, Furu Wei, Dongyan Zhao, and Rui Yan. Scale: Synergized collaboration of asymmetric language translation engines, 2023.

Michael Collins and Terry Koo. Discriminative reranking for natural language parsing. Comput. Linguistics , 2005.

Alexis Conneau, Kartikay Khandelwal, Naman Goyal, Vishrav Chaudhary, Guillaume Wenzek, Francisco Guzmán, Edouard Grave, Myle Ott, Luke Zettlemoyer, and Veselin Stoyanov. Unsupervised cross-lingual representation learning at scale. In Proc. of ACL , 2020.

Yuntian Deng, Anton Bakhtin, Myle Ott, Arthur Szlam, and Marc'Aurelio Ranzato. Residual energy-based models for text generation. In Proc. of ICLR , 2020.

Tanay Dixit, Bhargavi Paranjape, Hannaneh Hajishirzi, and Luke Zettlemoyer. CORE: A retrieve-then-edit framework for counterfactual data generation. In Proc. of EMNLP Findings , 2022.

Le Fang, Chunyuan Li, Jianfeng Gao, Wen Dong, and Changyou Chen. Implicit deep latent variable models for text generation. In Proc. of EMNLP , 2019.

Jiazhan Feng, Chongyang Tao, Zhen Li, Chang Liu, Tao Shen, and Dongyan Zhao. Reciprocal learning of knowledge retriever and response ranker for knowledge-grounded conversations. In Proc. of COLING , 2022.

Tingchen Fu, Xueliang Zhao, Chongyang Tao, Ji-Rong Wen, and Rui Yan. There are a thousand hamlets in a thousand people's eyes: Enhancing knowledge-grounded dialogue with personal memory. In Proc. of ACL , 2022.

Jiatao Gu, Yong Wang, Kyunghyun Cho, and Victor O. K. Li. Search engine guided neural machine translation. In Proc. of AAAI , 2018.

Kelvin Guu, Kenton Lee, Zora Tung, Panupong Pasupat, and Ming-Wei Chang. Retrieval augmented language model pre-training. In Proc. of ICML , 2020.

Tatsunori B. Hashimoto, Kelvin Guu, Yonatan Oren, and Percy Liang. A retrieve-and-edit framework for predicting structured outputs. In Proc. of NeurIPS , 2018.

Qiuxiang He, Guoping Huang, Qu Cui, Li Li, and Lemao Liu. Fast and accurate neural machine translation with translation memory. In Proc. of ACL , 2021.

Kenji Imamura and Eiichiro Sumita. Ensemble and reranking: Using multiple models in the NICT-2 neural machine translation system at WAT2017. In Proceedings of the 4th Workshop on Asian Translation, WAT@IJCNLP 2017, Taipei, Taiwan, November 27- December 1, 2017 , 2017.

Gautier Izacard and Edouard Grave. Leveraging passage retrieval with generative models for open domain question answering. In Proceedings of the 16th Conference of the European Chapter of the Association for Computational Linguistics: Main Volume , pages 874-880, Online, April 2021. Association for Computational Linguistics.

Amirhossein Kazemnejad, Mohammadreza Salehi, and Mahdieh Soleymani Baghshah. Paraphrase generation by learning how to edit from samples. In Proc. of ACL , 2020.

Urvashi Khandelwal, Angela Fan, Dan Jurafsky, Luke Zettlemoyer, and Mike Lewis. Nearest neighbor machine translation. In Proc. of ICLR , 2021.

Urvashi Khandelwal, Omer Levy, Dan Jurafsky, Luke Zettlemoyer, and Mike Lewis. Generalization through memorization: Nearest neighbor language models. In Proc. of ICLR , 2020.

Philipp Koehn. Statistical significance tests for machine translation evaluation. In Proc. of EMNLP , 2004.

Ankit Kumar, Ozan Irsoy, Peter Ondruska, Mohit Iyyer, James Bradbury, Ishaan Gulrajani, Victor Zhong, Romain Paulus, and Richard Socher. Ask me anything: Dynamic memory networks for natural language processing. In Proc. of ICML , 2016.

Ann Lee, Michael Auli, and Marc'Aurelio Ranzato. Discriminative reranking for neural machine translation. In Proc. of ACL , 2021.

Mike Lewis, Yinhan Liu, Naman Goyal, Marjan Ghazvininejad, Abdelrahman Mohamed, Omer Levy, Veselin Stoyanov, and Luke Zettlemoyer. BART: denoising sequence-to-sequence pre-training for natural language generation, translation, and comprehension. In Proc. of ACL , 2020.

Patrick S. H. Lewis, Ethan Perez, Aleksandra Piktus, Fabio Petroni, Vladimir Karpukhin, Naman Goyal, Heinrich Küttler, Mike Lewis, Wen-tau Yih, Tim Rocktäschel, Sebastian Riedel, and Douwe Kiela. Retrieval-augmented generation for knowledge-intensive NLP tasks. In Proc. of NeurIPS , 2020.

Jinpeng Li, Yingce Xia, Rui Yan, Hongda Sun, Dongyan Zhao, and Tie-Yan Liu. Stylized dialogue generation with multi-pass dual learning. In Proc. of NeurIPS , 2021.

Jiwei Li, Michel Galley, Chris Brockett, Jianfeng Gao, and Bill Dolan. A diversity-promoting objective function for neural conversation models. In Proc. of NAACL , 2016.

Yanran Li, Hui Su, Xiaoyu Shen, Wenjie Li, Ziqiang Cao, and Shuzi Niu. Dailydialog: A manually labelled multi-turn dialogue dataset. In Proceedings of the Eighth International Joint Conference on Natural Language Processing, IJCNLP 2017, Taipei, Taiwan, November 27 December 1, 2017 - Volume 1: Long Papers , 2017.

Zekang Li, Jinchao Zhang, Zhengcong Fei, Yang Feng, and Jie Zhou. Conversations are not flat: Modeling the dynamic information flow across dialogue utterances. In Proc. of ACL , 2021.

Zekun Li, Wenhu Chen, Shiyang Li, Hong Wang, Jing Qian, and Xifeng Yan. Controllable dialogue simulation with in-context learning. In Proc. of EMNLP Findings , 2022.

Chin-Yew Lin. ROUGE: A package for automatic evaluation of summaries. In Text Summarization Branches Out , 2004.

Xi Victoria Lin, Todor Mihaylov, Mikel Artetxe, Tianlu Wang, Shuohui Chen, Daniel Simig, Myle Ott, Naman Goyal, Shruti Bhosale, Jingfei Du, Ramakanth Pasunuru, Sam Shleifer, Punit Singh Koura, Vishrav Chaudhary, Brian O'Horo, Jeff Wang, Luke Zettlemoyer, Zornitsa Kozareva, Mona T. Diab, Veselin Stoyanov, and Xian Li. Few-shot learning with multilingual generative language models. In Proc. of EMNLP , 2022.

Jiachang Liu, Dinghan Shen, Yizhe Zhang, Bill Dolan, Lawrence Carin, and Weizhu Chen. What makes good in-context examples for gpt-3? In Proceedings of Deep Learning Inside Out: The 3rd Workshop on Knowledge Extraction and Integration for Deep Learning Architectures, DeeLIO@ACL 2022, Dublin, Ireland and Online, May 27, 2022 , 2022.

Lemao Liu, Hailong Cao, Taro Watanabe, Tiejun Zhao, Mo Yu, and Conghui Zhu. Locally training the log-linear model for SMT. In Proc. of EMNLP , 2012.

Yang Liu and Mirella Lapata. Text summarization with pretrained encoders. In Proc. of EMNLP , 2019.

Yinhan Liu, Myle Ott, Naman Goyal, Jingfei Du, Mandar Joshi, Danqi Chen, Omer Levy, Mike Lewis, Luke Zettlemoyer, and Veselin Stoyanov. Roberta: A robustly optimized BERT pretraining approach. CoRR , 2019.

Yixin Liu, Zi-Yi Dou, and Pengfei Liu. Refsum: Refactoring neural summarization. In Proc. of NAACL , 2021.

Yixin Liu and Pengfei Liu. Simcls: A simple framework for contrastive learning of abstractive summarization. In Proc. of ACL , 2021.

Yixin Liu, Pengfei Liu, Dragomir R. Radev, and Graham Neubig. BRIO: bringing order to abstractive summarization. In Proc. of ACL , 2022.

Yuchen Liu, Long Zhou, Yining Wang, Yang Zhao, Jiajun Zhang, and Chengqing Zong. A comparable study on model averaging, ensembling and reranking in NMT. In Proc. of NLPCC , 2018.

Alexander Long, Wei Yin, Thalaiyasingam Ajanthan, Vu Nguyen, Pulak Purkait, Ravi Garg, Alan Blair, Chunhua Shen, and Anton van den Hengel. Retrieval augmented classification for long-tail visual recognition. In Proc. of CVPR , 2022.

Yu Lu, Jiali Zeng, Jiajun Zhang, Shuangzhi Wu, and Mu Li. Learning confidence for transformer-based neural machine translation. In Proc. of ACL , 2022.

Clara Meister, Ryan Cotterell, and Tim Vieira. If beam search is the answer, what was the question? In Proc. of EMNLP , 2020.

Shashi Narayan, Shay B. Cohen, and Mirella Lapata. Don't give me the details, just the summary! topic-aware convolutional neural networks for extreme summarization. In Proc. of EMNLP , 2018.

Franz Josef Och, Daniel Gildea, Sanjeev Khudanpur, Anoop Sarkar, Kenji Yamada, Alexander M. Fraser, Shankar Kumar, Libin Shen, David Smith, Katherine Eng, Viren Jain, Zhen Jin, and Dragomir R. Radev. A smorgasbord of features for statistical machine translation. In Proc. of NAACL , 2004.

OpenAI. GPT-4 technical report. CoRR , 2023.

Md. Rizwan Parvez, Wasi Uddin Ahmad, Saikat Chakraborty, Baishakhi Ray, and Kai-Wei Chang. Retrieval augmented code generation and summarization. In Proc. of EMNLP Findings , 2021.

Hao Peng, Ankur P. Parikh, Manaal Faruqui, Bhuwan Dhingra, and Dipanjan Das. Text generation with exemplar-based adaptive decoding. In Proc. of NAACL , 2019.

Jonathan Pilault, Raymond Li, Sandeep Subramanian, and Chris Pal. On extractive and abstractive neural document summarization with transformer language models. In Proc. of EMNLP , 2020.

Matt Post. A call for clarity in reporting BLEU scores. In Proceedings of the Third Conference on Machine Translation: Research Papers, WMT 2018, Belgium, Brussels, October 31 November 1, 2018 , 2018.

Vikas Raunak, Siddharth Dalmia, Vivek Gupta, and Florian Metze. On long-tailed phenomena in neural machine translation. In Proc. of EMNLP Findings , 2020.

Mathieu Ravaut, Shafiq R. Joty, and Nancy F. Chen. Summareranker: A multi-task mixture-ofexperts re-ranking framework for abstractive summarization. In Proc. of ACL , 2022.

Mathieu Ravaut, Shafiq R. Joty, and Nancy F. Chen. Towards summary candidates fusion. CoRR , 2022.

Stephen E. Robertson and Hugo Zaragoza. The probabilistic relevance framework: BM25 and beyond. Found. Trends Inf. Retr. , 2009.

Julian Salazar, Davis Liang, Toan Q. Nguyen, and Katrin Kirchhoff. Masked language model scoring. In Proc. of ACL , 2020.

Rico Sennrich, Barry Haddow, and Alexandra Birch. Neural machine translation of rare words with subword units. In Proc. of ACL , 2016.

Eva Sharma, Chen Li, and Lu Wang. BIGPATENT: A large-scale dataset for abstractive and coherent summarization. In Proc. of ACL , 2019.

Noam Shazeer and Mitchell Stern. Adafactor: Adaptive learning rates with sublinear memory cost. In Proc. of ICML , 2018.

Libin Shen, Anoop Sarkar, and Franz Josef Och. Discriminative reranking for machine translation. In Proc. of NAACL , 2004.

Weizhou Shen, Yeyun Gong, Yelong Shen, Song Wang, Xiaojun Quan, Nan Duan, and Weizhu Chen. Joint generator-ranker learning for natural language generation. CoRR , 2022.

Weijia Shi, Sewon Min, Michihiro Yasunaga, Minjoon Seo, Rich James, Mike Lewis, Luke Zettlemoyer, and Wen tau Yih. Replug: Retrieval-augmented black-box language models, 2023.

Weijia Shi, Sewon Min, Michihiro Yasunaga, Minjoon Seo, Rich James, Mike Lewis, Luke Zettlemoyer, and Wen-tau Yih. REPLUG: retrieval-augmented black-box language models. CoRR , 2023.

Suzanna Sia and Kevin Duh. In-context learning as maintaining coherency: A study of on-the-fly machine translation using large language models. CoRR , 2023.

Michel Simard and Pierre Isabelle. Phrase-based machine translation in a computer-assisted translation environment. In Proceedings of Machine Translation Summit XII: Papers, MTSummit 2009, Ottawa, Canada, August 26-30, 2009 , 2009.

Yiping Song, Rui Yan, Xiang Li, Dongyan Zhao, and Ming Zhang. Two are better than one: An ensemble of retrieval- and generation-based dialog systems. CoRR , 2016.

Ralf Steinberger, Bruno Pouliquen, Anna Widiger, Camelia Ignat, Tomaz Erjavec, Dan Tufis, and Dániel Varga. The jrc-acquis: A multilingual aligned parallel corpus with 20+ languages. In Proc. of LREC , 2006.

Yixuan Su, David Vandyke, Simon Baker, Yan Wang, and Nigel Collier. Keep the primary, rewrite the secondary: A two-stage approach for paraphrase generation. In Proc. of ACL Findings , 2021.

Ashish Vaswani, Noam Shazeer, Niki Parmar, Jakob Uszkoreit, Llion Jones, Aidan N. Gomez, Lukasz Kaiser, and Illia Polosukhin. Attention is all you need. In Proc. of NeurIPS , 2017.

David Vilar, Markus Freitag, Colin Cherry, Jiaming Luo, Viresh Ratnakar, and George Foster. Prompting palm for translation: Assessing strategies and performance, 2023.

Oriol Vinyals and Quoc V. Le. A neural conversational model. CoRR , 2015.

Shuohang Wang, Yichong Xu, Yuwei Fang, Yang Liu, Siqi Sun, Ruochen Xu, Chenguang Zhu, and Michael Zeng. Training data is more valuable than you think: A simple and effective method by retrieving from training data. In Proc. of ACL , 2022.

Yuguang Wang, Shanbo Cheng, Liyang Jiang, Jiajun Yang, Wei Chen, Muze Li, Lin Shi, Yanfeng Wang, and Hongtao Yang. Sogou neural machine translation systems for WMT17. In Proceedings of the Second Conference on Machine Translation, WMT 2017, Copenhagen, Denmark, September 7-8, 2017 , 2017.

Jason Weston, Emily Dinan, and Alexander H. Miller. Retrieve and refine: Improved sequence generation models for dialogue. In Proceedings of the 2nd International Workshop on SearchOriented Conversational AI, SCAI@EMNLP 2018, Brussels, Belgium, October 31, 2018 , 2018.

Wenhao Wu, Wei Li, Xinyan Xiao, Jiachen Liu, Ziqiang Cao, Sujian Li, Hua Wu, and Haifeng Wang. BASS: boosting abstractive summarization with unified semantic graph. In Proc. of ACL , 2021.

Yu Wu, Furu Wei, Shaohan Huang, Yunli Wang, Zhoujun Li, and Ming Zhou. Response generation by context-aware prototype editing. In Proc. of AAAI , 2019.

Mengzhou Xia, Guoping Huang, Lemao Liu, and Shuming Shi. Graph based translation memory for neural machine translation. In Proc. of AAAI , 2019.

Jitao Xu, Josep Maria Crego, and Jean Senellart. Boosting neural machine translation with similar translations. In Proc. of ACL , 2020.

Masaru Yamada. The effect of translation memory databases on productivity. Translation research projects , 2011.

Michihiro Yasunaga, Armen Aghajanyan, Weijia Shi, Rich James, Jure Leskovec, Percy Liang, Mike Lewis, Luke Zettlemoyer, and Wen-tau Yih. Retrieval-augmented multimodal language modeling. CoRR , 2022.

Dani Yogatama, Cyprien de Masson d'Autume, and Lingpeng Kong. Adaptive semiparametric language models. Trans. Assoc. Comput. Linguistics , 2021.

Wenhao Yu, Dan Iter, Shuohang Wang, Yichong Xu, Mingxuan Ju, Soumya Sanyal, Chenguang Zhu, Michael Zeng, and Meng Jiang. Generate rather than retrieve: Large language models are strong context generators, 2023.

Manzil Zaheer, Guru Guruganesh, Kumar Avinava Dubey, Joshua Ainslie, Chris Alberti, Santiago Ontanon, Philip Pham, Anirudh Ravula, Qifan Wang, Li Yang, and Amr Ahmed. Big bird: Transformers for longer sequences. In Proc. of NeurIPS , 2020.

Yury Zemlyanskiy, Michiel de Jong, Joshua Ainslie, Panupong Pasupat, Peter Shaw, Linlu Qiu, Sumit Sanghai, and Fei Sha. Generate-and-retrieve: Use your predictions to improve retrieval for semantic parsing. In Proceedings of the 29th International Conference on Computational Linguistics , pages 4946-4951, Gyeongju, Republic of Korea, October 2022. International Committee on Computational Linguistics.

Jingqing Zhang, Yao Zhao, Mohammad Saleh, and Peter J. Liu. PEGASUS: pre-training with extracted gap-sentences for abstractive summarization. In Proc. of ICML , 2020.

Jingyi Zhang, Masao Utiyama, Eiichiro Sumita, Graham Neubig, and Satoshi Nakamura. Guiding neural machine translation with retrieved translation pieces. In Proc. of NAACL , 2018.

Tong Zhang, Wei Ye, Baosong Yang, Long Zhang, Xingzhang Ren, Dayiheng Liu, Jinan Sun, Shikun Zhang, Haibo Zhang, and Wen Zhao. Frequency-aware contrastive learning for neural machine translation. In Proc. of AAAI , 2022.

Xueliang Zhao, Lemao Liu, Tingchen Fu, Shuming Shi, Dongyan Zhao, and Rui Yan. Towards efficient dialogue pre-training with transferable and interpretable latent structure. CoRR , 2022.

Yinhe Zheng, Zikai Chen, Rongsheng Zhang, Shilei Huang, Xiaoxi Mao, and Minlie Huang. Stylized dialogue response generation using stylized unpaired texts. In Proc. of AAAI , 2021.

Zexuan Zhong, Tao Lei, and Danqi Chen. Training language models with memory augmentation. CoRR , 2022.

A Dataset Details

B Self Memory Details

For machine translation tasks, following [93, 8, 17] we use randomly initialize Transformerbase architecture [84] as G ξ . We use the joint-bpe algorithm [72] and share the parameters between the memory encoder and source encoder for dual encoder architecture. The hyper-parameter setting follows [17] with dropout 0.1, label smoothing 0.1, gradient clipping 1.0, Adafactor [74], warm-up steps 4000, maximum learning rate 4.4e-2 and training epochs 30 for total. The evaluation metrics are BLEU, TER and chrF++ from SACREBLEU [66]. The backbone of memory selector S θ is XLM-R base [22] with BLEU as ∆( · , · ) . The hyper-parameter setting for S θ follows [39] with τ 0.5, minmax normalization for candidates ranking, Adam optimizer with max learning rate 5e-5 and polynomial decay scheduler, and classifier dropout 0.2.

For Summarization, we init the G ξ with BARTbase [40] for BigPatent following [87] and state-of-theart BRIO [55] for XSum. Optimization is based on Adafactor with a maximum learning rate of 5e-3, warm-up steps 10000 and gradient clipping value 1.0. The maximum input length is 512 for XSum and 1024 for BigPatent. The evaluation metric is Rouge (R-1/2/L) [47].

For Dialogue Generation, we use BARTbase as the backbone for G ξ on DailyDialog. We tune the hyper-parameters from learning rate {5e-3,1e-3,4e-4} and set dropout 0.1, batch size 64, label smoothing factor 0.1, maximum input length 120 for DailyDialog. Following [4, 13], we evaluate our dialogue system with BLEU (B-1/2) and Distinct (D-1,2) [43]. For both Summarization and Dialogue Generation task, we follow [54, 26] and adopt RoBERTabase [52] as the backbone for S θ . We choose the linear combination of B-1/2 as ∆( · , · ) for Dialogue Generation and R-1/2/L for Summarization following [76]. We tune the hyper-parameters τ from {0.08,0.2,0.5,0.8}, learning rate from {5e-5,7e-5,2e-4}. The maximum input length for S θ is 512 and we truncate tokens from the longer input of source and candidate.

C Evaluation Details

Machine Translation We evaluate our MT system with BLEU, TER and chrF++ from SACREBLEU 3 [66]. The signatures for BLEU, TER and chrF++ are shown in Table 8.

Summarization We evaluate our Summarization system with standard ROUGE [47] Perl package 4 for evaluation. Following [55], we use PTB tokenizer 5 for tokenization. And the parameters for ROUGE are "-c 95 -r 1000 -n 2 -m".

Dialogue Generation Following [27], we evaluate our dialogue system with NLTK BLEU 6 with space as tokenizer and smoothing method1. The Distinction score is from [42].

D More results on translation tasks

E More Summarization Baselines

In this Table 10, we include more baselines on the benchmark dataset XSum and BigPatent . We also report the confidence region of SOTA model for XSum and BigPatent as shown in Table 11.

F Empirical analysis of latency

In Table 12, we present empirical results of Selfmem latency, measured in seconds. We compare Selfmem with a retrieval-augmented baseline model across various datasets and computational platforms, including CPU and CUDA. The number of iterations for Selfmem is set to one. All experiments are conducted on the same device, equipped with one NVIDIA A100 GPU and one AMDEPYC 7V13 64-Core Processor.

G Human and GPT-4 Evaluation

We employ both human annotators and GPT-4 (gpt-4-0314) annotators to perform pairwise ranking of the output generated by Selfmem and baseline systems. For GPT-4 annotators, we utilize the prompt from Alpaca Eval 7 . We randomly select 50 samples for translation tasks and 20 samples for summarization and dialogue tasks. The win rate of Selfmem versus retrieval-augmented baselines is depicted in Figure 1.

En-De

En-Es

XSum

DailyDialog

60

70

80

90

76.0

80.0

75.0

60.0

88.0

82.0

65.0

65.0

Human and GPT-4 Evaluation Results

Human Annotator

Peer reviews:
{"IS_META_REVIEW": false, "comments": "SUMMARY:\nThe paper proposes a novel framework called Selfmem to address the limitations of retrieval-augmented text generation. Compared with memory retrieval from a fixed corpus, Selfmem iteratively uses a retrieval-augmented generator to create an unbounded memory pool and select the candidate output as memory for the subsequent generation round. The paper evaluates the effectiveness of Selfmem on three text generation tasks and achieves state-of-the-art results. The paper also conducts thorough analyses of each component in the framework.\n\nSTRENGTHS:\nThe motivation of the primal problem and its duality is fairly intriguing. Preliminary experiments and supporting evidence demonstrate the rationality of the proposed method.\n\nThe proposed Selfmem achieves promising results on machine translation, summarization and dialogue generation, indicating its practical applicability.\n\nThe paper is overall clearly written. The problem setting and proposed approach is lucidly presented.\n\nWEAKNESSES:\nThere is a growing body of recent work [1] that is increasingly using LLMs to generate knowledge instead of retrieving it from external corpus. This paper shares similar ideas with the new paradigm and should incorporate and discuss it in the related work section.\n\nFurther clarification and empirical evaluation on the diversity of retrieval memory from Selfmem should be provided. Given the limited knowledge encoded in the trainable generator, it remains uncertain whether Selfmem can effectively handle knowledge-intensive tasks compared to retrieval from external sources.\n\nSelfmem appears to require more computational cost than direct retrieval. However, there is no further discussion on time computation in the paper.\n\n[1] Yu et al. Generate rather than Retrieve: Large Language Models are Strong Context Generators. ICLR 2023\n\nQUESTIONS:\n1.\tAlthough you mention the computational resources in the limitations section, can you offer an empirical comparison between Selfmem with other retrieval-based methods?\n\n2.\tFor a more vivid demonstration, can you provide any case studies or examples to illustrate how the generated contexts from Selfmem outperform the retrieval-based ones.\n\nLIMITATIONS:\nSee the weakness section.", "TITLE": "Official Review", "RECOMMENDATION": 6, "REVIEWER_CONFIDENCE": 3, "SOUNDNESS_CORRECTNESS": 3, "DATE": "2023-07-05"}

{"IS_META_REVIEW": false, "comments": "SUMMARY:\nThe paper presents a framework, Selfmem, aimed at enhancing text generation tasks via memory retrieval. The core uniqueness of the framework resides in its capacity to create an unbounded memory pool by utilizing its retrieval-augmented generator and memory selector components iteratively. This allows the model to leverage its output, referred to as self-memory, for improved generation. The authors apply this framework into neural machine translation, abstractive text summarization, and dialogue generation tasks, demonstrating its efficacy with state-of-the-art results\n\nSTRENGTHS:\n(1) The paper proposes the Selfmem framework, which addresses the constraint of fixed corpus in memory retrieval.\n(2) The paper demonstrates the effectiveness of Selfmem on various text generation tasks.\n(3) The paper provides detailed analyses of components within the Selfmem framework, revealing potential bottlenecks and providing  insights.\n\nWEAKNESSES:\n(1) It would be better if more experiments beyond neural machine translation, abstractive text summarization, and dialogue generation tasks could be conducted. It will be beneficial to see its application in other related areas since the proposed method is task-agnostic.\n(2) Much of the paper's experimentation relies on certain metrics (ROUGE-1) for evaluation. The universality of the findings might be cross-checked through the usage of other performance metrics.\n(3) Though the authors mention learnable retrievers, they don't discuss how the Selfmem framework may be adapted for use with such retrievers.\n\nQUESTIONS:\n(1) Have you considered using other evaluation methods (e.g., human evaluation) to further validate the performance of the framework?\n(2) How can the Selfmem framework be adapted for optimal usage with learnable retrievers?\n\nLIMITATIONS:\nA potential concern is, compared to human-written texts, machine-generated texts are noisier and with potential bias. Hence, is the proposed method able to avoid being affected by its own bias?", "TITLE": "Official Review", "RECOMMENDATION": 5, "REVIEWER_CONFIDENCE": 4, "SOUNDNESS_CORRECTNESS": 3, "DATE": "2023-07-05"}

{"IS_META_REVIEW": false, "comments": "SUMMARY:\nThe traditional approach for memory retrieval is constrained by the quality of the fixed corpus from which memory is retrieved. Based on the exploration that \"better generation also promotes better memory\", this work proposes the Selfmem framework, which iteratively employs a retrieval-augmented generator to create an unbounded memory pool and uses a memory selector to select a generated memory for the next round. Experimental results demonstrate the effectiveness of Selfmem in fine-tuned small models and few-shot LLMs for three different text generation tasks.\n\nSTRENGTHS:\n1. The motivation of this work is novel. It iteratively employs a retrieval-augmented generator to create an unbounded memory pool and uses a memory selector to select a generated memory for the next round. This enables the model to leverage its own output to improve generation.\n2. The method proposed in this paper has a good effect and has achieved improvement on multiple datasets on three text generation tasks. In addition, this work also verified the proposed method for LLM.\n3. The experimental analysis of this paper is sufficient. In addition to verifying the effectiveness of the proposed method on multiple datasets, this paper also conducts thorough analyses of each component in the Selfmem to identify bottlenecks and provide insights for future research.\n\nWEAKNESSES:\n1. The generality of the method is limited. When the proposed method uses the memory selector to select a candidate from the candidate pool, different metrics are used for different text generation tasks, which limits the generality of the method on different tasks.\n\n2. The cost of the proposed method is large. The Selfmem needs to execute retrieval and generation multiple times until converges. \n\n3. There are some writing problems in this paper:\n- There is an extra word “open” in line 215.\n- Line 141 \\phi(R) -> \\phi(\\mathcal{R}).\n- Equation (4) MemoryEncoder(m).\n- No details about how to generate candidate pool \\mathbb{C}.\n\n4. A generation example can be provided in the experimental results so that readers can feel the effect of the method in this paper more intuitively.\n\nQUESTIONS:\n1. When the proposed method uses the memory selector to select a candidate, different metrics are used for different text generation tasks, which limits the generality of the method on different tasks. Is there a good general method or metric that can be applied to different text generation tasks? How well does the proposed method perform when using a generic method or metric?\n\n2. How to preserve diversity when generating candidate pool \\mathbb{C}?\n\n3. How to draw the conclusion in Line 143 that 'the generator does not need to resort to external memory'?  The conclusion in Line 145 also confuses me a lot, e.g. 'This observation motivates us to select memory according to metrics'. Could you explain in more detail?\n\n4. For tasks such as dialogue generation, this work only uses automatic evaluation, lacking human evaluation. Automated assessments also lack justification.\n\nLIMITATIONS:\nThe authors discuss limitations in the appendix.", "TITLE": "Official Review", "RECOMMENDATION": 5, "REVIEWER_CONFIDENCE": 4, "SOUNDNESS_CORRECTNESS": 3, "DATE": "2023-07-06"}

{"IS_META_REVIEW": false, "comments": "SUMMARY:\nThis paper aims to address the retrieval-augmented text generation problem, where the principle of memory retrieval is to select examples similar to the input. The authors are motivated by an observation in their preliminary experiments that the memory examples that better resemble the data distribution during inference come from the model's output, instead of the training data. Therefore, they propose a novel framework, selfmem to address the retrieval-augmented text generation problem. The proposed framework consists of a memory-augmented generator and a memory selector. Differently, the generator can produce multiple candidates to serve as generated memory instead of retrieved memory from the training corpus. The authors experiment on three generation tasks: machine translation, summarization, and dialogue, and demonstrate performance improvement in various evaluation metrics.\n\nSTRENGTHS:\n1. The paper provides sufficient related work, which is helpful for readers to understand the research context.\n2. The research motivation is well-supported by preliminary experimental results, which is a good starting point for the research.\n3. The paper includes thorough experiments covering representative tasks in the generation field, including machine translation, summarization and dialogue generation, which could be a valuable contribution.\n\nWEAKNESSES:\n1. The novelty of the paper is limited. The authors propose a dual structure: Retrieval-augmented Generator and memory selector, each of which is built upon existing paradigms.\n\n2. The overall paper is not well-presented. \n\n(a)  Many parts of the paper lack clear descriptions. For example, the setting of the preliminary experimentation is unclear at all. \n\nIn L119-120, how does the number 38.89 in Table 1 come from? For example, what does the head of row (Memory, Hypothesis) mean? There is no brief description of this experimental setting in the context.  The results in Table 1 are not clear enough to understand the motivation.\n\n(b) Additionally, there are grammar errors and unclear sentence structures in the paper that hinder the reader's comprehension. \n\nIn L113-114, \"The primary motivation behind our framework stems from the observation that the memory, which is more similar in distribution to the data during inference, is not the training data (38.89 BLEU, as shown in Table 1).\" -> \"The primary motivation behind our framework stems from the observation that the memory is more similar in distribution to the data during inference, not the training data (38.89 BLEU, as shown in Table 1).\"\n\n3. The experimental results are not that convincing as the authors did not provide human evaluation results.  Due to the limitations of automatic evaluation metrics, it is usually necessary to conduct human evaluation to assess the quality of generated output in generation tasks. \n\nIn this paper, the generation is boosted with the examples that meet the automatic evaluation metrics. This naturally leads to an increase in the numerical values of automatic evaluation metrics.  It is still necessary to use human evaluation results to further determine the quality of the generated output.\n\nQUESTIONS:\nSuggestions:\n1. In Figure 2, the shading in the square exceeds the boundaries, which needs to be fixed.\n2. More captions are needed for Figure 2(b) to better illustrate the content.\n\nTypos & Grammar errors:\n1. Please refer to the comments in Weaknesses.\n2. In L215, there is an irrelevant word \"open\" that should be removed.\n\nLIMITATIONS:\nI didn't find the relevant discussion is this paper.", "TITLE": "Official Review", "RECOMMENDATION": 5, "REVIEWER_CONFIDENCE": 4, "SOUNDNESS_CORRECTNESS": 3, "DATE": "2023-07-06"}

{"IS_META_REVIEW": false, "comments": "SUMMARY:\nThe paper proposes an iterative text generation procedure. First, the authors augment the model with retrieval to generate the initial beam of predictions. Then they apply rescoring model (trained separately to maximize BLEU/ROUGE scores) to select the higher-quality predictions. Finally, the authors augment the model with these predictions and perform the next round of generation. The procedure is then repeated multiple times.\n\nSTRENGTHS:\nThe paper performs experiments on multiple benchmarks for machine translation and summarization, showing a strong performance of the method. The authors also experiment with several models, including (1) fine-tuned fusion-in-decoder, (2) fine-tuned fusion-in-decoder where input and retrievals encoded separately, (3) frozen decoder-only LLMs\n\nWEAKNESSES:\nThe paper could be restructured to make it easier to read. Specifically, the paper heavily relies on the notion of \"memory.\" This could be confusing since the paper only briefly explains what specific memory the method is using. For example, do we retrieve parallel sentences from the training data in machine translation? or do we retrieve from the monolingual corpus in source / target languages?\n\nMoreover, I'm not sure I'm a fan of the \"memory\" positioning of the paper. Does it matter that the first prediction is made using retrieval augmented model?\n\nFinally, it's unclear to me how much gains are due to the reward model and how much gains are due to iterative prediction. For example, while we know that doing four iterations works better than 1 -- would you similar gains from doing 1 iteration but with four times as large beam size? This way, the reward model could pick higher-quality predictions.\n\nQUESTIONS:\nWould it make sense to cite related works on the iterative prediction? For example, some of the potential candidates include\nZemlyanskiy et al., Generate-and-Retrieve: use your predictions to improve retrieval for semantic parsing, COLING 2022\nKumar et al., In-context Examples Selection for Machine Translation, arxiv\n\nWhat is the \"Memory\" column in Tables 1 and 3? Does it mean the highest (or average?) BLEU score of the retrieved sample if we use it as a prediction? If that's so, then I'm surprised that such a nearest neighbord -like classifier gets > 38.89 BLEU (Table 1).\n\nLIMITATIONS:\nYes", "TITLE": "Official Review", "RECOMMENDATION": 6, "REVIEWER_CONFIDENCE": 3, "SOUNDNESS_CORRECTNESS": 3, "DATE": "2023-07-07"}