Lift Yourself Up: Retrieval-augmented Text Generation with Self-Memory
Authors: Xin Cheng
Di Luo
Xiuying Chen
Lemao Liu
Dongyan Zhao
Rui Yan
Venue: NeurIPS 2023 poster (2023)
Keywords: natural language processing
retrieval-augmented text generation
self memory

Abstract: With direct access to human-written reference as memory, retrieval-augmented generation has achieved much progress in a wide range of text generation tasks. Since better memory would typically prompt better generation (we define this as primal problem). The traditional approach for memory retrieval involves selecting memory that exhibits the highest similarity to the input. However, this method is constrained by the quality of the fixed corpus from which memory is retrieved. In this paper, by exploring the duality of the primal problem: better generation also prompts better memory, we propose a novel framework, selfmem, which addresses this limitation by iteratively employing a retrieval-augmented generator to create an unbounded memory pool and using a memory selector to choose one output as memory for the subsequent generation round. This enables the model to leverage its own output, referred to as self-memory, for improved generation. We evaluate the effectiveness of selfmem on three distinct text generation tasks: neural machine translation, abstractive text summarization, and dialogue generation, under two generation paradigms: fine-tuned small model and few-shot LLM. Our approach achieves state-of-the-art results in four directions in JRC-Acquis translation dataset, 50.3 ROUGE-1 in XSum, and 62.9 ROUGE-1 in BigPatent, demonstrating the potential of self-memory in enhancing retrieval-augmented generation models. Furthermore, we conduct thorough analyses of each component in the selfmem framework to identify current system bottlenecks and provide insights for future research.

Lift Yourself Up: Retrieval-augmented Text Generation with Self-Memory
Xin Cheng'
Di Luo?
Xiuying Chen
Lemao Liu"
Dongyan Zhao'
Rui Yan?
Peking University
Remin University of China
KAUST
Tencent Al Lab
chengxin| 1998 @ @stu.pstuuuac....e stt..pku.edu.cn
1998
Abstract
With direct access to human-written reference memory, retrieval-augmennte generation has achieved much progress in wide range of text generation tasks. Since better memory would typically prompt better generation (we define this as primal problem). The traditional approach for memory retrieval involves selecting memory that exhibits the highest similarity to the input. However, this method is constrained by the quality of the fixed corpus from which memory is retrieved. In this paper, by exploring the duality of the primal problem: better generation also prompts better memory, we propose a novel framework, Selfmem, which generator
addresses this limitation by iteratively employing to create an unbounded memory pool and using a memory selector to choose one output memory for the subsequent generation round. This enables the model to leverage its own output, referred to as self memory, for improved generation. We evaluate the effectiveness of Selfmer fmem on three distinct text generation tasks: neural machine translation, abstractive text summarization, and dialogue generation, under two generation paradigms: fine-tuned small model and few-shot L.L.. Our approach achieves state-of-the-at results in four directions in JRC-Acquis translation dataset, 50.3 ROUGE-1 in XSum, and 62.9 ROUGE-1 in BigPatent, demonstrating the potential of self-memory in enhancing eetrieval-augmennn generation models. Furthermore, we conduct thorough analyses of each component in the Selfmem framework to identify current system bottlenecks and provide insights for future research
eetrieval-augmeene
Introduction
In recent years, aetreeval-augmentedd text generation has attracted growing interest across various fields, including neural machine translation 28 17 2), dialogue response generation [8T] 46). and language modeling 36 77 [9 This innovative generation paradigm initially equips fine-tuned small model or a large language model (LLM) with access to an external database (typically the training corpus) using information retrieval techniques. Subsequently, the generation process is conducted based on both the input text and the retrieved memory.
In this paradigm, the guiding principle for memory retrieval is find the memory that exhibits the highest similarity to the current input (36] 96 49 This aligns with the human intuition that a more similar demonstration sample typically offers more hints. As demonstrated in Figure [ for -aggenttd translation model, the memory similarity alone exhibits a strong correlation with the final translation quality, regardless of other factors that may influence translation quality (e.g..
Code and data available at: httpp:///ggtauu.oo////maaaaaa//a
37th Conference on Neural Information Processing Systems (NcurIPS 2023).

---

polysemy, morphology, and coreference). We define this as the primal problem: better memory prompts better generation. Consequently, numerous studies have focused on how to retrieve better memory, ranging from sparse retrieval to dense retrieval I0 63 from fixed retriever to learnable retriever
and from sentence-level memory to more fine-grained token-level memory 36
However, a fundamental limitation exists in all previous works: the memory is retrieved from a fixed corpus and is constrained by the corpus's 's quality. Due to the finite retrieval space, bounded memory significantly restricts the potential of models [97]. In this paper, we explore the duality of the primal problem. which posits that better generation also prompts better memory. We propose a novel framework called Selfmem, which iteratively employs a retrieval- -ugmeette generator to create an unbounded memory pool and uses a memory selector to choose one output as memory for the subsequent generation round. By combining the primal and dual problem, a retrieval- augmented generation model can elevate itself using its own output, referred to as self- memory. The key insight behind Selfmem is that the text more closely resembling the data distribution during inference is not the training data 887, but the model's own output.
Hypothesis BLEU
0.6
generation
0.7
02
0.6
0.8
Memory Similarity
Figure 1: Relation between memory and hy- pothesis on JRC-Acquis En-+De dataset. The hypothesis is generated by retrieval- augmented translator whose memory is re- trieved from the training set. The X-axis represents the similarity between memory and the reference.
Sel fmem consists of two complementary components:
a setrivval-augmented generator and a memory selector. The generator operates under two distinct paradigms: fine-tuning small model or few-shot prompting an LLM. For the former, we train the generator with labeled data and retrieved memory, while for the latter, we employ a fixed black-box LLM exclusively for inference alongside retrieved in-context learning samples. We then use the generator's 's output to train a memory selector based on a specific performance metric. By simply replacing the retrieved memory with unbounded generated memory, achieve higher-quality generation output (primal problem), which subsequently serves as memory for the next round after being refined by the memory selector (dual problem).
To evaluate the efficacy of the Sel fmem, we carry out comprehensive experiments in three distinct text generation tasks: neural machine translation, abstractive text summarization, and dialogue generation. We witness substantial enhancements over robust baselines, attaining state-of-the-an outcomes in JRC-Acquis (four directions), XSum (50.3 ROUGE-I), and BigPatent (62.9 ROUGE-I). To gain deeper insights into the Selfmem, we meticulously investigate each crucial component and pinpoint the existing system bottleneck to guide future research endeavors.
2 Related Work
2.1 Retriceal-augmenned Text Generation
Since the world is not a snapshot once the training corpus is collected, we can never expect an ever-large model to capture everything in its parameters, even for LLMs like GPT-4 [62]. Therefore, it is crucial to equip these models with an external memory bank to store additional knowledge or useful demonstration examples for solving various NLP tasks 41 78 95].
In the translation domain, retrieval techniques have long been employed by the localization industry to enhance human translators' productivity and consistency even before the advent of machine translation [94]. Early works on machine translation primarily focused on utilizing memory for statistical machine translation (SMT) systems 80 50]. For neural machine translation (NMT),
(2 were the first to use search engines to retrieve memory from the training set and incorporate with an external memory network. Subsequent research explored various aspects of retrieval- augmented NMT, such as memory encoding methods [92] 93 311, joint training of retrievers and generators with monolingual data R memory granularity 35 and memory diversity [17]. For few-shot LLM generation, strategies for in-context example selection have been proposed to improve translation quality 22. Furthermore, in-context machine translation has been shown to be effective for on-the-fly adaptation (79). For dialogue response generation tasks, employing exemplat/template

---

To investigate the first hypothesis, we conduct experiments under the oracle and random scenarios by using the reference as memory (Reference) and randomly sampled sentences as memory (Random). generator (trained with retrieval as an intermediate step has proven advantageous for generating informative responses 89
91
77. In-context learning example retrieval also aids controllable dialogue 46]. Other
applications include abstractive summarization [64] 14
I8
IS code generation (30, paraphrase
generation 34
language modeling [36 [05], counterfactual data generation open domain
question answering
12 33
and semantic parsing [99].
2.2 Neural Text Reranking
By alleviating the discrepancy between training and inference (i.e., exposure bias) and directly optimizing desired metrics, two-stage reranking methods have facilitated significant progress in various text generation tasks. In machine translation, pioneering works by [75] and [61] introduced and popularized discriminative reranking for SMT. In the context of NMT, research has focused on two primary reranking approaches: generative reranking [56] 32 88) and discriminative reranking [39 71 23).
For syntactic parsing, | were the first employ two-stage reranking method to select outputs from a base parser, while [1] introduced a maximum entropy reranker. In text summarization, RefSum [53] proposed a second-stage summarization framework to address train-test distribution mismatches. SimCLS [34] used pairwise Learning To Rank (LTR) to select candidates with the highest matching scores. SummaReranker (68] adopted a multi-task mixture-of-experts framework to leverage different metrics capturing various aspects of generated candidates. BRIO (55) reused the base model for a second round of fine-tuning with both cross-entropy loss and candidate-level ranking loss. JGR (76] employed an alternate training paradigm to train the generator and reranker.
A key limitation of these reranking methods is that they only represent one process, wherein the selected candidates become the system's 's final output. In contrast, our framework innovatively utilizes the chosen candidates as memory for the subsequent generation round of Strieval-augmentte generator, which can produce better candidates with enhanced memory.
3 Methods
In this section, we begin with a motivating experiment on generation as memory ($ (.I). Then, we
introduce Sel Smem, framework comprising relrieval-uugmentee generator (8 3.2) and memory
selector ($ 3.3) The complete framework and algorithm are illustrated in Figure and Algorithm
3.1
Generation as Memory
The primary motivation behind our framework stems from the observation that the memory, which is more similar in distribution to the data during inference, not the training data (38.89 BLEU, as shown in the first row of (T. Instead, it is the model's 's own output (58.58 BLEU) within the unbounded generation space. One interesting exploration involves directly utilizing the generated output as memory in relation to the primal problem: better memory prompts better generation.
We conduct experiments on the JRC-Acquis En-+ee dataset. The first row in Table II represents conventional retrieval-aummented training with retrieved memory and achieves 58.58 BLEU score. However, directly in- corporating beam output of this trained model as memory (Beam) back into the generation model does not yield any improvements (row 2). despite its higher similarity to the reference compared to the retrieved ones. We hypoth- esize two potential reasons for this: (1) the petrieval-augmente generator may not gen- eralize effectively in this context due to the memory distribution shift (from 38.89 to 58.58), and (2) the beam memory does not offer any information gain compared to the retrieved one, even it exhibits more overlap with the references.
Table 1:
Experiments on the relation between mem- ory quality and the final hypothesis quality, measured by the BLEU score with ground truth translation. The eetrieval-augmenneed translator keeps fixed while the memory is obtained from different sources.
Memory Source
Memory Quality Hypothesis Quality
Retrieval Beam
38.89
58.58
58.58
58.43
Reference
100
90.43
Random
1.14
49.08
The result is shown in Table I and it illustrates that
3

---

Predicted Distribution
memory target
candidates
training
KL Loss
Primal
Frozen LLM Trainable
Retrieval
NLL Loss
2
Target
(a) Retrenedl-uggmented Generator
(b) Memory Selector
augmmeted genera- tor (a) and memory selector (b), For the primal problem, (a) takes source and memory as input to generate candidates for (b). For the dual problem, (b) takes as input source and generated candidates to select memory for (a).
Figure 2: Overall framework. There are two components in Selfmem
retrieved memory) has already learned to discriminate between different memories in both oracle and random scenarios, without updating the model weights.
To evaluate the second conjecture, and beam memory as R, M, and B, respectively. The overlap token set, denoted by O, is defined the tokens that overlap with the references in the beam memory but not in the retrieved memory, which is represented as R.B R r M. O is considered as the additional information provided by the beam memory. Inspired by the confidence analysis of NMT model [58]. compute the set confidence score, (b(.), as follows:
first define the token sets of the reference, retrieved memory,
(()
Σ p(u v<c)) E.E.
where p(yi|zz, Vec)) is defined by the generation model. (b() measures the confidence with which the generation model generates the tokens. The value of (R. 0.58, while that of is 0.76, indicating that the generator is relatively confident in generating tokens in O, and therefore does not need to resort to external memory [38]. Beam search ranks generated candidates based on p(y æ)), where the selected memory falls within the confidence region of the generator and consequently provides information gain. This observation motivates us to select memory according to metrics other than p(y(2=) in the memory selector ($3.3
3.2 Retrieval-augmmented Generator
Given text pair (z, y). where {x1, {xi,...,, is the source, y (y1,..., Vlyl is the target. They could be (document, summary) in summarization, (context, response) in dialogue generation or (source, target) in machine translation. The eetrivval-uugmented generation would first use to retrieve memory m from datastore D. Then the generator Gg(æ, m), parameterized by E. would take both æ and input generate the target sentence y. In this paper, following standard practice,
we choose the training set as D = {x', y For LLM as Gş, we use the standard in-context learning format to give (I. y) as demonstration example. For tunable generator Gg, only keep the target side of top-/ retrieval results as memory and we consider two commonly used architectures: and
Joint-Encoder
29 87
9
Joint-Encoder This architecture is the standard encoder-deccoerr-aase model 3 84
The input is the concatenation of and m. The encoder would first map the input into the hidden states H:
H Encoder(æ [SEP] m)
(2)

---

And the decoder would incorporate H by attention mechanism and generate tokens in an auto- regressive manner: h Decoder(CrossAtt((H)),
Pay (-12,,,, Softmax(h")
Dual-Encoder Instead of treating æ and as a long sequence, this architecture has two encoders, one for z: and the other for m. Their outputs are sequentially attended by the decoder with dual cross attention as in 177
H₂ SourceEncoder(x)) Hm MemoryEncoderr(m)
(4)
h' Pecooer(CrrosAtttn(Ie mm), Mcs)
(5)
We use Transformer 84 as the building block for both architectures and optimize Gg with NLL loss:
M
Lill
log Pa. (Melar, m, yec))
(6)
3.3
Memory Selector
The role of memory selector Sa Sa c), parameterized by 0, is to select one candidate с from the candidate pool C generated by Ge based specific metric A(, The chosen candidate is then utilized as memory m for the subsequent generation round of Gg. As discussed in $3.1 using PG, (y|æz) as the metric (---) would result in falling into the confidence region of Gg. leading to information gain. Moreover, larger value of PG₃ (y/æ:) does not necessarily guarantee improved generation quality [59]. Consequently, we define A(..) as model-free metrics that are widely employed for assessing generation quality, such as BLEU for Neural Machine Translation (NMT) and ROUGE for Summarization. Our memory selector takes the concatenation of the source and candidate G as imput, and produces a multinomial distribution ps. (-|z) over C.
In this paper, we focus on the role of the memory selector, So(r, c), which is parameterized by 6. The objective of this selector choose a single candidate e from the candidate pool C, generated by Gg. based on specific metric, ((:.).
exp(S [SEP] c.)
PS. (c(l)
C Σj exp (x [SEP] (2))
In accordance with [39], the training goal for So is minimize the discrepancy between the So's predictions and the scores determined by ... This divergence quantified using the Kullback- Leibler (KL) divergence.
PM (m:))loggss (c((z)) where PM (c:) = exp( A (C v)/T)
Lu =
Σ 1=1
(8)
E (xp(A(c,.))/T)
T is the temperature to control the smoothness of the distribution. At inference, the output of the So arg max Ps, (c, (c.(ar).
cEc
3.4 Combine Generator and Selector
We define two generation modes for Gç. The first mode, referred to as the hypothesis mode, generates single output for each input, which is utilized for system evaluation. The second mode, known as the candidate mode, produces N outputs for given input, and is employed for training Sg as well as memory selection. By integrating two modes together, we present the complete framework of our proposed model, Selfmem, as illustrated in Algorithm
Experimental Setup
4.1 Dataset
assess the performance of Smlfmem on three generation tasks, utilizing total of seven datasets. Translation. We evaluate framework JRC-As datasets [82] collection of parallel
5

---

Algorithm 1 Selfmem Framework
Require: a dataset D, retriever R. a memory selection metric A At.,), a aurrival-augmente
generator Gg, and memory selector S
1: retrieve memory MI in TD with R
2: train Gg with TD and M (if not L.M)
3: use Gg to generate candidate pool C with M in candidate mode
4: train Sq on C with A(..)
5: while not converged in the validation set do
6: So selects memory from as M
7: Ge generates candidate pool C with Mi in candidate mode
8: end while
9: Gg generates the final hypothesis with MI in hypothesis mode
legislative text of European Union Law. It is the benchmark dataset used in translation memory- augmented NMT task [8 92 I7.. We choose translation directions, namely, Spanish<-English (Es++En), G (De++En). Summarization. We evaluate on 2 summarization datasets: 1) XSum [60], extreme summarization, single-document summarization dataset with highly abstrac- tive articles from British Broadcasting Corporation. 2) BigPatent 773,. consisting of 1.3 million records of U.S. patent documents along with numan-writter abstractive summaries. Dialogue, We experiment on DailyDialog [44), which contains multi-turn dialogs on daily life topics and is used The detailed statistics for these datasets can be found in the Appendix A
by 13 [03].
4.2 Implementation Details
We utilize the BM25 algorithm [70 for retrieval purposes. For all tasks, the candidate generation method consists of beam search with beam width of 50. The number of iterations is determined by the performance on the validation set. For translation, we follow the approach of [93] g 17],
employing a randomly initialized Transformergue architecture as Ge for trainable small model and XGLM [48) for LLM in-context learning. Evaluation metrics include BLEU, TER, and chrF++ obtained from SACREBLEU[66] The memory selector Sy utilizes an XLM-R [22] as back- bone, with BLEU serving -....) For summarization,
initialize Gg with BART buse [40] for BigPatent and employ BRIO 55 for XSum. The evaluation metric comprises ROUGE (R- 1/2/L) [47. For dialogue generation, BART base serves the backbone for Gç. Our dialogue system is evaluated using BLEU (B-1/2) and Distinct (D-1/2) scores [43]. For both dialogue and summariza- tion tasks, we adhere the methods of (54 26). adopting RoBERTasss [52) the backbone for Sg. The linear combination of B-1/2 is chosen as (::-)) for Dialogue Generation, while R- 2-1/ is used for Summarization, following (76). For further implementation details, please refer to the Appendix
B and Appendix C for evaluation metrics.
Experimental Results
5.1 Machine Translation
We select four translation directions and experiment with two generation paradigms: trainable small models and few-shot prompted LLMs 855 20 For trainable models, we explore two architectures (joint and dual, as detailed in $3.2). The baselines comprise two types of translation systems: one being the vanilla seguencc-to-sqqueee consisting of strreval-augmeent
model B 84 without memory augmentation, and the other translation models focusing on memory encoding [28] [22, memory construction [101], memory retrieval (8), and memory diversity [17]. Based on the experimental results?) shown in Table 2 Selfmem significantly enhances the performance of Ge across four translation datasets and two different architectures. This is noteworthy, given that the parameters of the Ge remain fixed, with the only variable being the input memory. This finding is consistent with the primal problem which posits that improved memory typically leads to better generation results.
2 As higher BLEU scores in this range do not necessarily guarantee a superior translation system evaluate system using TER and chrF++. The results can be found in the Appendix D
9 we also

---

Table 2: Results of translation task JRC-Asquis measured by BLEU. Models denoted by the same symbol (* and f) have the same parameters and only differ in memory as input. The bolded numbers show the SOTA performance and the underlined numbers show the second-best result. denotes the system significantly better than baselines with p-value < 0.05 tested by 37
Es En
En-
De-+En
En-+De
System
Dev
Test
Dev
Test
Dev
Test
Dev
Test
None Memory
RNNsearch
55.02
59.34
50.54
50.48
50.20
49.74
44.94
43.98
Transformer
64.08
64.63
62.02
61.80
60.18
60.16
54.65
55.43
Retrieval Memory
SEG-NMT
28
60.28
59.34
$7.62
$7.27
55.63
55.33
49.26
48.80
NMT-pieces 101
63.97
64.30
61.50
61.56
60.10
60.26
55.54
55.14
G-TFM 92
66.37
66.21
62.50
62.76
61.85
61.72
57.43
56.88
MonoNMT
67.73
67.42
64.18
63.86
64.48
64.62
58.77
58.42
CMM 17
67. 48
67.76
63.84
64.04
64.22
64.33
58.94
58.69
dransformoFduul*
66.87
67.12
63.14
63.54
64.09
63.36
58.69
58.06
Transformer,ni
67.74
67.32
63.93
64.12
64.50
64.40
58.16
58.58
Self-Memory
TransformeFanl** 68.63*
69.20*
64.12*
64.67*
65.06*
64.98*
59.26*
59.49*
Transformer,mf
68.26"
68,80*
66.07
65.94"
65.32*
65.65*
59.88
60.11*
Table 3: Comparison between retrieval memory and self-memory. The quality of memory and hypothesis is measured by the n-gram overlap with reference (BLEU). All experiments conducted with Transformer on JRC-Acquis.
Retrieval
Self
memory
hypothesis
memory
hypothesis
38.89
58.58
57.92
60.11
En-De
42.56
64.40
64.32
65.65
40.67
64.12
63.57
65.94
En-Es
43.05
67.32
67.78
68.80
The dual problem is revealed Table 3 Self-memory, which essentially represents the model's own output, exhibits greater similarity with the ground truth and serves more effective memory for generating the final output. This observation highlights a key distinction between Selfmem and previous reranking works [39 68). Reranking aims to select candidates of higher quality than the beam output, whereas Selfmem, the chosen candidates serve as memory for the retrieval-augmentee generator and do not necessarily need to surpass the quality of the beam hypotheses.
Table 4: Evaluation results of in-context learning with self-memory.
XGLM--.7B
XGLM-4.5B
XGLM-7.5B
Random
kNN
Self
Random
kNN
Self
Random
kNN
Self
11.51
37.87
40.94
17.51
37.60
38.25
18.48
47.82
48.32
En-De
27.42
51,00
51.88
30.62
48.12
48.36
33.03
55.65
55.12
23.87
46.20
48.56
31.83
48.37
49.17
29.97
53.86
54.32
En-Es
25.29
51.55
53.13
32.16
48.55
49.22
35.22
57.25
57.56
In 4 we present the results of LLM with self self-memory. We employ XGLM 48 as our backbone generator, with three different sizes ranging from 1.7B to 7.5B. We utilize the recommended prompt as described in [88 We select three in-context learning examples and report the average scores from three separate runs, taking into account the sensitivity of example selection in ICI. [49]. From the table, we first observe general trend where few-shot translation performance improves the
7

---

size of the model increases. Furthermore, we find that more similar translation demonstrations significantly enhance performance across all model sizes (from random, kNN to Self). This suggests that demonstration examples in in-context learning not only act triggers for model ability but also adhere to the primal problem, where better demonstration example leads to better generation. Also, by comparing the results in Table and Table we can conclude that the cross-lingual L.M with designed examples still falls short of the supervised baselines this task.
5.2 Summarization
In this paper, we compare the performance of our trainable model with those of REINA 887 PEGASUS [100], and BART [40]. The results are presented in Table5 Initially, it can be observed different datasets. The enhancement brought by memory the
that memory has varying impacts BigPatent dataset is significantly larger than that in the XSum dataset. This can be attributed to the inherent characteristics of the BigPatent dataset, which consists of official patent documents that exhibit considerable similarity. Consequently, this greatly improves the summarization quality in accordance with the primal problem. Furthermore, we discovered that self self-memory substantially enhances the performance of both BRIO (+1.22 R1) and BART (+18.5 R1), achieving state-of-the-a results on both datasets. We selected these baselines for a fair comparison, as they share the same base generator. Due to space constraints, additional comparisons and the confidence region of the SOTA model can be found in the Appendix B
Table 5: Results of summarization task on XSum and BigPatent measured by ROUGE.
System
Memory
R-1
R-2
R-L
System
Memory
R-1
R-2
R-I.
XSum
BigPatent
PEGASUS
None
47.2
24.6
39.3
PEGASUS
None
53.6
33.2
43.2
BRIO
None
49.1
25.6
40.4
BART
None
44.4
21.3
31.0
REINA (PG)
Retrieval
48.2
26.0
REINA (B)
Retrieval
59.5
42.6
50.6
REINA (B)
Retrieval
43.2
21.0
35.5
REINA (L)
Retrieval
60.7
43.3
51.3
REINA (L)
Retrieval
46.5
24.1
38.6
REINA (PG)
Retrieval
44.6
21.5
33.3
BRIO dual
Retrieval
48.6
26.1
40.6
BART dual
Retrieval
57.4
43.3
49.7
BRIO joint
Retrieval
49.5
26.5
41.2
BART Joint
Retrieval
59.6
43.4
51.0
BRIO dual
Self
49.2
26.2
40.8
BART &Aul
Self
61.2
44.6
52.3
BRIO joint
Self
50.3
26.7
41.6
BART joint"
Self
62.9
48.1
59.6
5.3 Dialogue Generation
As demonstrated in Table the self-memory significantly enhances the performance of the retrieval- augmented generator for dialogue generation tasks. By optimizing memory using BLEU ((, -). the self-memory improves the B-1.2 score over retrieved memory by 3.08 B-1 and 0.6 B-2 on BART joint- Intriguingly, although fmem surpasses the baselines in terms of B- it falls behind in D-1 and D-2, which can be attributed the trade-off between BLEU score and Distinct score when evaluating a dialogue system [104]. To address this issue, we opt for D-1,2 as ((: when optimizing Sp. denoted as BART joint (D). The results Table 6 highlight the remarkable flexibility Selfmem by directly optimizing memory to achieve the desired attributes for diverse and informative dialogue.
6 Further Analysis
To gain deeper insight into Selfmem, first examine the impact of each key component, namely Ge and Sp. Subsequently, we perform a detailed token-level analysis the generated output concerning their frequency in the training set. Experiments conducted the JRC-Acquis En->De dataset. We also include latency analysis and human evaluation on Appendix Fandd
Tuning Sg We explored various S, by direct selection from the candidate pool based on gold rankings. As shown in Figure 3al both architectures with enhanced S₃ significantly outperform the current SOTA performance (60.11 11 BLEU). Moreover, we assessed the candidate pool quality during this iterative process using oracle Sg. displayed in Figure 3b A clear pattern emerges

---

Table 6: Results of dialogue generation task DailyDialog measured by B-1/2 and D-1/2.
BART joint (D) denotes the metric ((,,) for Sg the average of D-1 and D-2.
System
Memory
B-1
B-2
D-1
D-2
NCM
88
None
33.60
26.80
3.00
12.80
iVAE
22
None
30.90
24.90
2.90
25.00
PLATO-2
None
34.80
25.12
3.54
25.11
DialoFlow
45
None
36.17
27.67
4.56
27.12
BART
None
20.72
11.36
3.92
19.44
BART dual*
Retrieval
29.50
21.89
4.74
26.01
BART joint"
Retrieval
36.72
31.55
6.13
35.65
BART dual*
Self
33.43
22.85
4.66
26.16
BART joint?
Self
39.80
32.15
5.84
32.16
BART joint (D)
Self
36.92
32.09
9.12
37.05
Tra Transformer-joint
Transformar-dual
ranks
Hyputheedi BLEU
60
60
Cand Maneldatesssoo BLEU
iteration
Iberation
Reraton
(a) Hypothesis
(b) Candidates
Figure 3: (a) shows generation quality the iteration process with different Sg in both trainable generator architectures. (b) shows candidates quality in the iteration process with an oracle Sg.
in this boxplot, revealing improvements in the oracle, quartile, average. and minimum scores of the candidate pool. These two experiments jointly clarify the Selfmem's underlying intuition: generator profits from superior memory, which can be chosen from its own unbounded output, and subsequently, the generator with improved memory produces a higher-quality candidate pool for the next selection round. Consequently, the model lift itself up.
retrieval-augmentee
Tuning Ge As discussed in $3.1 we demonstrated that trained retrieval-an -aagmeeetd generator, with fixed parameters, possesses the ability distinguish between "good" and "bad" memory. This observation not only justifies our decision to maintain fixed generator within framework but also im- plies that the Gg is not the current bottleneck the Selfmem.
1.00
Saff Retrievol
None
0.85
0.75
0.65
The Least
20%
40%
605
Most
Frequency Analysis We conduct a comprehensive token- level analysis by computing the I-gram Fl scores for generated translations and subsequently categorizing the tokens based on their frequency in the training set. The results are depicted in Figure 4 A noticeable pattern emerges, suggesting that the more frequently a model encounters token during training,
Frequency
Figure 4: 1-gram F1 score sorted by training corpus frequency.
the higher the accuracy of the generated output [022] Moreover, our findings indicate that retrieval- augmented models, partieularly those incorporating self-memory augmentation, exhibit superior performance in handling long-tail tail inputs which are challenges for parametric models (67 57

---

7 Conclusion
For the first time, we investigate the fundamental limitation of bounded memory the current retrieval-augmented literature. We combine the primal and dual problems together and propose Selfmem, general framework for retrieval-augmented text generation by uplifting generation model with its own output. We conduct comprehensive experiments across various text generation tasks and different generation paradigms, including trainable small model and few-shot prompted LLM. We surpass strong baselines and improve the state-of- state-of-the-art performance in serval datasets. We also meticulously investigate each crucial component and pinpoint the existing system bottleneck to guide future research endeavors.
Limitations
We discuss the limitations of our framework follows:
(1) Although Selfmem greatly improves the generation quality compared with other retrieval- augmented generation models, it requires selection process. For large dataset with long context (e.g.. BigPatent), would become a more crucial problem considering the quadratic time complexity of transformer architecture.
computational resources with respect to the memory
(2) This paper proposes general idea for the retrieval-ugmmentedd generation. But we only experiment with transformer-ban based architecture for both generator and memory selector and the architecture of generator and memory selector keeps the same across all text generation tasks. We believe the task-specific design for the model architecture, training objective and generation methods in different text generation scenarios would further improve the performance.
Acknowledgement
This work was supported by the National Key Research and Development Program of China and National Natural Science Foundation of China (NSFC Grant No.62122089). We appreciate the anonymous reviewers for their helpful comments. Dongyan Zhao and Rui Yan the corresponding authors.
References
[1] Armen Aghajanyan, Anchit Gupta, Akshat Shrivastava, Xilun Chen, Luke Zettlemoyer, and Sonal Gupta. Muppet: Massive multi-task representations with pre-finctuning. In Proc. of EMNLP, 2021.
[2] Sweta Agrawal, Chunting Zhou, Mike Lewis, Luke Zettlemoyer, and Marjan Ghazvininejad. In-context examples selection for machine translation. CoRR, 2022.
[3] Dzmitry Bahdanau, Kyunghyun Cho, and Yoshua Bengio. Neural machine translation by jointly learning to align and translate. In Proc. of ICLR, 2015.
[4] Siqi Bao, Huang He, Fan Wang, Hua Wu, and Haifeng Wang. PLATO: pre-trained dialogue generation model with discrete latent variable. In Proc. of ACL, 2020.
[5] Siqi Bao, Huang He, Fan Wang. Hua Wu, Haifeng Wang, Wenquan Wu, Zhen Guo, Zhibin Liu, and Xinchao Xu. PLATO-2: Towards building open-domain chatbot via curriculum learning. In Proc. of ACL Findings, 2021.
[6] Deng Cai, Yan Wang, Wei Bi, Zhaopeng Tu, Xiaojiang Liu, Wai Lam, and Shuming Shi. -xeleoontto-respposse: Dialogue generation guided by retrieval memory. In Proc. of NAACL, 2019.
[7] Deng Cai, Yan Wang. Wei Bi, Zhaopeng Tu, Xiaojiang Liu, and Shuming Shi. Retrieval-guie guided dialogue response generation via matching- tornng-tt-geeeration framework, In Proc. of EMNLP, 2019.
10

---

[8] Deng Cai, Yan Wang, Huayang Li, Wai Lam, and Lemao Liu. Neural machine translation with monolingual translation memory. In Proc. of ACL, 2021.
[9] Chris Callison- Burch, Miles Osborne, and Philipp Koehn. Re-evaluating the role of Bleu in machine translation research. In Proc. of EACI, 2006.
[10] Qian Cao and Deyi Xiong. Encoding gated translation memory into neural machine translation. In Proc. of EMNLP, 2018.
[11] Eugene Charniak and Mark Johnson. Coarse-to-fine n-best parsing and maxent discriminative reranking. In Proc. of ACL, 2005.
[12] Danqi Chen, Adam Fisch, Jason Weston, and Antoine Bordes. Reading wikipedia to answer open-domain questions. In Proceedings of the 55th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), pages 1870-1879, 2017.
[13] Wei Chen, Yeyun Gong, Song Wang, Bolun Yao, Weizhen Qi, Zhongyu Wei, Xiaowu Hu, Bartuer Zhou, Yi Mao, Weizhu Chen, Biao Cheng, and Nan Duan. Dialogved: A pre-trained latent variable encoder-decoder model for dialog response generation. In Proc. ACI, 2022.
[14] Xiuying Chen, Hind Alamro, Mingzhe Li, Shen Gao, Rui Yan, Xin Gao, and Xiangliang Zhang. Target-aware abstractive related work generation with contrastive learning. In Proc. of SIGIR, 2022.
[15] Xiuying Chen, Mingzhe Li, Shen Gao, Xin Cheng, Qiang Yang, Qishen Zhang, Xin Gao, and Xiangliang Zhang. topie-aware summarization framework with different modal side information. Proc. of SIGIR, 2023.
[16] Xiuying Chen, Mingzhe Li, Xin Gao, and Xiangliang Zhang. Towards improving faithfulness in abstractive summarization. In Proc. of NeurlPS, 2022.
[17] Xin Cheng, Shen Gao, Lemao Liu, Dongyan Zhao, and Rui Yan. Neural machine translation with contrastive translation memories. In Proc. of EMNLP, 2022.
[18] Xin Cheng, Shen Gao, Yuchi Zhang, Yongliang Wang. Xiuying Chen, Mingzhe Li, Dongyan Zhao, and Rui Yan. Towards personalized review summarization by modeling historical reviews from customer and product separately. arXiv preprint arXiv:2301. 1682, 2023.
[19] Xin Cheng, Yankai Lin, Xiuying Chen, Dongyan Zhao, and Rui Yan. Decouple knowledge from paramters for plug-and-pla, language modeling. In Findings of the Association for Computational Linguistics: ACL 2023, pages 14388-14308, Toronto, Canada, July 2023. Association for Computational Linguistics.
[20] Xin Cheng, Xun Wang, Tao Ge, Si-Qing Chen, Furu Wei, Dongyan Zhao, and Rui Yan. Scale: Synergized collaboration of asymmetric language translation engines, 2023.
[21] Michael Collins and Terry Koo. Discriminative reranking for natural language parsing. Comput. Linguistics, 2005.
[22] Alexis Conneau, Kartikay Khandelwal, Naman Goyal, Vishrav Chaudhary, Guillaume Wenzek, Francisco Guzmán, Edouard Grave, Myle Ott, Luke Zettlemoyer. and Veselin Stoyanov. Unsupervised cross-lingual representation learning scale. In Proc. of ACL, 2020.
[23] Yuntian Deng, Anton Bakhtin, Myle Ott, Arthur Szlam, and Marc' Aurelio Ranzato, Residual energy-based models for text generation. In Proc. of ICLR, 2020.
[24] Tanay Dixit, Bhargavi Paranjape. Hannaneh Hajishirzi, and Luke Zettlemoyer. CORE: A retrieve-then-edit framework for counterfactual data generation. Proc. of EMNLP Findings, 2022.
[25] Le Fang, Chunyuan Li, Jianfeng Gao, Wen Dong, and Changyou Chen. Implicit deep latent variable models for text generation. In Proc. of EMNLP, 2019.
11

---

[26] Jiazhan Feng, Chongyang Tao, Zhen Li, Chang Liu, Tao Shen, and Dongyan Zhao. Reciprocal learning of knowledge retriever and response ranker for knowledge-grouneg conversations. In Proc. of COLING, 2022.
[27] Tingchen Fu, Xueliang Zhao, Chongyang Tao, Ji-Rong Wen, and Rui Yan. There are thousand hamlets in a thousand people's eyes: Enhancing snowledge-grounded dialogue with personal memory. In Proc. of ACL, 2022.
[28] Jiatao Gu, Yong Wang, Kyunghyun Cho, and Victor O. K. Li. Search engine guided neural machine translation. Proc. of AAAI, 2018.
[29] Kelvin Guu, Kenton Lee, Zora Tung, Panupong Pasupat, and Ming-Wor Wei Chang. Retrieval augmented language model pre-training. In Proc. of ICMI,, 2020.
[30] Tatsunori B. Hashimoto, Kelvin Guu, Yonatan Oren, and Percy Liang. retrieve-and-edit framework for predicting structured outputs. In Proc. of NeurIPS, 2018.
[31] Qiuxiang He, Guoping Huang, Qu Cui, Li Li, and Lemao Liu. Fast and accurate neural machine translation with translation memory. In Proc. of ACL, 2021.
[32] Kenji Imamura and Eiichiro Sumita. Ensemble and reranking: Using multiple models in the NICT-2 neural machine translation system WAT2017. In Proceedings of the 4th Workshop on Asian Translation, IAT& 2017, Taipei, Taiwan, November 27- December 1, 2017, 2017.
[33] Gautier Izacard and Edouard Grave. Leveraging passage retrieval with generative models for open domain question answering. Proceedings of the 16th Conference of the European Chapter of the Association for Computational Linguistics: Main Volume, pages 874-880, Online, April 2021. Association for Computational Linguistics.
[34] Amirhossein Kazemnejad, Mohammadreza Salehi, and Mahdieh Soleymani Baghshah. Para- phrase generation by learning how to edit from samples. In Proc. of ACL, 2020.
[35] Urvashi Khandelwal, Angela Fan, Dan Jurafsky, Luke Zettlemoyer, and Mike Lewis. Nearest neighbor machine translation. In Proc. of ICLR, 2021.
[36] Urvashi Khandelwal, Omer Levy, Dan Jurafsky, Luke Zettlemoyer, and Mike Lewis. Gen- eralization through memorization: Nearest neighbor language models. In Proc. of ICLR, 2020.
[37] Philipp Koehn. Statistical significance tests for machine translation evaluation. In Proc. of EMNLP, 2004.
[38] Ankit Kumar, Ozan Irsoy, Peter Ondruska, Mohit lyyer, James Bradbury, Ishaan Gulrajani, Victor Zhong, Romain Paulus, and Richard Socher. Ask me anything: Dynamic memory networks for natural language processing. In Proc. of ICML, 2016.
[39] Ann Lee, Michael Auli, and Mare' Aurelio Ranzato. Discriminative reranking for neural machine translation. In Proc. of ACI,, 2021.
[40] Mike Lewis, Yinhan Liu, Naman Goyal, Marjan Ghazvininejad, Abdelrahman Mohamed, Omer Levy, Veselin Stoyanov, and Luke Zettlemoyer. BART: denoising pre-training for natural language generation, translation, and comprehension. In Proc. of ACL, 2020.
[41] Patrick H. Lewis, Ethan Perez, Aleksandra Piktus, Fabio Petroni, Vladimir Karpukhin, Naman Goyal, Heinrich Küttler, Mike Lewis, Wen-tau Yih, Tim Rocktäschel, Sebastian Riedel, and Douwe Kiela. eetrieval-augmeent generation for knowledge inoeledge-intensive NLP tasks. In Proc. of NeurlPS, 2020.
[42] Jinpeng Li, Yingee Xia, Rui Yan, Hongda Sun, Dongyan Zhao, and Tie-Yan Liu. Stylized dialogue generation with multi-pass dual learning. In Proc. of NeurIPS, 2021.
[43] Jiwei Li, Michel Galley, Chris Brockett, Jianfeng Gao, and Bill Dolan. A inversity-promoting objective function for neural conversation models. In Proc. of NAACL, 2016.
12

---

[44] Yanran Li. Hui Su, Xiaoyu Shen. Wenjie Li, Ziqiang Cao, and Shuzi Niu. Dailydialog: A manually labelled multi-turn dialogue dataset. In Proceedings of the Eighth International Joint Conference on Natural Language Processing, IJCNLP 2017, Taipei, Taiwan, November 27 December I, 2017- Volume I: Long Pupers, 2017.
[45] Zekang Li, Jinchao Zhang, Zhengcong Fei, Yang Feng, and Jie Zhou. Conversations are not flat: Modeling the dynamic information flow across dialogue utterances. In Proc. of ACI, 2021.
[46] Zekun Li, Wenhu Chen, Shiyang Li, Hong Wang, Jing Qian, and Xifeng Yan. Controllable dialogue simulation with in-context learning. In Proc. of EMNLP Findings, 2022.
[47] Chin- Chin-Yew Lin. ROUGE: A package for automatic evaluation of summaries. In Text Summa- rization Branches Out, 2004.
[48] Xi Victoria Lin, Todor Mihaylov, Mikel Artetxe, Tianlu Wang. Shuohui Chen, Daniel Simig, Myle Ott, Naman Goyal, Shruti Bhosale, Jingfei Du, Ramakanth Pasunuru, Sam Shleifer, Punit Singh Koura, Vishrav Chaudhary, Brian O' O'Horo, Jeff Wang. Luke Zettlemoyer, Zornitsa Kozareva, Mona T. Diab, Veselin Stoyanov, and Xian Li. Few-shot learning with multilingual generative language models. In Proc. of EMNLP, 2022.
[49] Jiachang Liu, Dinghan Shen, Yizhe Zhang, Bill Dolan, Lawrence Carin, and Weizhu Chen. What makes good in-context examples for 3pt-3? In Proceedings of Deep Learning Inside Out: The 3rd Workshop Knowledge Extraction and Integration for Deep Learning Architectures, DeelIO@ ACL 2022, Dublin, Ireland and Online, May 27, 2022, 2022.
[50] Lemao Liu, Hailong Cao, Taro Watanabe, Tiejun Zhao, Mo Yu, and Conghui Zhu. Locally training the log-linear model for SMT. In Proc. of EMNLP, 2012.
[51] Yang Liu and Mirella Lapata. Text summarization with pretrained encoders. In Proc. of EMNLP, 2019.
[52] Yinhan Liu, Myle Out, Naman Goyal, Jingfei Du, Mandar Joshi, Danqi Chen, Omer Levy, Mike Lewis, Luke Zettlemoyer, and Veselin Stoyanov. Roberta: A robustly optimized BERT pretraining approach. CoRR, 2019.
[53] Yixin Liu, Zi-Yi Dou, and Pengfei Liu. Refsum: Refactoring neural summarization. In Proc. of NAACL, 2021.
[54] Yixin Liu and Pengfei Liu. Simels: A simple framework for contrastive learning of abstractive summarization. Proc. of ACL. 2021.
[55] Yixin Liu, Pengfei Liu, Dragomir R. Radev, and Graham Neubig. BRIO: bringing order to abstractive summarization. In Proc. of ACL, 2022.
[56] Yuchen Liu, Long Zhou, Yining Wang, Yang Zhao, Jiajun Zhang, and Chengqing Zong. A comparable study on model averaging, ensembling and reranking in NMT. In Proc. of NLPCC, 2018.
[57] Alexander Long, Wei Yin, Thalaiyasingam Ajanthan, Vu Nguyen, Pulak Purkait, Ravi Garg, Alan Blair, Chunhua Shen, and Anton van den Hengel. Retrieval augmented classification for long-tail visual recognition. In Proc. of CVPR, 2022.
[58] Yu Lu, Jiali Zeng, Jiajun Zhang, Shuangzhi Wu, and Mu Li. Learning confidence for transformer-based neural machine translation. Proc. ACI, 2022.
[59] Clara Meister, Ryan Cotterell, and Tim Vieira. If beam search is the answer, what was the question? In Proc. of EMNLP, 2020.
[60] Shashi Narayan, Shay B. Cohen, and Mirella Lapata. Don't give me the details, just the summary! topic-aware convolutional neural networks for extreme summarization. In Proc. of EMNLP, 2018.
13

---

[61] Franz Josef Och, Daniel Gildea, Sanjeev Khudanpur, Anoop Sarkar, Kenji Yamada, Alexan- der M. Fraser, Shankar Kumar, Libin Shen, David Smith, Katherine Eng, Viren Jain, Zhen Jin, and Dragomir R. Radev. A smorgashord of features for statistical machine translation. In Proc. of NAACI, 2004.
[62] OpenAL. GPT-4 technical report. CoRR, 2023.
[63] Md. Rizwan Parvez, Wasi Uddin Ahmad, Saikat Chakraborty, Baishakhi Ray, and Kai- Chang. Retrieval augmented code generation and summarization. In Proc. of EMNLP Findings, 2021.
[64] Hao Peng, Ankur P. Parikh, Manaal Faruqui, Bhuwan Dhingra, and Dipanjan Das. Text generation with exemplar-based adaptive decoding. In Proc. of NAACI, 2019.
[65] Jonathan Pilault, Raymond Li, Sandeep Subramanian, and Chris Pal. On extractive and abstractive neural document summarization with transformer language models. In Proc. of EMNLP, 2020.
[66] Matt Post. A call for clarity in reporting BLEU scores. In Proceedings of the Third Conference on Machine Translation: Research Papers, WMT 2018, Belgium, Brussels, October 31 November 1, 2018, 2018.
[67] Vikas Raunak, Siddharth Dalmia, Vivek Gupta, and Florian Metze. On long-tailed phenomena in neural machine translation. In Proc. of EMNLP Findings, 2020.
[68] Mathieu Ravaut, Shafiq R. Joty, and Nancy F. Chen. Summareranker: A multi-task mixture-of- experts re-ranking framework for abstractive summarization. In Proc. of ACL, 2022.
[69] Mathieu Ravaut, Shafiq R. Joty, and Nancy F. Chen. Towards summary candidates fusion. CoRR, 2022.
[70] Stephen E. Robertson and Hugo Zaragoza. The probabilistic relevance framework: BM25 and beyond. Found. Trends Inf. Retr. 2009,
[71] Julian Salazar, Davis Liang, Toan Q. Nguyen, and Katrin Kirchhoff. Masked language model scoring. In Proc. of ACL, 2020.
[72] Rico Sennrich, Barry Haddow, and Alexandra Birch. Neural machine translation of rare words with subword units. In Proc. of ACL, 2016.
[73] Eva Sharma, Chen Li, and Lu Wang. BIGPATENT: A large-scale dataset for abstractive and coherent summarization. Proc. of ACL. 2019.
[74] Noam Shazeer and Mitchell Stem. Adafactor: Adaptive learning rates with sublinear memory cost. In Proc. of ICML, 2018.
[75] Libin Shen, Anoop Sarkar, and Franz Josef Och. Discriminative reranking for machine translation. In Proc. of NAACL,2004
[76] Weizhou Shen, Yeyun Gong, Yelong Shen, Song Wang, Xiaojun Quan, Nan Duan, and Weizhu Chen. Joint generator-ranker learning for natural language generation. CoRR, 2022.
[77] Weijia Shi, Sewon Min, Michihiro Yasunaga, Minjoon Seo, Rich James, Mike Lewis, Luke Zettlemoyer, and Wen tau Yih. Replug: eetrieval-augmennt black-box language models, 2023.
[78] Weijia Shi, Sewon Min, Michihiro Yasunaga, Minjoon Seo, Rich James, Mike Lewis, Luke Zeulleooyrr, and Wen-tau Yih. REPLUG: retrieval- -uumenttd black-box language models. CoRR, 2023.
[79] Suzanna Sia and Kevin Duh. In-context learning as maintaining coherency: A study of on-the-fly machine translation using large language models. CoRR, 2023.
14

---

[80] Michel Simard and Pierre Isabelle. Phrase- based machine translation in computer-assister translation environment. In Proceedings of Machine Translation Summit XII: Papers, MTSum- mit 2009, Otlawa, Canada, August 26-30, 2009, 2009.
[81] Yiping Song, Rui Yan, Xiang Li, Dongyan Zhao, and Ming Zhang. Two are better than one: An ensemble of retrieval- and generation-based dialog systems. CoRR, 2016.
[82] Ralf Steinberger, Bruno Pouliquen, Anna Widiger, Camelia Ignat, Tomaz Erjavec, Dan Tufis, and Dániel Varga. The jrc-aequis: A multilingual aligned parallel corpus with 20+ languages. In Proc. of LREC, 2006.
[83] Yixuan Su, David Vandyke, Simon Baker, Yan Wang, and Nigel Collier. Keep the primary, rewrite the secondary: A two-stage approach for paraphrase generation. In Proc. of ACL Findings, 2021.
[84] Ashish Vaswani, Noam Shazeer, Niki Parmar, Jakob Uszkoreit, Llion Jones, Aidan N. Gomez, Lukasz Kaiser, and Illia Polosukhin. Attention is all you need. In Proc. of NeurIPS, 2017..
[85] David Vilar, Markus Freitag, Colin Cherry, Jiaming Luo, Viresh Ratnakar, and George Foster. Prompting palm for translation: Assessing strategies and performance, 2023.
[86] Oriol Vinyals and Quoc V. Le. neural conversational model. CoRR, 2015.
[87] Shuohang Wang, Yichong Xu, Yuwei Fang, Yang Liu, Siqi Sun, Ruochen Xu, Chenguang Zhu, and Michael Zeng. Training data is more valuable than you think: A simple and effective method by retrieving from training data. In Proc. of ACL, 2022.
[88] Yuguang Wang, Shanbo Cheng, Liyang Jiang, Jiajun Yang, Wei Chen, Muze Li, Lin Shi, Yanfeng Wang, and Hongtao Yang. Sogou neural machine translation systems for WMT17. In Proceedings of the Second Conference on Machine Translation, WMT 2017, Copenhagen, Denmark, September 7-8, 2017, 2017.
[89] Jason Weston, Emily Dinan, and Alexander H. Miller. Retrieve and refine: Improved sequence generation models for dialogue. In Proceedings of the 2nd International Workshop on Search- Oriented Conversational AI, SCAI@ EMNLP 2018, Brussels, Belgium, October 31, 2018, 2018.
[90] Wenhao Wu, Wei Li, Xinyan Xiao, Jiachen Liu, Ziqiang Cao, Sujian Li, Hua Wu, and Haifeng Wang. BASS: boosting abstractive summarization with unified semantic graph. In Proc. of ACL, 2021.
[91] Yu Wu, Furu Wei, Shaohan Huang, Yunli Wang, Zhoujun Li, and Ming Zhou. Response generation by context-aware prototype editing. In Proc. of AAAI, 2019.
[92] Mengzhou Xia, Guoping Huang, Lemao Liu, and Shuming Shi. Graph based translation memory for neural machine translation. In Proc. of AAAI, 2019.
[93] Jitao Xu, Josep Maria Crego, and Jean Senellart. Boosting neural machine translation with similar translations. In Proc. of ACL, 2020.
[94] Masaru Yamada. The effect of translation memory databases on productivity. Translation research projects, 2011.
[95] Michihiro Yasunaga, Armen Aghajanyan, Weijia Shi, Rich James, Jure Leskovec, Percy Liang, Mike Lewis, Luke Zettlemoyer, and Wen-tau Yih. ootrieval-augment multimodal language modeling. CoRR, 2022.
[96] Dani Yogatama. Cyprien de Masson d' Autume, and Lingpeng Kong. Adaptive semiparametric language models. Trans. Assoc. Comput. Linguistics, 2021.
[97] Wenhao Yu, Dan Iter, Shuohang Wang, Yichong Xu, Mingxuan Ju, Soumya Sanyal, Chenguang Zhu, Michael Zeng, and Meng Jiang. Generate rather than retrieve: Large language models are strong context generators, 2023.
15

---

[98] Manzil Zaheer, Guru Guruganesh, Kumar Avinava Dubey, Joshua Ainslie, Chris Alberti, Santiago Ontanon, Philipp Pham, Anirudh Ravula, Qifan Wang, Li Yang, and Amr Ahmed. Big bird: Transformers for longer sequences. In Proc. of NeurIPS, 2020.
[99]
Yury Zemlyanskiy, Michiel de Jong, Joshua Ainslie, Panupong Pasupat, Peter Shaw, Linlu Qiu, Sumit Sanghai, and Fei Sha. Generate-ande red-rett Use your predictions to improve retrieval for semantic parsing. In Proceedings of the 29th International Conference Computational Linguistics, pages 4946-4951, Gycongju, Republic Korea, October 2022. International Committee on Computational Linguistics.
[100] Jingqing Zhang, Yao Zhao, Mohammad Saleh, and Peter J. Liu. PEGASUS: pre-training with extracted gap-sentences for abstractive summarization, In Proc. of ICML, 2020.
[101] Jingyi Zhang, Masao Utiyama, Eiichiro Sumita, Graham Neubig, and Satoshi Nakamura. Guiding neural machine translation with retrieved translation pieces. In Proc. of NAACL, 2018.
[102] Tong Zhang, Wei Ye, Baosong Yang, Long Zhang, Xingzhang Ren, Dayiheng Liu, Jinan Sun, Shikun Zhang, Haibo Zhang, and Wen Zhao. Frequency-awaid contrastive learning for neural machine translation. Proc. of AAAI, 2022.
[103] Xucliang Zhao, Lemao Liu, Tingchen Fu, Shuming Shi, Dongyan Zhao, and Rui Yan. Towards efficient dialogue pre-training with transferable and interpretable latent structure. CoRR, 2022.
[104] Yinhe Zheng, Zikai Chen, Rongsheng Zhang, Shilei Huang, Xiaoxi Mao, and Minlie Huang. Stylized dialogue response generation using stylized unpaired texts. In Proc. of AAAI, 2021.
[105] Zexuan Zhong, Tao Lei, and Danqi Chen. Training language models with memory augmenta- tion. CoRR, 2022.
16

---

A Dataset Details
Table 7: Dataset statistics for three tasks.
Task
Dataset
#Train
#Dev
#Test
JRC (en ++ de)
663,487
2,454
2,483
Translation
JRC (en < es)
653,127
2,533
2,596
BigPatent
1,207,222 204,045
67,068
67,072
Summarization
XSum
11,332
11,334
Dialogue
DailyDialog
87,170
8,069
7,740
B Self Memory Details
For machine translation tasks, following 93 8 17
we use randomly initialize Transformenhues
architecture (84) as Gg. We the joint-bpe algorithm [72 and share the parameters between the memory encoder and source encoder for dual encoder architecture. The hyper-parameter setting follows [17] with dropout 0.1, label smoothing 0. gradient clipping 1.0, Adafactor (74), warm-up steps 4000, maximum learning rate 4.4c-2 and training epochs 30 for total. The evaluation metrics are BLEU, TER and chrF++ from SACREBLEU [66]. The backbone of memory selector Sa is XLM-R (2) with BLEU as -:: The hyper-pan eyper-parameter setting for So follows [9 with r 0.5, minmax normalization for candidates ranking, Adam optimizer with max learning rate 5e-5 and polynomial decay scheduler, and classifier dropout 0.2.
For Summarization, init the G5 with BART base 440] for BigPatent following 87 and state-of-the- art BRIO (55) for XSum. Optimization is based on Adafactor with a maximum learning rate of 5e-3, warm-up steps 10000 and gradient clipping value 1.0. The maximum input length is 512 for XSum and 1024 for BigPatent. The evaluation metric is Rouge (R-1/2/L) [47].
For Dialogue Generation, use BART base as the backbone for Ge DailyDialog. We tune the hyper-parameters from learning rate smoothing factor 0.1, maximum input length 120 for DailyDialog. Following [4] [3], we evaluate our dialogue system with BLEU (B-1/2) and Distinct (D-1,2) [43] For both Summarization and Dialogue Generation task, follow [54] 26 and adopt RoBERTa [52 the backbone for Sg. We choose the linear combination of B-1/2 as ((---) for Dialogue Generation and R-1/2/L for Summarization following (66.. We tune the hyper-parameters from (0.08.0.2,0.5,0.33 learning rate from 55-5,7e-5,2e-44 The maximum input length for Sp is 512 and we truncate tokens from the longer input of source and candidate.
and set dropout 0.1, batch size 64, label
C Evaluation Details
Machine Translation We evaluate our MT system with BLEU, TER and chrF++ from SACRE-
BLEU (6). The signatures for BLEU, TER and chrF++ are shown in Table
Table 8: Signature from SACREBLEU.
[c]Signature
nrefs: htt.ll/riiiiddeiioooee/aaaaaeaem
nrefs: neeeHoaa..ccccccoommmmmmooaaaaaa
nrefs: htt.://eeeeneneeeeeeeeeeeeeeeeee
17

---

Summarization We evaluate our Summarization system with standard ROUGE [47] Perl package for evaluation. Following [55, we use PTB tokenizer for tokenization. And the parameters for ROUGE are "-c 95 -r 1000 -n -m".
Dialogue Generation Following (27] we evaluate our dialogue system with NLTK BLEU with space as tokenizer and smoothing method1. The Distinction score is from 42
D More results on translation tasks
Table 9: Evaluation results on JRC-Acquis En->De measured by BLEU, TER and chrF+++
System
Memory
BLEU 1
chrF++
TER
Transformer
None
55.43
70.31
36.35
Transformer
Retrieval
58.06
71.58
35,41
Transformerjoiat
Retrieval
58.58
72.22
34.39
TransformeTaual
Self Self
59.49
72.62
34.04
Transformerjoin
60.11
73.25
32.62
E More Summarization Baselines
In this Table TO include baselines the benchmark dataset XSum and BigPatent. We also report the confidence region of SOTA model for XSum and BigPatent as shown in Table
Table 10: More baselines on XSum and BigPatent.
System
R-1
R-2
R-L
System
R-1
R-2
R-L
XSum
BigPatent
5T
38.8
16.5
31.3
40
45.1
22.3
37.3
100
53.6
33.1
42.3
100
47.2
24.6
39.3
40
44.4
21.3
31.0
54
47.6
24.6
39.4
98
60.6
42.5
50.0
155
49.1
25.6
40.4
65
38.7
12.3
34.1
87 (PG)
48.2
26.0
40.2
900
45.0
20.3
39.2
88 (B)
43.1
21.0
35.5
1
52.3
33.5
42.8
87 (L)
46.5
24.1
38.6
87
(B)
59.5
42.6
50.6
48.1
25.0
40.0
(L)
60.7
43.3
51.3
69
47.1
24.1
38.8
87
(PG)
44.6
21.5
33.3
16
47.8
25.0
39.7
Selfmem
62.9
48.1
59.6
Selfmem
50.3
26.7
41.6
F Empirical analysis of latency
In Table 12 we present empirical results Sel fmem latency, measured in seconds. We compare Selfmem with retrieval-augmented baseline model across various datasets and computational platforms, including CPU and CUDA. The number iterations for Selfmem is set to one. All experiments conducted the same device, equipped with one NVIDIA A100 GPU and one AMD EPYC 7V13 64-Core Processor.
https://nlp.. http:://...aaaaaecce//////////aa PIBTokenizer.hta
https://www mltk. http:::/www.llcooog//mmmoooooee
18

---

Table 11: Confidence region for SOTA model in XSum and BigPatent.
System
ROUGE-1/2/L
95 %
XSum
50.3
0.49986 0.50602
BRIO joint
26.7
0.26300 0.26989
41.6
0.41231 0.41900
BigPatent
62.9
0.62664 0.63080
BART joint
48.1
0.47783 -0.48333
59.6
0.59401 0.59847
Table 12: Generation Latency analysis.
NMT
XSum
BigPatent
MailyDialog
Average Input Length
87
512
1024
71
Average Output Length
44
75
127
16
CPU
Retrieval-augmmeneed Baseline
0.97
1.79
3.16
0.32
Candidate Generation
3.20
7.50
15.00
1.02
Selfmem
Memory Selection
0.50
0.52
0.95
0.14
Hypothesis Generation
0.97
1.79
3.00
0.32
x4.80
x5.47
x6.04
x4.63
CUDA
Retreeval-augmeened Baseline
0.29
0.44
0.75
0.10
Candidate Generation
0.51
1.00
1.72
0.18
Selfmem
Memory Selection
0.01
0.01
0.01
0.01
Hypothesis Generation
0.29
0.44
0.75
0.10
X2.76
x2.99
x3.35
x2.91
19

---

G Human and GPT-4 Evaluation
We employ both human annotators and GPT-4 (gpt-4-0314) annotators perform pairwise ranking of the output generated by Selfmem and baseline systems. For GPT-4 annotators, we utilize the prompt from Alpaca Eval We randomly select 50 samples for translation tasks and 20 samples for summarization and dialogue tasks. The win rate of Selfmem versus eetreval-auggmente baselines is depicted Figure I.
Human and GPT-4 Evaluation Results
90
88.0
GPT-4
Human Annotator
Win Rate
80
80.0
76.0
75.0
70
65.0
60
60.0
En-De
En-Es
XSum
DailyDialog
Figure 5: Human and GPT-4 evaluation results.
txt
20


Reviews:
[{"review_id": "jg0jENImOp", "reviewer": "Reviewer_LVYL", "summary": "The paper proposes an iterative text generation procedure. First, the authors augment the model with retrieval to generate the initial beam of predictions. Then they apply rescoring model (trained separately to maximize BLEU/ROUGE scores) to select the higher-quality predictions. Finally, the authors augment the model with these predictions and perform the next round of generation. The procedure is then repeated multiple times. ", "questions": "Would it make sense to cite related works on the iterative prediction? For example, some of the potential candidates include\nZemlyanskiy et al., Generate-and-Retrieve: use your predictions to improve retrieval for semantic parsing, COLING 2022\nKumar et al., In-context Examples Selection for Machine Translation, arxiv\n\nWhat is the \"Memory\" column in Tables 1 and 3? Does it mean the highest (or average?) BLEU score of the retrieved sample if we use it as a prediction? If that's so, then I'm surprised that such a nearest neighbord -like classifier gets > 38.89 BLEU (Table 1).", "limitations": "Yes", "rating": 6, "confidence": 3, "soundness": 3, "presentation": 2, "contribution": 3, "strengths": "The paper performs experiments on multiple benchmarks for machine translation and summarization, showing a strong performance of the method. The authors also experiment with several models, including (1) fine-tuned fusion-in-decoder, (2) fine-tuned fusion-in-decoder where input and retrievals encoded separately, (3) frozen decoder-only LLMs", "weaknesses": "The paper could be restructured to make it easier to read. Specifically, the paper heavily relies on the notion of \"memory.\" This could be confusing since the paper only briefly explains what specific memory the method is using. For example, do we retrieve parallel sentences from the training data in machine translation? or do we retrieve from the monolingual corpus in source / target languages?\n\nMoreover, I'm not sure I'm a fan of the \"memory\" positioning of the paper. Does it matter that the first prediction is made using retrieval augmented model?\n\nFinally, it's unclear to me how much gains are due to the reward model and how much gains are due to iterative prediction. For example, while we know that doing four iterations works better than 1 -- would you similar gains from doing 1 iteration but with four times as large beam size? This way, the reward model could pick higher-quality predictions.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "Firstly, we would like to express our sincere gratitude for the time and effort you have dedicated to reviewing our paper. We will address each comment in a point-by-point manner.\n- Comment 1: Regarding the explanation and positioning of memory\n- Response 1: We maintain that memory plays a crucial role in our system and deserves emphasis. To illustrate this, let us consider a machine translation task as an example. In the first step, we perform a retrieval operation in bilingual corpora, using the retrieved sentence (target side) as our initial memory, as outlined in line 1 of Algorithm 1 and lines 154-155 of the paper. Subsequently, we employ the retrieved memory to train a memory-augmented generator, enabling us to leverage the selected memory in the subsequent iterative process. While it is not essential for the first prediction to be made using a retrieval-augmented model, it is indeed crucial for us to have a retrieval-augmented model to utilize the first prediction (after selection by the memory selector).\n\n---\n- Comment 2: Would you achieve similar gains from conducting one iteration but with a beam size four times larger?\n- Response 2: We acknowledge the possibility of further improvement by increasing the candidate pool size within a single iteration round. However, we believe that such a setting may not be highly relevant to the main contribution of our paper. As demonstrated in Figure 2 of [1] and Figure 2 of [2], it is a well-established conclusion that a larger candidate pool results in higher performance for a reward model. Our paper distinctly demonstrates that the iterative process within the Selfmem framework can also yield superior outcomes. As illustrated in Figure 3(b) of our paper, the quality of the candidate set (measured in terms of oracle, quartile, average, and minimum scores) consistently improves throughout the iteration process. This result could not be attained solely by increasing the candidate pool size. The core essence of the framework lies in the steady improvement between the generator and reward model. We argue that an expanded candidate set can be viewed as an enhancement of our framework without changing its fundamental nature. Consequently, the potential outcome of an enlarged candidate pool would manifest as an overall upward shift in Figure 3(b).\n\n---\n\n- Comment 3: Missing reference\n- Response 3: We appreciate your attention to this matter. We concur that the two references you mentioned are indeed relevant to our paper, particularly the first one by Zemlyanskiy et al. We will ensure their inclusion in the final version of our paper.\n\n---\n\n- Comment 4: Memory column in Table 1 and 3\n- Response 4: The memory columns featured in Tables 1 and 3 display the average BLEU scores calculated from the retrieved samples. To mitigate any potential confusion, we will enhance these tables with more detailed captions. The high BLEU score achieved by a nearest-neighbor-like classifier can primarily be attributed to the dataset we have utilized. Our choice is the JRC-Acquis dataset, which consists of parallel legislative texts pertaining to European Union law and applicable to its member states. Owing to its highly relevant and well-structured data, this corpus serves as an exemplary testbed for assessing memory-augmented neural machine translation (NMT) systems. As a result, a significant number of pertinent studies[3][4] make use of this dataset, which is why we have also opted for it.\n\n[1] Ann Lee, et al. Discriminative Reranking for Neural Machine Translation\n\n[2] Liu, Yixin, et al. SimCLS: A simple framework for contrastive learning of abstractive summarization\n\n[3] Gu, Jiatao, et al. Search engine guided neural machine translation\n\n[4] Cai, Deng, et al. Neural machine translation with monolingual translation memory"}, {"review_id": "dxBtPFj1eD", "reviewer": "Reviewer_Z8iX", "summary": "This paper aims to address the retrieval-augmented text generation problem, where the principle of memory retrieval is to select examples similar to the input. The authors are motivated by an observation in their preliminary experiments that the memory examples that better resemble the data distribution during inference come from the model's output, instead of the training data. Therefore, they propose a novel framework, selfmem to address the retrieval-augmented text generation problem. The proposed framework consists of a memory-augmented generator and a memory selector. Differently, the generator can produce multiple candidates to serve as generated memory instead of retrieved memory from the training corpus. The authors experiment on three generation tasks: machine translation, summarization, and dialogue, and demonstrate performance improvement in various evaluation metrics.", "questions": "Suggestions:\n1. In Figure 2, the shading in the square exceeds the boundaries, which needs to be fixed.\n2. More captions are needed for Figure 2(b) to better illustrate the content.\n\nTypos & Grammar errors:\n1. Please refer to the comments in Weaknesses.\n2. In L215, there is an irrelevant word \"open\" that should be removed.", "limitations": "I didn't find the relevant discussion is this paper.", "rating": 5, "confidence": 4, "soundness": 3, "presentation": 2, "contribution": 2, "strengths": "1. The paper provides sufficient related work, which is helpful for readers to understand the research context.\n2. The research motivation is well-supported by preliminary experimental results, which is a good starting point for the research.\n3. The paper includes thorough experiments covering representative tasks in the generation field, including machine translation, summarization and dialogue generation, which could be a valuable contribution.", "weaknesses": "1. The novelty of the paper is limited. The authors propose a dual structure: Retrieval-augmented Generator and memory selector, each of which is built upon existing paradigms.\n\n2. The overall paper is not well-presented. \n\n(a)  Many parts of the paper lack clear descriptions. For example, the setting of the preliminary experimentation is unclear at all. \n\nIn L119-120, how does the number 38.89 in Table 1 come from? For example, what does the head of row (Memory, Hypothesis) mean? There is no brief description of this experimental setting in the context.  The results in Table 1 are not clear enough to understand the motivation.\n\n(b) Additionally, there are grammar errors and unclear sentence structures in the paper that hinder the reader's comprehension. \n\nIn L113-114, \"The primary motivation behind our framework stems from the observation that the memory, which is more similar in distribution to the data during inference, is not the training data (38.89 BLEU, as shown in Table 1).\" -> \"The primary motivation behind our framework stems from the observation that the memory is more similar in distribution to the data during inference, not the training data (38.89 BLEU, as shown in Table 1).\"\n\n3. The experimental results are not that convincing as the authors did not provide human evaluation results.  Due to the limitations of automatic evaluation metrics, it is usually necessary to conduct human evaluation to assess the quality of generated output in generation tasks. \n\nIn this paper, the generation is boosted with the examples that meet the automatic evaluation metrics. This naturally leads to an increase in the numerical values of automatic evaluation metrics.  It is still necessary to use human evaluation results to further determine the quality of the generated output.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "Firstly, we would like to express our sincere gratitude for the time and effort you have dedicated to reviewing our paper. We will address each comment in a point-by-point manner.\n\n- Comment 1: Limited novelty\n- Response 1: We acknowledge that our study's two primary components, namely the retrieval-augmented generator and the two-stage reward model, are indeed rooted in existing literature. We have conscientiously cited these sources throughout our paper to duly credit their contributions. However, we contend that the novelty of our work should not be downplayed, analogous to how GAN (Ian J. Goodfellow et al.) should not be viewed as merely a fusion of a generator and a discriminator. Our research, for the first time, delves into the fundamental limitations of bounded memory within the current retrieval-augmented literature and proposes a novel self-uplifting framework by integrating the primal and dual problems. This unique aspect of our study has also been acknowledged by other Reviewers. Furthermore, we have executed a rigorous empirical evaluation across a diverse range of text generation tasks, encompassing three distinct categories, two generation paradigms, and seven datasets. The results reveal that our method consistently outperforms existing works employing either one of the two components.\n\n---\n\n- Comment 2: Unclear description of preliminary experiment and grammar errors.\n- Response 2: First, we apologize for any confusion that our presentation may have caused. To clarify, the value 38.89 in Table 1 denotes the average BLEU score of the retrieved memory in relation to the ground-truth reference. In this table, the number under the head(Memory/Hypothesis) refers to the quality of the memory and the hypothesis, as assessed by the BLEU score. The underlying motivation and rationale for this preliminary section can be summarized as follows:\n    1. Given the primal problem that better memory prompts better generation, we first investigate whether the output from the generation model could directly serve as a more effective memory, considering its higher similarity to the ground truth compared to the retrieved memory (please refer to the first and second rows under the Memory head in Table 1).\n    2. The results do not support this assumption (as seen in the first and second rows under the Hypothesis head in Table 1), prompting us to consider two potential reasons for this outcome. We then conduct another experiment to eliminate the first possibility (evident in the third row of Table 1) and opt for the second reason, supported by uncertainty analysis. This finding encourages us to avoid selecting memory based on p(y|x) and reaffirms the necessity of a memory selector in our framework. \n    \n    We hope that this explanation clarifies our motivation and we are more than willing to engage in further discussions to address any potential misunderstandings.\n\n---\n\n- Comment 3: Lack of human evaluation results\n- Response 3: We appreciate your suggestion and have included human evaluation results as well as GPT-4 evaluation results in the attached global rebuttal PDF file for your reference.\n\n---\n\n- Comment 4: Lack of discussion of limitation\n- Response 4: The discussion can be found in the appendix, which is included in the supplementary zip file."}, {"review_id": "IQvbyRTP3X", "reviewer": "Reviewer_dohg", "summary": "The traditional approach for memory retrieval is constrained by the quality of the fixed corpus from which memory is retrieved. Based on the exploration that \"better generation also promotes better memory\", this work proposes the Selfmem framework, which iteratively employs a retrieval-augmented generator to create an unbounded memory pool and uses a memory selector to select a generated memory for the next round. Experimental results demonstrate the effectiveness of Selfmem in fine-tuned small models and few-shot LLMs for three different text generation tasks.", "questions": "1. When the proposed method uses the memory selector to select a candidate, different metrics are used for different text generation tasks, which limits the generality of the method on different tasks. Is there a good general method or metric that can be applied to different text generation tasks? How well does the proposed method perform when using a generic method or metric?\n\n2. How to preserve diversity when generating candidate pool \\mathbb{C}?\n\n3. How to draw the conclusion in Line 143 that 'the generator does not need to resort to external memory'?  The conclusion in Line 145 also confuses me a lot, e.g. 'This observation motivates us to select memory according to metrics'. Could you explain in more detail?\n\n4. For tasks such as dialogue generation, this work only uses automatic evaluation, lacking human evaluation. Automated assessments also lack justification.", "limitations": "The authors discuss limitations in the appendix.", "rating": 5, "confidence": 4, "soundness": 3, "presentation": 2, "contribution": 3, "strengths": "1. The motivation of this work is novel. It iteratively employs a retrieval-augmented generator to create an unbounded memory pool and uses a memory selector to select a generated memory for the next round. This enables the model to leverage its own output to improve generation.\n2. The method proposed in this paper has a good effect and has achieved improvement on multiple datasets on three text generation tasks. In addition, this work also verified the proposed method for LLM.\n3. The experimental analysis of this paper is sufficient. In addition to verifying the effectiveness of the proposed method on multiple datasets, this paper also conducts thorough analyses of each component in the Selfmem to identify bottlenecks and provide insights for future research.\n", "weaknesses": "1. The generality of the method is limited. When the proposed method uses the memory selector to select a candidate from the candidate pool, different metrics are used for different text generation tasks, which limits the generality of the method on different tasks.\n\n2. The cost of the proposed method is large. The Selfmem needs to execute retrieval and generation multiple times until converges. \n\n3. There are some writing problems in this paper:\n- There is an extra word “open” in line 215.\n- Line 141 \\phi(R) -> \\phi(\\mathcal{R}).\n- Equation (4) MemoryEncoder(m).\n- No details about how to generate candidate pool \\mathbb{C}.\n\n4. A generation example can be provided in the experimental results so that readers can feel the effect of the method in this paper more intuitively.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "Firstly, we would like to express our sincere gratitude for the time and effort you have dedicated to reviewing our paper. We will address each comment in a point-by-point manner.\n\n- Comment 1: Limited generality and universal metrics for different text generation tasks.\n\nResponse 1:  \n- We are grateful for the reviewer's insightful comment concerning the generality and applicability of universal metrics across various text generation tasks. While our work indeed employs different metrics for specific generation tasks, we respectfully disagree with the notion that this approach restricts the generality of our framework. Instead, we contend that this demonstrates the versatility and adaptability of our method.\n- The rationale for utilizing distinct metrics for individual tasks stems from the unique characteristics and objectives of each generation task, and what our framework does is to provide a method to optimize these attributes accordingly. For instance, in neural summarization tasks, the ROUGE score is deemed more suitable than the BLEU score for several reasons, including its emphasis on recall over precision of ground truth summaries and the absence of a length penalty for summaries of varying lengths. The ROUGE score has also been shown to achieve stronger correlation with human judgments[1]. This rationale informed our decision to use ROUGE for optimizing our memory selector and, consequently, our generator. If a generation task needs diversity, we optimize for diversity. If it needs fidelity, we optimize for fidelity. The only modification needed is adjusting the memory selection metric in Algorithm 1 of our paper. \n- Furthermore, we believe that the experimental results obtained across three different tasks within this single framework provide empirical evidence that our framework is not a task-specific one.\n- As for the interesting question about the the existence of a universal metric suitable for all text generation tasks, irrespective of their specific requirements, we consider such a metric unlikely. If one were available, there would be no need to divide text generation tasks into multiple subfields. However, one potential solution to this challenge is the use of human preference, as employed in the optimization of Language Models (LLMs) like ChatGPT. Our framework can also be adapted to this context by adjusting the memory selection metric to reflect human preference. \n\nWe express our sincere gratitude for raising this thought-provoking issue and welcome further discussion on the subject.\n\n [1] AlexanderR. Fabbriet al.SummEval: Re-evaluating Summarization Evaluation\n\n---\n\n- Comment 2: Latency caused by selfmem\n- Response 2: We appreciate the reviewer's concern regarding the latency caused by our method, as mentioned in the limitations section in the Appendix. We acknowledge that our proposed method may result in higher latency compared to conventional generation scenarios. However, it is essential to note that this paper represents the first attempt in this direction, and our primary focus is on the novel aspects of the approach rather than efficiency. We believe there is significant potential for improvement in future work, such as incorporating an efficient dual encoder in place of a cross-encoder for memory selection, and reducing the number of iterations by increasing the size of the candidate pool, as suggested by Reviewer LVYL. We are eager to explore these possibilities and engage in further discussions to address the issue of latency while maintaining the integrity and effectiveness of our approach.\n\n---\n\n- Comment 3: Writing problems\n- Responses 3: Thank you for your thorough review and we will complete another round of proofreading in our final version. As for the candidate pool generation method, we have included it in Section 4.2: \"For all tasks, the candidate generation method involves a beam search with a beam width of 50.”\n\n---\n\n- Comment 4: Generation example\n- Response 4: Thank you for bringing this to our attention. Upon the paper's acceptance, we will release all outputs, including Selfmem and baselines. Additionally, we will endeavor to find examples that best illustrate our method.\n\n---\n\n- Comment 5: Preserving diversity in candidate pool\n- Response 5: As demonstrated in previous work [1], employing beam search is adequate for achieving significant improvement, and thus, we have not incorporated diversity into this paper's scope.\n    \n    [1] Ann Lee, et al. Discriminative Reranking for Neural Machine Translation\n\n---\n- Comment 6: Explanation of the conclusion in Line 143 and Line 145\n- Response 6: We apologize for any confusion that our presentation may have caused, and appreciate the opportunity to clarify. The conclusion in Line 143 is drawn from the observation that the generation confidence of the overlapping token set was generally higher than the average. This indicates that the generator is more likely to produce these low-perplexity tokens even without the provided memory. As a result, the conclusion from Line 145 could be more accurately stated as: \"This observation motivates us to reconsider selecting memory based on p(y|x).\" If there is any remaining confusion, we encourage you to refer to our Response 2 to Reviewer Z8iX.\n\n---\n- Comment 7: Lack of human evaluation results\n- Response 7: We appreciate your suggestion and have included human evaluation results as well as GPT-4 evaluation results in the attached rebuttal global rebuttal PDF file for your reference.\n"}, {"review_id": "vjklnL0pyy", "reviewer": "Reviewer_Mggy", "summary": "The paper presents a framework, Selfmem, aimed at enhancing text generation tasks via memory retrieval. The core uniqueness of the framework resides in its capacity to create an unbounded memory pool by utilizing its retrieval-augmented generator and memory selector components iteratively. This allows the model to leverage its output, referred to as self-memory, for improved generation. The authors apply this framework into neural machine translation, abstractive text summarization, and dialogue generation tasks, demonstrating its efficacy with state-of-the-art results", "questions": "(1) Have you considered using other evaluation methods (e.g., human evaluation) to further validate the performance of the framework?\n(2) How can the Selfmem framework be adapted for optimal usage with learnable retrievers?\n", "limitations": "A potential concern is, compared to human-written texts, machine-generated texts are noisier and with potential bias. Hence, is the proposed method able to avoid being affected by its own bias?", "rating": 5, "confidence": 4, "soundness": 3, "presentation": 3, "contribution": 2, "strengths": "(1) The paper proposes the Selfmem framework, which addresses the constraint of fixed corpus in memory retrieval.\n(2) The paper demonstrates the effectiveness of Selfmem on various text generation tasks.\n(3) The paper provides detailed analyses of components within the Selfmem framework, revealing potential bottlenecks and providing  insights.\n", "weaknesses": "(1) It would be better if more experiments beyond neural machine translation, abstractive text summarization, and dialogue generation tasks could be conducted. It will be beneficial to see its application in other related areas since the proposed method is task-agnostic.\n(2) Much of the paper's experimentation relies on certain metrics (ROUGE-1) for evaluation. The universality of the findings might be cross-checked through the usage of other performance metrics.\n(3) Though the authors mention learnable retrievers, they don't discuss how the Selfmem framework may be adapted for use with such retrievers.\n", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "Firstly, we would like to express our sincere gratitude for the time and effort you have dedicated to reviewing our paper. We will address each comment in a point-by-point manner.\n\n- Comment 1: More generation tasks needed.\n- Response 1:  Thank you for recognizing the versatility of our method. In our current study, we have chosen three generation tasks across seven datasets to assess our framework. This decision is based on two main factors: (1) we believe that these three tasks, specifically machine translation, summarization, and dialogue generation, effectively represent text generation tasks, and (2) these tasks have been extensively employed in retrieval-augmented literature [1][2][3], providing a relatively comparable testbed for benchmarking our method. Additionally, we plan to make our code open-source, allowing researchers to verify its effectiveness in other tasks.\n\n[1] Jason Weston et al. Retrieve and Refine: Improved Sequence Generation Models for Dialogue\n\n[2] Jiatao Gu et al. Search Engine Guided Neural Machine Translation\n\n[3] Nabil Hossain et al. Simple and Effective Retrieve-Edit-Rerank Text Generation\n\n---\n\n- Comment 2: Metrics other than ROUGE for evaluation\n- Response 2: First, we would like to clarify that we did not heavily rely on ROUGE-1 for evaluation. For various generation tasks with distinct desired attributes, we employed different well-acknowledged metrics for assessment. Moreover, to cross-check our findings, we have included additional evaluation results of TER and chrF++ for translation tasks in the appendix. Furthermore, human evaluation and GPT-4 evaluation results are available in the global response PDF file for your reference.\n\n---\n\n- Comment 3: Combination with learnable retriever\n- Response 3:  Although we briefly mentioned learnable retrievers in our paper, we do not believe it is of significant relevance to Selfmem. As illustrated in Algorithm 1, the retriever's role within our framework is mainly confined to the initial step. While a learnable retriever might outperform a non-learnable one, incorporating it would not fundamentally change our framework. This is because the core concept of Selfmem lies in generating memory rather than retrieving it, based on the observation that the quality of retrieved memory is always constrained by the memory pool. Furthermore, the empirical results in Table 2 provide a direct comparison between our framework and a generation model with a learnable retriever (MonoNMT), demonstrating the superiority of self-memory over retrieved-memory.\n\n---\n\n- Comment 4: Debias from machine-generated text\n- Response 4: Thank you for bringing up this intriguing issue. We believe that our framework has the potential to reduce bias in machine-generated text. Considering that the entire framework is optimized by the memory selection metrics $\\Delta(\\cdot,\\cdot)$, it is feasible to introduce a debiasing term or even incorporate human feedback into the $\\Delta(\\cdot,\\cdot)$ to improve the quality of text generation. We will consider delving deeper into this aspect in future research."}, {"review_id": "etTJcdHVLi", "reviewer": "Reviewer_Ysdi", "summary": "The paper proposes a novel framework called Selfmem to address the limitations of retrieval-augmented text generation. Compared with memory retrieval from a fixed corpus, Selfmem iteratively uses a retrieval-augmented generator to create an unbounded memory pool and select the candidate output as memory for the subsequent generation round. The paper evaluates the effectiveness of Selfmem on three text generation tasks and achieves state-of-the-art results. The paper also conducts thorough analyses of each component in the framework.", "questions": "1.\tAlthough you mention the computational resources in the limitations section, can you offer an empirical comparison between Selfmem with other retrieval-based methods?\n\n2.\tFor a more vivid demonstration, can you provide any case studies or examples to illustrate how the generated contexts from Selfmem outperform the retrieval-based ones.\n", "limitations": "See the weakness section.", "rating": 6, "confidence": 3, "soundness": 3, "presentation": 3, "contribution": 3, "strengths": "The motivation of the primal problem and its duality is fairly intriguing. Preliminary experiments and supporting evidence demonstrate the rationality of the proposed method.\n\nThe proposed Selfmem achieves promising results on machine translation, summarization and dialogue generation, indicating its practical applicability.\n\nThe paper is overall clearly written. The problem setting and proposed approach is lucidly presented.\n", "weaknesses": "There is a growing body of recent work [1] that is increasingly using LLMs to generate knowledge instead of retrieving it from external corpus. This paper shares similar ideas with the new paradigm and should incorporate and discuss it in the related work section.\n\nFurther clarification and empirical evaluation on the diversity of retrieval memory from Selfmem should be provided. Given the limited knowledge encoded in the trainable generator, it remains uncertain whether Selfmem can effectively handle knowledge-intensive tasks compared to retrieval from external sources.\n\nSelfmem appears to require more computational cost than direct retrieval. However, there is no further discussion on time computation in the paper.\n\n[1] Yu et al. Generate rather than Retrieve: Large Language Models are Strong Context Generators. ICLR 2023\n", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "Firstly, we would like to express our sincere gratitude for the time and effort you have dedicated to reviewing our paper. We will address each comment in a point-by-point manner.\n\n- Comment 1: discussion about using LLM to generate knowledge instead of retrieval\n- Response 1: Thank you for mentioning the relevant reference by Yu et al. and saving us from LLM paper flood. We would include this paper, as well as any other pertinent studies, in the final version of our work.\n\n---\n\n- Comment 2: Evaluation of knowledge-intensive tasks\n- Response 2: We appreciate your suggestion to incorporate knowledge into our framework for evaluation on knowledge-intensive tasks. However, it is important to note that the primary focus of this paper is not on this aspect. In the retrieval-augmented literature, there are generally two types of working systems: one that directly retrieves kNN samples from training set, wherein task knowledge is implicitly expressed through format, style, and word selection for tasks such as machine translation[1] and language modeling[2]; and the other that focuses on external knowledge retrieval for knowledge-intensive tasks, such as open domain QA[3] and fact verification[4]. Our work is primarily concerned with the former type and we leave the latter for future exploration.\n\n[1] Urvashi Khandelwal et al. Nearest Neighbor Machine Translation\n\n[2] Urvashi Khandelwal et al. Generalization through Memorization: Nearest Neighbor Language Models\n\n[3] Gautier Izacard et al. Leveraging Passage Retrieval with Generative Models for Open Domain Question Answering\n\n[4] Patrick Lewis et al. Retrieval-Augmented Generation for Knowledge-Intensive NLP Tasks\n\n---\n\n- Comment 3: Empirical results of computation overhead\n- Response 3: Thank you for highlighting this important aspect. We have provided a detailed analysis of the computation overhead of Selfmem in comparison to baseline systems in the global rebuttal pdf file. We hope this addresses your concerns.\n\n---\n\n- Comment 4: Request for a more vivid demonstration\n- Response 4: Upon the paper's acceptance, we will release all outputs, including those from Selfmem and baselines, and we will make an effort to identify examples that best showcase our method."}]

Author rebuttal:
Dear Reviewers,

We would like to express our deepest appreciation for the time and effort you have devoted to reviewing our conference paper. In response to the questions raised by each reviewer, we have provided detailed answers in the corresponding sections, and we hope that these clarifications will address some of your concerns.

In light of the shared concerns regarding latency evaluation and human evaluation, we have prepared an additional PDF file that is attached to this response for your reference. We are eager to engage in further discussions about our paper and to continue refining and enhancing our work with the invaluable assistance of esteemed reviewers.

Once again, thank you for your valuable insights, and we look forward to your continued guidance and support.

Sincerely,

Authors

Meta-review: The paper empirically demonstrates that an intuitive iterative candidate generation scheme can lead to a well-performing model for retrieval-augmented text generation. The idea is sensible and a nice extension of recent generate-rather-than-retrieve line of works (Yu et al., 2023; inter alia), and the performance is strong. However, the paper makes a "memory"-based narrative which makes the technical content obfuscated, and at the same time leaves a few experimental settings underspecified (but the code is supplied). Despite the subpar presentation, the technical contributions seem strong enough to justify an acceptance.
Decision: Accept (poster)