Memorisation Cartography: Mapping out the Memorisation-Generalisation Continuum in Neural Machine Translation
Authors: Verna Dankers
Ivan Titov
Dieuwke Hupkes
Venue: EMNLP 2023 Main (2023)
Keywords: interpretability
neural machine translation
memorization
TL;DR: Which features underlie memorisation in NMT and how does memorisation relate to model performance? To examine this, we create and analyse memorisation maps for NMT that position 5M training set examples on a memorisation-generalisation continuum.

Abstract: When training a neural network, it will quickly memorise some source-target mappings from your dataset but never learn some others.
Yet, memorisation is not easily expressed as a binary feature that is good or bad: individual datapoints lie on a memorisation-generalisation continuum.
What determines a datapoint's position on that spectrum, and how does that spectrum influence neural models' performance?
We address these two questions for neural machine translation (NMT) models. We use the counterfactual memorisation metric to
(1) build a resource that places 5M NMT datapoints on a memorisation-generalisation map,
(2) illustrate how the datapoints' surface-level characteristics and a models' per-datum training signals are predictive of memorisation in NMT,
(3) and describe the influence that subsets of that map have on NMT systems' performance.

Memorisation Cartography: Mapping out the Memorisation- Generalisation Continuum in Neural Machine Translation
Verna Dankers +f and Ivan Titov and Dieuwke Hupkes
ILCC, University of Edinburgh
ILLC, University of Amsterdam <FAIR, Meta AI
vernadanker com, i ti tov@inf .ed.ac. uk,
dieuwkehupkesseneta.c com
Abstract
and here
general Leatton score
Semarrky waters: paraphrased
trans) attons and alightly insczurate
When training a neural network, it will quickly memorise some source-target mappings from your dataset but never learn some others. Yet, memorisation is not easily expressed as bi- nary feature that good or bad: individual dat- apoints lie continuum. What determines a datapoint's 's po- sition on that spectrum, and how does that spec- trum influence neural models' performance? We address these two questions for neural ma- chine translation (NMT) models. We the counterfactual memorisation metric to (I) build a resource that places 5M NMT datapoints on memorisation intooissttiongenealllsttttoo map, (2) illustrate how the datapoints' surface-level characteris- ties and models' per-datum training sigmals are predictive of memorisation in NMT, (3) and describe the influence that subsets that map have NMT systems' performance.!
56 nore counterfactual govorisation mears transiation wore hall lurinationn
6E these examples beost #odel performance
54 which factors contribute to higt OF frequenctes, sequence longths IIEE sngaeitstion rate
craining resorisation
$4 stscllgred examples 54 the more source and target evertap are likely to end up here the wore examples nove up this diagonal.
Figure 1: Illustrative summary of findings for different areas of the memorisation map. Counterfactual memori- sation subtracts the y-coordinate from the z-coordinate.
(Carlini et al., 2022); ChatGPT can recall detailed information from its training data, such as named entities from books (Chang et al., 2023); GPT-2 memorises personally identifiable information, in- cluding phone numbers (Carlini et al., 2021); GPT- 2 and Transformer NMT systems fail to memorise certain idioms (Dankers et al., 2022; Haviv et al.. 2023); and in the presence of some source prefixes, NMT systems have memorised always to emit the same translation (Raunak and Menezes, 2022).
1 Introduction
When training neural networks, we aim for them to learn a generic input-output mapping, that does not overfit too much on the examples in the train- ing set and provides correct outputs for new inputs. In other words, expect models to generalise without memorising too much. Yet, adequately fitting a training dataset that contains natural lan- guage data inevitably means that models will have to memorise the idiosyncracies of that data (Feld- man, 2020; Feldman and Zhang, 2020; Kharitonov et al., 2021). The resulting memorisation patterns are both concerning and elusive: both memorisa- tion and task performance increase with model size
These examples illustrate the multi-faceted re- lation between memorisation and generalisation. In machine learning, memorisation has tradition- ally been associated with overfitting and over- parameterisation: Dietterich (1995, p.326) already discusses the concern of "fit[ting] the noise in the data by memorizing various peculiarities". Yet, for deep learning, overfitting has even been referred to as benign when models overfit the training data but still have low generalisation error (Zhang et al., 2017: 1: Bartlett et al., 2020). Besides, memorisation is not just considered detrimental; for instance, it is needed the natural data distribution is long-tailed
"ook partially conducted during an internship at FAIR. Cllikk here to interactively explore the NMT memorisation maps in our demo.

---

(Feldman, 2020) or if factual information needs to be stored (Haviv et al.. 2023). At the same time, benign overfitting is still concerning if it makes models less robust (Sanyal et al., 2021) and intro- duces privacy concerns (Carlini et al., 2019).
2 Related work
How has memorisation been measured in NLP, and what have we learnt as a result? In this section, we first dive into memorisation metrics from NLP generically and then report the limited set of related work that exists for NMT.
In this work, we get step closer to understand- ing the relation between memorisation and data- points' characteristics for neural machine transla- tion (NMT). Instead of focusing on cases that are memorised verbatim, we take 5M examples from five language pairs, put them on memorisation- generalisation map, learn to predict examples' places on the map and analyse the relation be- tween the map and model performance. The map is centred around the counterfactual memorisation (CM) metric (Zheng and Jiang, 2022) (see $3), and available to readers via our repository. Using the map, we address the following research questions, for which we illustrate takeaways and interesting findings in Figure 1:
Memorisation metrics in NLP In NLP, memo- risation has most often been quantified via binary metrics that identify examples that are memorised verbatin. Carlini et al. (2021) measure k-eidetic memorisation (i.e. a string appears at most k times and can be extracted from the model). Other stud- ies omit the constraints on I and simply exam- ine whether, after feeding the context, parts of a sentence can be extracted verbatim (Carlini et al., 2019; Kharitonov et al., 2021: Carlini et al., 2022: Mireshghallah et al., 2022; Tirumala al., 2022;
Chang et al., 2023). Sometimes, the training data is known or even modified to include 'canaries' (e.g.
Mireshghallah et al., 2022) while in other cases, the training data is unknown (e.g. for ChatGPT, Chang et al., 2023). These studies have raised privacy concerns by pointing out that personally identi- fiable information and copyright prrregtt-prttctted text is memorised and have identified potential causes of memorisation, such repetition, large vocabulary sizes, large model sizes and particular fine-tuning techniques.
1. How do churacteristics of datapoints re- late to their position on the memorisation- generalisation map? In $4, we compute 28 quantitative features and annotate a data sub- set manually using 7 additional features. We discuss how souree-target similarity, input and output length, token frequency and tokens' segmentation relate the memorisation map. 2. Can we approximate memorisation metrics using datapoints' characteristics? ? In $5, we use datapoints' characteristics to predict mem- orisation values using multi-layer perceptrons (MLPs) to consolidate findings from $4, to compare different languages to one another and to understand whether resource-intensive memorisation computation has cheaper ap- proximates. We find that the MLPs gen- eralise cross-lingually: characteristics' re- lation to memorisation largely language- independent.
A second approach has been to rate memori- sation on a scale (Zhang et al., 2021; Zheng and Jiang, 2022). Zhang et al. measure counterfactual memorisation (CM), metric from computer vi- sion (Feldman and Zhang, 2020) that assigns high values examples a model can only predict cor- rectly if they are in the training set. Zhang et al. identify sources of CM smaller language models (trained with 2M examples), such as the presence of non-English tokens in English sentences. Inspired by the CM metric, Zheng and Jiang (2022) use self-influence to quantify the change in parameters when down-weighting sure memorisation in sentiment analysis, natural language inference and question answering.
3. How does training on examples from differ- ent regions of the memorisation map change relate
training example, to mea-
models' performance? Finally, different parts of the map to the quality of NMT systems in terms of BLEU, targets' log- probability and hallucination tendency (sce $6). Our results confirm previous work from other tasks - examples with high CM are most relevant for models' performance - yet there are caveats worth mentioning, in particular for the hallucination tendency.
Memorisation metrics in NMT For NMT, mem- orisation is less well explored, but we still observe similar divide of metrics. Raunak and Menezes (2022) propose extractive memorisation, a binary metric that identifies source sentences with a prefix for which models generate the same translation, independent of the prefix's 's ending. Raunak et al.

---

(2021) compute CM scores in a low-resource NMT setup show that hallucinations are more promi- nent among examples with higher CM values.
1.00
memorisation
generalisation score
0.9-1.0
We, too, treat memorisation as graded phe- nomenon by using CM-based metrics. Whereas Raunak et al. (2021) solely explore CM in the con- text of hallucinations, we build multilingual re- source of memorisation metrics, examine the char- acteristics of datapoints that influence their position on the memorisation map, and investigate the rela- tion to models' performance.
0.75
0.6-0.9
0.7-0.88
0.50
0.5-0.6
D.4-0.5
0.25
0.2
0.1-0.2
0.00
0.0-0.1
0.00
0.25 0.50
0.75 1.00
training memorisation
Figure 2: The memorisation map for EN-Es. Colours indicate counterfactual memorisation from 0 (dark blue, along the diagonal) to (dark red, bottom right).
3 Experimental setup
This section details the memorisation metrics em- ployed and the experimental setup for the model training that is required to compute those metrics.
hypotheses and reproduce subset of the findings with those alternative maps.
Memorisation metries To obtain graded notion memorisation, we employ the counterfactual memorisation (CM) metric Feldman and Zhang (2020) and Zhang et al. (2021).² Assume an exam- ple with input æ and target y, and model with the
CM thus consists of two components which we refer to as training memorisation (TM, that ex- presses how well model performs on training examples) and the generalisation score (GS, that expresses how well model performs on unseen examples). CM the difference between the two is thus high for examples that can be predicted correctly if they are in the training set but that a model cannot generalise to if they are not. Instead of approaching CM as a one-dimensional metric, we examine patterns that underlie all three metrics.
parameters t trained on all training data, and ffsst trained on all examples except (I. y). CM can be computed follows:
CM(x, y)
po (1/
Pinaa (y/xx
training memorisation
gencralisation score
As leaving individual datapoints is too expen- sive, computationally, we leave out data subsets, similar to Zhang et al. (2021). To then compute the CM of an individual datapoint, one can collect models and average the target probability pew. (ylz) over all models 8t e in a collection, where O" and est represent sets of models that did and did not train on (zz, y). respectively. Since con- sider the generation of sequences, we also aggre- gate probabilities over tokens in the target sequence of length using the geometric mean. We combine averaging over models and averaging over tokens in a likelihood metric (LL), that is computed for O and tst to obtain the CM value of an example:
Data Even when leaving out data subsets, com- puting the memorisation metrics is still resource- intensive. To balance the efficiency of the compu- tation with the quality of the NMT systems, we use corpora with 1M examples for five language pairs: English is the source language, and the target lan- guages are German, Dutch, French, Spanish, and Italian. To enable direct comparison between lan- guages, we collect parallel data by taking sentence pairs from the intersection of the OPUS corpora for these languages (Tiedemann and Thottingal, 2020). Using multiple languages aids in assuring that the conclusions are not language-specifee Ap- pendix A. details how the corpora were filtered and the 1M examples were selected. The resulting data is relatively 'clean'. What happens when us- ing random OPUS subset with much more noisy data? Appendix D. elaborates on this.
1 Σ
LL (2, 0M)
II (El
pog (Mllve)
m
Appendix D.3, we replace the probability- based measure with BLEU scores for greedily decoded
Training models to obtain memorisation metries We train 40 models to compute our metrics repeat- edly on randomly sampled 50% of the training data, while testing on the remaining 50%. The mod-
We refer to as counterfactual memorisation. following work of Zhang et al. (2021). ). The metric is also known as labe! memorisation in other articles.

---

els transformer-baae models (Vaswani et al., 2017), trained with fairseq for 100 epochs (Ott et al., 2019). ensure that this leads to reliable CM scores, we compare the scores computed over the first 20 seeds to those computed using the sec- ond 20 seeds: these scores correlate with Pearson's 's 0=0.94. When combining 40 seeds, the metrics are thus even more reliable. We evaluate models using the FLORES-200 'dev' set (Costa-jussà et al., 2022), dataset created by expert translators for Wikimedia data. Appendix A.2 provides details on model training and the development set's BLEU scores.
0.
0.5
1.0
0.5
0,5
trairing memorisation
generaliiatio score
counterfactual mem.
(a) Minimum target word log-frequency
0.0
05
1.
0.0
0.5
1.0
0.0
0.5
1.0
training memorisation
generalication scom
counterfactual mam.
(b) Source length
dersity
0.0
0.
1.0
0.5
1.0
0.0
0.5
4
Data characterisation: what lies where on the memorisation map?
trairing memorization
generalisation score
counteriactual mem.
(c) Target segmentation ratio:
1d]
bucktersa
We now have values for our memorisation met- rics for 5M source-target pairs across five language pairs. We can view each source-targel pair as a co- ordinate on map based on the train and test perfor- mance associated with that example; the offset of the diagonal indicates the CM. Figure 2 illustrates the coordinate system for EN-Es. It represents dat- apoints using scattered dots, coloured according to CM. As is to be expected, the TM values exceed the GS values, meaning that generating an input's translation is easiest when that example is in the training set. Examples with high CM are rare: few examples are very easily memorised during training while also having very low generalisation score. Our interactive demo can be used examine indi- vidual instances on the map.
0.5
0.5
training memorisation generalisation score counterfactual mem.
(d) Backtranslation edit distance between and target
Figure 3: Illustration of how four different features relate the three memorisation metrics. Appendix C displays these graphs for additional features.
lap and the aligmment monotonicity (as per the Fuzzy Reordering Score, Talbot et al., 2011). For each feature, we compute Spearman's 's rank correla- tion (p) for TM, GS and CM, combining datapoints from all five language pairs. All correlations contained in Figure 14, and report the most prominent patterns in the remainder of this subsec- tion. Appendix provides implementation details per feature.
To better understand which characteristics inffu- ence a datapoint's 's position on this map, we next analyse the correlation between datapoints' fea- tures (automatically computed and manually anno- tated ones) and different regions of this landscape.
Frequency The frequency features are strong pre- dictors for CM (e.g. for the minimum target log- frequency feature, PCM=-0.48 0.46, depicted in Fig- ure 3a). Examples with low-frequency tokens can be learnt during training, but models are much less likely assign a high probability to targets with low-frequency tokens during testing.
4.1 Analysis of feature groups
We compute 28 language-indeee inggeeeennndeeendeett features that together we believe to cover a broad spectrum of surface-level features from both the source and target. 19 features describe the source and target separately based on the length, word frequency, the number of target repetitions, BPE segmenta- tion, and digit and punctuation ratios. The nine remaining features capture the source-target lap with the edit distance, edit distance of the get's back translation (computed with models from Tiedemann and Thottingal, 2020), length differ- ences, ratios of unaligned words, word/token over-
Length The length characteristics correlate more strongly with CM than with TM GS (e.g. for the source length, PcM=0.26. also visualised in Fig- ure 3b). This means that longer sequences tend have larger difference in performance between training and testing time, compared to shorter se- quences.

---

Token segmentation Thirdly, the segmentation of tokens into subtokens positively correlates with CM (ACM=0. -00==0000//M000.37 for source/target seg-
1.0
paraphrase formatting nearly wfw removal addition inaccurate
generallisaton score
mentation, respectively), as shown in Figure 3c.
The segmentation compares the number of white-
0.6
[e]
space-based tokens to BPE tokens: 1
0.4
word for word
Repetitions A feature that is a positive predic- tor for TM and GS is the repetition of the target (prm=0.15, P02== PCM=-0. 0.15). This ex- pected, considering that similar targets have similar sources and are thus more easily memorised. Pre- vious work already noted that repetition-related characteristics (repeated sentence 'template') lead to high TM (Zhang et al., 2021).
0.0 training memorisation
0,50 0.75 1.00
Figure 4: Centroids and marginal distributions of exam- ples grouped through the manual annotation for EN-NL.
4.2 Manual annotation
The previous subsection discussed coarse patterns that relate datapoints' features to memorisation metrics. To understand whether similar patterns up- pear when we qualitatively examine source-target pairs, we annotate 250 Ex-NI. examples, uniformly sampled from different parts of the coordinate sys- tem, with lengths / for which 10 < / < 15. We annotate them using the following labels: (nearly) word-for-we word translation; paraphrase: target omits content from the source; target adds content; inac- curate translation; target uses different formatting in terms of punctuation or capitalisation. Multiple labels can apply to one example. For an elabora- tion on how the labels were assigned and processed, see Appendix C. Figure 4 summarises the results. Firstly, these results consolidate the observation garding source-target overlap: word-for-word trans- lations, e.g. Example (1), are positioned closer the top right corner compared inaccuracies, e.g. Example (2), and paraphrases, e.g. Example (3).
Source-target overlap The remaining features that correlate rather strongly with TM and GS are: the target's backtranslation edit dis- tance to the source (РTM=-0-m) 0.49, PGs= 0.56, Figure 3d), the souree-target edit distance (P3M=-0.30, -0.26),,,, and the fraction of un- aligned tokens (PTM=-0.3) pas --.=2 for get tokens, PTM= 0.28, pas= 0.29 for source tokens).² Apart from negative correlations, there are weak positive predictors, e.g. token overlap (01M=0.15, PG)=0.13) and digit ratio (PrM=0.13, 00000. These features express (a lack of) source-target overlap: source words are absent in the target, or vice versa. Because they predic- tive of both TM and GS, they are not that strongly correlated with CM: they predict where along the diagonal an example lies but not its offset to the diagonal. While you might expect that examples with little source-target overlap require memorisa- tion, their TM values remain low throughout the 100 epochs. The only relation to CM we observe is that, typically, examples in the mid-range (i.e. with some overlap) have higher CM than examples with extreme values (i.e. full or no overlap). CM rea-
(1) 8 EN: Leave a few empty rows and columns on either side of the values.
6 NL: Laat enkele rijen en kolommen leeg aan beide zijden van de waarden. (TM=0.85, GS=0.55)
thus highlights what models can memorise sonable amount of training time. This provides a lesson for NMT practitioners: models are unlikely to memorise the noisiest examples, which might be one of the reasons why semi automatccally scraped corpora, rife with noisy data, have driven the suc- cess behind SOTA NMT systems (c.g. Schwenk al., 2021a.b).
(2) The last years of my life has been one big lie.
L NL: "De afgelopen twee jaren van mijn leven zijn een grote leven geweest. (leven /= lie, TM=0.28, 1S=
EN: donn't know how she did it, but she did it.
'Backtranslations are computed using Mar ian-MT models trained on OPUS (Tiedemann and Thottingal, 2020) to ensure the accuracy of the feature. Alignments are computed with eflomal (Östling and Tiedemann, 2016).
t NL: Geen idee hoe, maar 7e deed (under-
lined portions are paraphrases, TM=0.23, GS=0.14)

---

Yet, paraphrases and inaccurate translations have similar centroids on the map: the differences be- tween those two types are subtle and are not well reflected the memorisation metrics. Lastly, what is not that easily captured by one automated feature, but does show up in these results, is that targets that remove content from the source, e.g. Example (4), are easier to memorise during training than those that add content, e.g. Example (5).
+-0441
0:42
0r-0.47 0:48
Cr55559
0.63
0.41
0.45
0.45
0.51
0.52
[t-
0.
(a) TM
(b) GS
(c) CM
Figure 5: Comparison of the memorisation metrics across the five languages, using Pearson's
(4) EN: Then we had our little adventure up in Alaska and things started to change. NL: Toen waren we Alaska en begonnen
NL: Het is een lang verhaal.
L2 IT: Sarebbe troppo lungo spiegarsi. (It takes too long to explain)
dingen te veranderen. (TM=0.80, GS=0.22)
(5) EN: There are periods and stages in the col- lective life of humanity.
What about examples that move from the top right to the bottom right, i.c. go from easily Iearnable to only learnable if they in the train set? Generally, they seem deviate from source sequences in more subtle ways e.g. they are missing term or phrase, as the case in Examples (9), (10) and (11).
Evenzo zijn er perioden en fasen in het collectieve leven van de mensheid. (TM=0.45, GS=0.34)
NL:
4.3 Comparing metries for five languages
(9) EN: Kenneth, what are you doing here?
The trends of what complicates or eases memori- sation are consistent for all five languages, upon
ES: Kenneth, &qué haces aquf?
t2 FR: Que fais-tu ici (What are you doing
which elaborate in $5. Does this mean that one source sentence will have a very similar memorisa- tion score across the five different languages in our
here?)
(10)s EN: Is this hunting game?
parallel corpus? Not necessarily, as shown Fig- ure 5, which, for the three memorisation metrics, reports the correlation between scores associated with the same source sequence (but different target sequences) across the different languages.
11 FR: C'est un jeu de chasse?
t2 NL: Is dit een spelletje? (Is this game?)
(11)s EN: We need a viable suspect.
ES: Necesitamos sospechoso viable.
DE: Wir brauchen einen Verdächtigen. (We need suspect)
Source sequences with different positions on the memorisation maps from two language pairs give insight into how the relation between the source and target affects memorisation. Examples that move from the top right in one language to the bottom left in another show how targets go from easily learnable to unlcarnable: in Examples (6) and (7) target (tz) seems misaligned. In Example (8) f2 is contextually relevant but not translation.
Approximating memorisation measures
Having examined correlations between datapoints' features and memorisation values, step further and treat this as regression problem: given the characteristics of datapoint, can we predict memorisation values? We include the pre- viously mentioned features and additional ones ob- tained from an NMT system during training. We examine the performance of our feedforward pre- dictors and explore how well the predictors gener- alisc across languages. The analysis aids in consol- idating findings from $4 and improves under- standing of how sanguagg-innependert our findings are. Since computing CM is resource-intensive, the predictors can also serve as memorisation approxi- mators (we circle back to this in $6.2).
go
(6) EN: She's not a child anymore.
t1 ES: Ya no es una niña.
t2 DE: Du hast das Kind verwöhnt, Matthew,
(You spoiled the child, Matthew)
(7) EN: is an international obligation.
t ES: Es una obligación internacional.
22 FR: bargo. (Nianias on the advisability of this embargo)
Nianias sur l'opportunité de cet em-
( S EN: It's a long story.

---

absluute difference
0.15
els trained with other target languages. Note that this does not mean that models for the different languages behave similarly for the same source sentences, but instead that models trained on dif- ferent language pairs behave similarly for source- target pairs with the same features. In practice, this means that we can make an educated guess about the amount of memorisation required for new datapoint the same source sequence in another language, using predictors trained on a subset of the data or using a different but related language." 4
trairing mem.
Pearson 's
gen. score
counterfactual mem.
0.10
0.00
de
de
m1
(a) Features only
1.0
absolute difference
Pearson's
0.8
0.10
0.05
0.6
Memorisation and performance
(b) Features training signals
Finally, we examine the relation that different re- gions of the map have to models" performance. Firstly (in $6.1). by leaving out data subsets while training using our training examples from $3, and, secondly (in $6.2), by sampling specialised train- ing corpora from a larger dataset of 30M examples. The previous sections showed that results across language pairs are highly comparable, which is why in this section, we employ EN-N1. data only.
Figure 6: Predicting memorisation using an MLP, based on examples' features and models' training signals. The MLP is trained on EN-DE and applied to all languages.
Experimental setup To extract training signals, we train one Transformer model per language pair on the full dataset, acting as our diagnostic run from which we extract the following signals: I) the confidence and 2) variability of the target likeli- hood (Swayamdipta et al., 2020, mean and standard deviation computed over epochs). 3) the target like- lihood in the final epoch, 4) a forgetting metric (a counter that accumulates drops in target likelihood during training, Toneva et al., 2019), 5) the hypothe- ses' likelihood in the final epoch, and 6) metric I subtracted from 3, since initial experiments sug- gested those two correlated most strongly with GS and TM. Next, we train a shallow MLP to predict the memorisation metries. We train one MLP on the datapoints' features from $4, and one on the features and the training signals, and report their performance using Pearson's 's correlation and the ab- solute difference between predictions and memori- sation scores. Appendix B details the experimental setup of training the MIPs.
6.1 Importance of different regions
How do examples from specific regions of the mem- orisation map influence NMT models trained on that data? We now investigate that by training mod- els while systematically removing groups of exam- ples.
Experimental setup We train models on datasets from which examples have been removed based on the coordinates from the memorisation map. For 55 coordinates (i.j), where i, E {.1,.2.... 1}. j S create training subsets by removing the nearest examples (up to 750k source tokens total). For each training subset, we then train models with three seeds. Depending on the number of examples surrounding a coordinate, the datapoints can lic closer or further away before reaching the limit.
Results Here, we show the results of MLPs trained on EN-DE and applied to all other language pairs only. The predictions of the MLP trained only on datapoints' characteristics already posi- tively correlate with the memorisation metrics, with Pearson's 's around 0.7 and mean absolute differ- ence around 0.1, Figure 6a. Combining the features and training signals further boosts perfor- mance (see Figure 6b).
We evaluate models according to three perfor- mance metrics: (i) BLEU scores for the FLORES- 200 dev set (Goyal et al., 2022); (ii) the mean log- probability of target, averaged over datapoints from the FLORES-200 dev set; and (iii) halluci- nation tendency computed using the approach of Lee et al. (2018), which involves the insertion of a token into a source sentence and repeating that for more than 300 tokens (high-frequency, mid- frequency and low-frequency subtokens and punc-
Since we applied the EN-DE MLPs to the other languages, these figures illustrate that an MLP trained on one language is transferrable to mod-
WWe comment on the set of languages used in the limita- tions section, see $7.

---

1.0
loy by
bypecsaiity
general Gatton score
0.8
0.5
genora bation
0.6-
0.5
random
2L.6 tendency
10g probabitty
0.4
0.2
0.
1.0
0.5
0.0
0.0
Figure 8: Areas from which we select examples when specialising certain metric.
0.00
0.25
0.30
0.75
1.00
0.00
0.25
0.30
0.75
1.00
training memorisation
training memorisatio
(a) Most relevant
(b) Least relevant
with high CM values (evidenced by more unique n n-grams), making them more informative training data. Secondly, removing examples with a high CM score negatively affects models' predictions for cow-probability tokens, in particular; therefore, including them as training material may quite lit- erally reserve probability mass for the long tail of the output distribution.
Figure 7: The 10 worst- and 10 best-performing regions on the memorisation map per performance metric.
tuation marks), and four token positions. A hal- lucination is recorded if BLEU (Lee al., 2018) drops below after insertion. We apply this to 1000 examples (500 from FLORES, 500 from our parallel OPUS) and, following Lee al. (2018), measure the ratio of source sequences which can be perturbed to a hallucination.
6.2 Specialising NMT systems using memorisation metries
Results To express example's impact, we av- erage the performance of all models for which that example not in the training set. The more negatively the performance is affected, the more important an example is. We aggregate over re- gions of examples and exclude regions that repre- sent <2k datapoints. We then compute the ten most relevant regions (Figure 7a) and the ten least rel- evant ones (Figure 7b). Most relevant means that the BLEU score or log reg-robabblity decreases the most if you remove this group that the halluci- nation tendency increases the most. Least relevant means the opposite. In general, the figures suggest that examples with a higher CM value are more beneficial, and examples closest to the diagonal are the least relevant. This in accordance with related work from computer vision (Feldman and Zhang, 2020) and NLP classification tasks (Zheng and Jiang, 2022), where examples with high CM values had larger (positive) contribution to the accuracy than examples with lower CM values.
In $6.1, we related the maps to models' perfor- mance, but all within our original IM EN-NL dat- apoints. To understand whether our findings ex- trapolate to a larger dataset, we perform proof-of- concept study to show that we can put the lessons learnt to use with new data: memorisation metrics can be predicted using datapoints' features and dis- tinct regions of the map have different roles. We now use these lessons for targeted model training.
Experimental setup We again train NMT sys- tems in low-resource setup, yet, different from the previous sections, we now select examples from a larger set of OPUS examples for EN-NL (30M ex- amples) based on their memorisation score as pre- dicted using the features-only MLP from $4. We first sample 1M random examples, and then sample one dataset based on the most beneficial region for the log- probbabbbilty metric, and one dataset based on the region that is most beneficial for BLEU and the hallucination tendency. We mark the regions on the memorisation map in Figure 8. Examples are ran- domly sampled from those areas until they match the random dataset in the number of tokens. For those three datasets, we train three model seeds.
Why might this be the case? In image classi- fication, Feldman and Zhang (2020) observe that training examples with high CM usually are atyp- ical 'long-tail' examples and mainly improve per- formance visually similar test examples. Anal- ogous processes might be at play for translation. Yet, there may be benefits examples with high CM values even without similar test examples. Pre- liminary investigations (see Appendix D.2) sug- gest that there less redundancy among examples
We compare the specialised models model trained on 1M random examples (Fig- ure and observe that, indeed, the models are somewhat specialised, with the largest relative im- provement observed for the hallucination tendency. During training, examples with higher (predicted)
Results

---

-2.75
25.0
0.06
more than simply learning a function that trans- lates words from one language into another and rearranges words. It requires understanding which words form a phrase and should be translated to- gether, which words from the source should be ignored, which words can be copied from source to target, and in which contexts "eggs in a basket" are no typical eggs. NMT systems need memorisation of patterns that out of the ordinary.
buluuiitin tendency
0.05
-2.80
24.5
loggrobabiilty
0.04
24.0
0.02
0.01
Figure 9: Results of comparing specialised models (in colour) to models trained on randomly selected OPUS data (in gray, with hatches). Error bars show the SE.
There are, however, many open questions re- garding what memorisation is, when it is desirable and how to measure it. In this work, we took a step toward answering those by creating a map of the memorisation landscape for 5M datapoints. We used graded metrics based on CM to position each example on the memorisation map. We iden- tified salient features for each of the metries ($4), illustrated that we can approximate memorisation metrics using surface-level features ($5) and drew connections between models' performance and re- gions of the memorisation map ($6). We found that findings from other tasks and domains about CM transfer NMT: CM highlights examples that contribute most models' performance.
counterfartial men.
subsection
0.6
6.1
0.4-
0.0
other
hallucination
Figure 10: Using all hallucinations from in $ and $6.2. we trace the CM score of the unperturbed sequence and display the distributions. Error bars report SD.
CM scores can thus be beneficial. At the same time, Raunak et al. (2021) reported that when try- ing elicit hallucinations from the model using its training examples, examples with high CM scores lead to more hallucinations. To determine whether our results also reflect that, Figure 10 displays the distribution of CM scores associated with each hal- lucination from the current and previous subsection. For the models from $6.1 (trained on these exam- ples), but not $6.2 (not trained these examples), hallucinations indeed more associated with ex- amples with higher CM scores. Together, these re- sults mean that the examples with high CM scores are beneficial training examples, but when generat- ing translations using models trained on them, they are more likely to turn into a hallucination than examples with lower CM scores.
Furthermore, our results illustrate that memori- sation is not one-dimensional: CM assigns simi- lar scores to paraphrases and slightly inaccurate translations, examples with high CM scores can be beneficial and introduce vulnerabilities at the same time, and there are nuances about which region of the map is most beneficial depending on the perfor- mance metric used. We recommend caution when discussing different phenomena under the umbrella term of 'memorisation'. Instead, we encourage future work examining more memorisation maps to further our understanding of the intricacies of task-specific memorisation patterns.
All in all, the small-scale experiment presented here provides proof-of-concept;; even when using heuristies (i.e. applying the MLP to new datapoints) we can start use memorisation metries in a delib- erate way when training NMT systems. However, the hallucination results underscore that the rela- tion between examples with high CM scores and model performance is not straightforward: exam- ples that are most beneficial for systems' quality can introduce vulnerabilities. at the same time.
Limitations
We identify four main limitations with our work:
The experimental setup used is rather com- putationally expensive due to the repeated model training as explained in $3. We coun- teracted this by opting for a much much smaller dataset than state-of-the-art NMT sys- tems would use (OPUS contains hundreds of millions of examples per high-resource lan- guage pair), but it still limits the applicability the methodology to other tasks and for other researchers.
7 Conclusion
Learning the onput-output mapping that is repre- sented by NMT training data involves so much

---

We did not investigate the impact of major changes to the experimental setup, such as using a different model or model size or using a different or larger dataset. Even though our findings are expected to extend beyond our specific experimental setup, the precise memorisation scores we obtained are specific to our setup- e.g. a larger system is likely to memorise more, and systems trained for much much longer are likely to see increased memorisation. We do experiment with the data used in Appendix D.I.
for Doctoral Training in Natural Language Process- ing, funded by the UKRI (grant EP/S022481/1) and the University of Edinburgh, School of Informaties and School of Philosophy, Psychology & Language Sciences. IT is supported by the Dutch National Science Foundation (NWO Vici VI.C.212.003).
References
Rachit Bansal, Danish Pruthi, and Yonatan Belinkov. 2022. Mcasures of information reflect memoriza- tion patterns. In Advances in Neural Information Processing Systems (NeurIPS).
We discussed memorisation based on signals that can be observed after model training based on models' outputs. There is the un- derlying assumption that memorisation hap- pens over time and can thus be observed post- training. However, memorisation not only happens over time, is also expected mani- fest in a particular way in the space of the model parameters (e.g. see Bansal et al., 2022), which might not be observable by in- specting output probabilities of tokens. We recommend the analysis of spatial memorisa- tion in NMT as future work.
Peter L Bartlett Philip M Long, Gábor Lugosi, and Alexander Tsigler. 2020. Benign overfitting in linear regression. Proceedingy of the National Academy of Sciences, 117(48):300633300
Nicholas Carlini, Daphne Ippolito, Matthew Jagielski, Katherine Lee, Florian Tramer, and Chiyuan Zhang, 2022. Quantifying memorization across neural lan- guage models. In The Eleventh International Confer- ence Learning Representations (ICLR).
Nicholas Carlini, Chang Liu, Úlfar Erlingsson, Jernej Kos, and Dawn Song. 2019. The secret sharer: Eval- uating and testing unintended memorization in neu- ral networks. In 28th USENIX Security Symposium (USENI Security 19), pages 267-284.
All the languages that we consider in this article are considered be high-resource languages. While these might not be the languages most in need of language technol- ogy or analysis, the experimental setup of us- ing parallel corpus limited our possibilities to include lower-resource languages. When taking the intersection of existing NMT cor- pora, there were not enough remaining exam- ples when including low-resource languages. In preliminary experiments, we also experi- mented with Afrikaans (together with German and Dutch), and many of the qualitative pat- terns observed also applied to memorisation measures computed for Afrikaans. If we had used five languages but without the parallel data, it would, however, have been hard to dis- tinguish changes in the metrics due differ- ences in the dataset from differences between the languages.
Nicholas Carlini, Florian Tramer, Eric Wallace, Matthew Jagielski, Ariel Herbert- Voss, Katherine Lee, Adam Roberts, Tom Brown, Dawn Song, Ulfar Erlingsson, et al. 2021. Extracting training data from large language models. In 30th USENIX Security Symposium (USENIX Security 21), pages 2633-2650.
Kent K Chang, Mackenzie Cramer, Sandeep Soni, and David Bamman. 2023. Speak, memory: An archaeol- ogy of books known to chatgpt/gpt-4. arXiv preprint 2335v:2305.00118
Marta R Costa-jussà, James Cross, Onur Çelebi, Maha Elbayad, Kenneth Heafield, Kevin Heffernan, Elahe Kalbassi, Janice Lam, Daniel Licht, Jean Maillard, et al. 2022. No language left behind: Scaling human-centered machine translation. arXiv preprint arXiv:2207.04677 04672.
Verna Dankers, Christopher Lucas, and Ivan Titov. 2022. Can transformer be too compositional? Analysing idiom processing in neural machine translation. Proceedings of the 60th Annual Meeting of the As- sociation for Computational Linguistics (Volume 1: Long Papers), pages 3608 3626.
Acknowledgements
Tom Dietterich. 1995. Overfitting and undercomput- ing in machine learning. ACM computing surveys (CSUR), 22633222
We thank Brenden Lake and Jake Russin for their input during the early stages of this project, and thank Max Müller- Müiller-Eberstein for his comments on the write-up. VD is supported by the UKRI Centre
Daniel D D'souza, Zach Nussbaum, Chirag Agarwal, and Sara Hooker. 2021. A tale of two long tails. In ICML

---

2021 Workshop on Uncertainty and Robustness in Deep Learning.
Vikas Raunak and Arul Menezes. 2022. Finding memo: Extractive memorization in constrained sequence generation tasks. In Findings of the Association for Computational Linguistics: EMNLP 2022, pages 5153-5162.
Vitaly Feldman. 2020. Does learning require memoriza- tion? A short tale about a long tail. In Proceedings of the 52nd Annual ACM SIGACT Symposium on Theory of Computing, pages 954-959.
Vikas Raunak, Arul Menezes, and Marcin Junczys- Dowmunt. 2021. The curious case of hallucinations in neural machine translation. In Proceedings of the 2021 Conference of the North American Chapter of the Association for Computational Linguistics: Hu- man Language Technologies, pages 1172-1183.
Vitaly Feldman and Chiyuan Zhang. 2020. What neural networks memorize and why: Discovering the long tail via influence estimation. Advances in Neural Information Processing Systems (NeurIPS), 33:28811 2891.
Amartya Sanyal, Puneet K. Dokania, Varun Kanade, and Philip H. S. Torr. 2021. How benign is benign overfitting? In The Ninth International Conference on Learning Representations (ICLR).
Naman Goyal, Cynthia Gao, Vishrav Chaudhary, Peng- Jen Chen, Guillaume Wenzek, Da Ju. Sanjana Kr- ishnan, Marc' Aurelio Ranzato, Francisco Guzman, and Angela Fan. 2022. The flores-10) evaluation benchmark for low-resource and multilingual chine translation. Transuctions of the Association for Computational Linguistics, 10:522-530 538.
Holger Schwenk, Vishrav Chaudhary, Shuo Sun, Hongyu Gong, and Francisco Guzmán. 2021a. Wiki- matrix: Mining 135m parallel sentences in 1620 lan- guage pairs from wikipedia. In Proceedings of the 16th Conference of the European Chapter of the Asso- ciation for Computational Linguistics; Main Volume, pages 1351-1361.
Nuno M Guerreiro, Elena Voita, and André FT Martins. 2023. Looking for needle in a haystack: A com- prehensive study of hallucinations in neural machine translation. In Proceedings of the 17th Conference of the European Chapter of the Association for Compu- tational Linguistics, pages 1059-1075.
Holger Schwenk, Guillaume Wenzek, Sergey Edunov, Edouard Grave, Armand Joulin, and Angela Fan. 202 CCMatrix: Mining billions of high-quality parallel sentences on the web, In Proceedings of the 59th Annual Meeting of the Association for Compu- tational Linguistics and the 11th International Joint Conference on Natural Language Processing (Vol- ume 1: Long Papers), pages 6490-6500.
Adi Haviv, Ido Cohen, Jacob Gidron, Roei Schuster, Yoav Goldberg, and Mor Geva. 2023. Understand- ing transformer memorization recall through idioms. In Proceedings of the 17th Conference of the Euro- pean Chapter of the Association for Computational Linguistics, pages 248-264.
Rico Sennrich, Barry Haddow, and Alexandra Birch. 2016. Neural machine translation of rare words with subword units. In 54th Annual Meeting of the Asso- ciation for Computational Linguistics, pages 1715- 1725.
Eugene Kharitonov, Marco Baroni, and Dieuwke Hup- kes. 2021. How BPE affects memorization in trans- formers. arXiv preprint arXiv:21
Swabha Swayamdipta, Roy Schwartz, Nicholas Lourie, Yizhong Wang, Hannaneh Hajishirzi, Noah A Smith, and Yejin Choi. 2020. Dataset cartography: Mapping and diagnosing datasets with training dynamics. In Proceedings of the 2020 Conference on Empirical Methods in Natural Language Processing (EMNLP). pages David Talbot, Hideto Kazawa, Hiroshi Ichikawa, Ja- son Katz-Brown, Masakazu Seno, and Franz, J Och. 2011. A lightweight evaluation framework for ma- chine translation reordering. Proceedings of the Sixth Workshop on Statistical Machine Translation, pages 12-21.
Katherine Lee, Orhan Firat, Ashish Agarwal, Clara Fan- njiang, and David Sussillo. 2018. Hallucinations in neural machine translation. In NeurIPS 2018 Work- shop on Interpretability and Robusiness for Audio, Speech, and Language.
Fatemehsadat Mireshghallah, Archit Uniyal, Tianhao Wang, David Evans, and Taylor Berg- Kirk-aiikattrick. 2022. Memorization in NLP fine-tuning methods. In First Workshop on Pre-training: Perspectives, Pit- falls, and Paths Forward at ICMI. 2022.
Robert Östling and Jörg Tiedemann. 2016. Effi- cient word alignment with Markov Chain Monte Carlo. Prague Bulletin of Mathematical Linguisties. 106:125-146.
Jörg Tiedemann and Santhosh Thottingal. 2020. OPUS- MT building open translation services for the world. In Proceedings of the 22nd Annual Conference of the European Association for Machine Translation. European Association for Machine Translation.
Myle Ott, Sergey Edunov, Alexci Baevski, Angela Fan, Sam Gross, Nathan Ng, David Grangier, and Michael Auli. 2019. fairseq: A fast, extensible toolkit for sequence modeling. In Proceedings of the 2019 Con- ference of the North American Chapter of the Associa- tion for Computational Linguistics (Demonstrations)) pages 48-53.
Kushal Tinumala, Aram H Markosyan, Luke Zettle- moyer, and Armen Aghajanyan. 2022. Memoriza- tion without overfitting: Analyzing the training dy- namics of large language models. arXiv preprint NKiv:2205.10772

---

Mariya Toneva, Alessandro Sordoni, Remi Tachet des Combes, Adam Trischler. Yoshua Bengio, and Geof- frey J. Gordon, 2019. An empirical study of example forgetting during deep neural network learning. In The Seventh International Conference on Learning Representations (ICLR).
Ashish Vaswani, Noam Shazeer, Niki Parmar, Jakob Uszkoreit, Llion Jones, Aidan N Gomez, Lukasz Kaiser, and Illia Polosukhin. 2017. Attention all you need. Advances in neural information processing systems (NeurIPS). 30.
Elena Voita, Rico Sennrich, and Ivan Titov, 2021. Lan- guage modeling, lexical translation, reordering: The training process of NMT through the lens of classical SMT. In Proceedings of the 2021 Conference on Empirical Methods in Natural Language Processing, pages 8478 8491.
Biao Zhang, Philip Williams, Ivan Titov, and Rico Sen- nrich. 2020. Improving massively multilingual neu- ral machine translation and zero-shot translation. In Proceedings of the 58th Annual Meeting of the Asso- ciation for Computational Linguistics, pages 1628- 1639.
Chiyuan Zhang, Samy Bengio, Moritz Hardt. Benjamin Recht, and Oriol Vinyals. 2017. Understanding deep learning requires rethinking generalization. In Inter- national Conference on Learning Representations.
Chiyuan Zhang. Daphne Ippolito, Katherine Lee. Matthew Jagielski, Florian Tramèr, and Nicholas Car- lini. 2021. Counterfactual memorization neural language models. arXiv preprint arXiv:2112.122: 12993
Xiaosen Zheng and Jing Jiang. 2022. An empirical study of memorization in NLP. In Proceedings of the 60th Annual Meeting of the Association for Compu- tational Linguistics (Volume 1: Long Papers), pages 6265-6278.

---

A Experimental setup
A.1 Data collection
To construct parallel corpus of 1M sentence pairs across five language pairs, obtain the eng-deu, hhmp/lllssnni---P//ettbbba-lalll
eng-fra. eng-nld, eng-spa and eng-ita data from httpr//nn.////////.i.o..///////a
tree/master/data (version v2021 -0--07 .d). The raw intersection contained 4M sentence pairs, from which we select sentences based on four criteria:
1. The length of the source divided by the length of the target is between 3 and 3 2.
2. The punctuation ratio of the source and target sequence lies below 0.5.
3. Less than 30% of the words in the source appear in the target, as well.
4. More than 90% of the numbers contained in the target should also appear in the source sequence. -oi-ooomeeriall-Shheeeee ciaerrrarellSaallllal ike
The Tatoeba repository has the license International, which allows us use and redistribute the data, given appropriate attribution.
4.
A.2 Model training
Before commencing training, we tokenise the data using the Moses tokeniser, $ and then compute subtokens using byte pair encodings (BPE) (Sennrich et al., 2016) to create a joint vocabulary per language pair, performed using Fairseq, version 0.12.1. We train
with a size of 64k tokens. Model training encoder and 6 decoder layers, with embedding size and hidden size of 512, and feedforward size of 2048. All models were trained used the following setup, with the number of total training steps being dependent on the experiment conducted:
transformer-basee
To obtain memorisation scores $3, we trained for 100 epochs on training datasets of 500k sentence pairs. This involves model training beyond the point of convergence to investigate memorisation.
The remaining models discussed in the paper are all trained for 50 epochs.
We train using the following command, modelled after exemplar Fairseq translation setups. We did not further tune hyperparameters but did increase max-tokens to better utilise the GPU capacity.
fairseq-train <DATA_DIR>>
--arch <MODEL> --save-die <MODEL_DIR> --share all embeddings
--fp16 --max-update 200000
--optimizer adam --adammbett '00.9,0.3 98) --clip- 0.0
--1r 0.0805 -che-scheduler inverse_sqre
--warmup-uddated 4080 --warmup-inis init--1 '1e-07'
--Label-smoothing 0.1 --criterion label_
--dropout 0.3 --weeghh--eeaal 0001
--maxtooees 10000 --uddat--fre 2
-save-interval --max- <MAXEPOCH>
--seed <SEED> -wwlidate-innerreeeeee
--eval-bleu --eval salau-bee--rgss '("beam":5) -beal-bleu-rrmoee bpe
Tesla V100-SXM2-32GB GPUs are used for model train- ing in $3. We train each model on a single GPU, on which one epoch of 500k training set lasted up to 4 minutes, and full training approximately 6 hours. Training all seeds for the five language pairs thus cost 1.2k GPU hours. In $6 we train seeds for 54 coordinates using NVIDIA A100-SXM- 80GB GPUs, and the training of model can take up 2.5 hours. This thus cost approximately 400 GPU hours. Figure illustrates the BLEU scores on a development set over the course of training. At the time of writing, the top FLORES-200 'dev' performances on the OPUS-MT leaderboard are 40.4, 27.1, 51.5, 28.0 and 29.2 for DE. Nt., FR, Es and IT, respectively. Of course, our models trained on a fraction of SOTA MT datasets underperform, but our relative differences in BLEU across languages similar.
40
20
10
40
epoch
Figure 11: BLEU the evaluation dataset FLORES-200, when training to obtain the memorisation scores.
thttps://github hart:s:://gituub..mm//eeeee httph//...oooeeeericc//c///eeeee hhttps: htwpetttt..ooeeeeeooeeeeeeeeerra

---

B Extended discussion of approximating memorisation measures from $5
In $5, we questioned whether there is a way to predict memorisation metrics without actually computing counterfactual memorisation and the generalisation score, which are 'expensive" since they can only be computed when examples are not present in the training set. To do that, we trained one model per language pair for 50 epochs, acting as our diagnostic run from which estimate memorisation measures using training signals obtained for the full dataset:
Confidence and variability: the mean and standard deviation of the target likelihood averaged over all epochs (adapted metrics from Swayamdipta et al., 2020);
Final frain likelihood: the likelihood of the target in the final training epoch:
Forgetting: the sum of all decreases target likelihood observed for consecutive epochs (adapted
metric from Toneva et al., 2019);
Hypotheses' likelihood obtained in the final epoch. Uncertainty can aid in detecting out-of-domain
data (D souza et al., 2021), I), and hallucinations (Guerreiro et al., 2023);
We also included final train likelihood confidence since initial experiments suggested those two correlated most strongly with training memorisation and generalisation score, and counterfactual memorisation is known to be combination of those two signals.
Apart from the hypotheses' likelihood, these signals a model using teacher forcing. Firstly, we inspect what extent these signals naturally correlate with memorisation measures. Afterwards, we train shallow MLPs predict the memorisation metrics: firstly only using the datapoints' features discussed in $4, and, secondly, using the features and training signals. Those MLPs were trained for 20 epochs maximum, using Adam according to default hyperparameters in the sklearr karn. wtw.e.eeeaaaaaaaaaa...... neural_ nww.ork. neural_network. MLPClassifi fier class. The MLP takes 28 inputs when training with features only, and 28 6 when adding the training signals, and has two hidden layers of 100 units. It predicts all memorisation metrics at the same time.
you would naturally obtain while training
Results Do these signals correlate with memorisation measures? Figure 12a shows this for EN-NL. The strongest correlations (around 0.95 Pearson's 's is observed between the confidence and the generalisation score. The confidence score expresses when an example is learnt during training, as it will be high for examples that are learnt immediately, low for examples that are learnt very late, and close to zero for examples that are not learnt at all. These results suggest that that temporal indication of when an example is learnt, strongly correlates with the performance model would have on an example, if that example had been in the test set.
For counterfactual memorisation, correlations are much lower (understandably so since it depends both train and test likelihood), even when we look at the combined feature. As discussed in the main text, improve upon this by training MLP to predict the memorisation measures from both the features and the training signals. The main text discussed this for MLPs trained on EN-DE and applied other languages, but we can train on any language and still obtain strong results on the other languages. Figure 12b illustrates this for CM.
training memorisation-
0.57
0.27
2:43
0.42
5.47
2.92
0.4
2000 ano 012b36c0
0.000
Parrsons
train on langt
generallsation
0.0
0,055
counterfactual memorivation-
2.023
0.0157
.023
confidencs
vanabiily
mixtur
brgetiig
50
L1.00
-0,00
hye.
test language
(a) Correlations between training signals and metries (EN-NL) (b) Applying MLPs across languages when predicting CM
Figure 12: Correlations for predictive signals and memorisation measures in terms of Pearson's and comparisons MLPs' applicability across languages terms of the absolute difference between the predictions and the target.

---

C
Extended data characterisation feature analysis from $4
Experimental setup
Counterfactual memorisation is capped since negative counterfactual memorisation score is likely to be a side effect of noise in the estimation of the memorisation measures, and typically the train likelihood of the target should outperform the test likelihood. The examples for which we had to cap this measure are rare.
The features we analyse fall into three categories:
(1) Source/target-only features
|s] source length (x2, BPE tokenised and white space tokenised)
|1] target length (x2, BPE tokenised and white space tokenised)
T (x2, BPE tokenised and white space tokenised)
Is
Average log frequency source tokens (x2, BPE tokenised and white space tokenised)
Average log frequency of target tokens (x2, BPE tokenised and white space tokenised)
Minimum log frequency of source tokens (x2, BPE tokenised and white space tokenised)
Minimum log frequency of target tokens (x2, BPE tokenised and white space tokenised)
Number of repetitions of this target
Segmentation of the source:
means no segmentation beyond the token level |вpø|
Iswal
Segmentation of the target: 1 -
[wwl
0 means segmentation beyond the token level
Itarg!
Digit ratio: how many tokens in the source are digits
Punctuation ratio: how many tokens in the source are punctuation
(2) Source-target interaction features
Token-level Levenshtein edit-distance between source and target
Comparison by backtranslation, obtained with Marian-MT models trained on OPUS by Tiede- mann and Thottingal (2020), by computing the token-level Levenshtein edit-distance between source and target
|s] [4] (2x, BPE tokenised and white space tokenised)
Ratio of unaligned source words, alignments are obtained with ef lomal (Östling and Tiedemann,
2016)
Ratio of unaligned target words, alignments are obtained with eomal (Östling and Tiedemann,
2016)
Alignment monotonicity, computed as the Fuzzy Reordering Score, implementation obtained from Voita et al. (2021)
Token overlap: how many tokens from the source also occur in the target
Word overlap: how many words from the source also occur in the target, excluding punctuation

---

Additional feature visualisations We provide the correlations between these features and our memori- sation metries (see Figure 13a), along with the correlations between features (sce Figure 13b), and also include additional feature visualisations in Figure 14.
0.5
TM-
022 0.04 040
000
030 0:28
613-333
Speerrmaat
GS
0.100.000 020.0000000000000000 5.37 0.0063 RJS
0.9 111 031011 111122 92292211 12960.0000 6.1-9224
-0.0
St length trg length
sic length tokenised
trg length tokenised
length difference takenised.
sengin ratio erc trg tokenised
length difference trg-
leng ratio insiation ed
wdft distance sre trg
word overfap
Loker overlap p-
digit ratio
punctuant ratio
avg sir freq
avg trg freq min freqq min trg frea
ane are frea tokenise ave tno free Mokennie min min ing free
st free toleriiee
rig eeoeeeaaee dasiiee
big wew.enaad.ee
0.5
waalgnee
ba
(a) Correlations between memorisation metries and features.
trg length
src lingth tokenised
0.
trg length tnkenised
inbenised-
length takenised
length difference
longth ratio sic tra-
backtramlatiion
word
taken everlap
digit ratio
punctuation relnn
avg arc treq
min sre free
min trg freq-
sng src freq takenised
avg trg trg tokenised
0.2
freq takenised-
min tng freq takenised trg repested unaligned fwd- unalggned dering-
-0.4
segmentation
segmentation
viEuo
ddeaii sood
and ur ti
retoo
bou ca unu
seettagg bay but Bae
peeeuree beg fi Biee
500 0000
cooogge bou 6z mju
unur
(b) Corvelations among features, displaying correlations >0.5 or <-0.5. The figure is mirrored in the diagonal, but both sides are shown to ease inspection by the viewer.
Figure 13: Correlations between memorisation metrics and features, and correlations among features.

---

danety
10
0.5
0.0
0.3
1.0
0.0
0.5
1.0
0.0
0.5
training memerisation generalisation
counterfactual
training memorisation
generalisation counterfactual mem.
(a) Average souree frequency
(b) Average target frequency
1.0
training memerisation gencralisation countertactual
training memorisation gencralisation counterfactual
(c) Minimum source frequency
(d) Minimum larget frequency
RATOUB
4-
denalty
0.3
1.0
training generalisation counterfactual
training memorisation
generalisation
counterfactual
(e) Punctuation ratio
(f) Digit ratio
dencity
0.0
1:0
0.0
training memorisation
generalisation score
counterfactual
training memorisation generalissation counterfactual mem.
(g) Segmentation degree
(h) Segmentation degree target
ocinity
2.5
0.0
1.0
0.0
0.5 generalisation score
0.0 counterfactual mem.
1.0
0.3
0.0
0.5
0.0
0.5 co counterfactual mem.
training memorisation
training memorisation
generaliation score
(i) Source length
(i) Target repeated
dersby
5.0
0.0
1.0
0.0
0.5
0.0
0.5
1.0
0.0
0.5
1.0
0.0
0.5
1.0
0.0
0.5
training memortsation
gencralisation score
counterfactual mem.
training memorisation
ocnera liattoon score
counterfactual mam.
(k) Backuranslation edit distance
(I) Edit distance src-trg
MAM
knnth
condly
0.0
0.
1.
0.0
05
1.0
0.0
0.3
1.0
0.0
0,5
1.0
0.0
0.5
1.0
0.0
0.5
1.0
training memorisation
generalisation score
counterfactual mem.
training memarisation
generatisation Score
counterfactual mem.
(m) Length ratio sre/trg (tok.)
(n) Fuzzy reordering score
20
30-
0.0
0.5
1.
0.0
1.0
0.5
0.0
0.5
0.0
1.0
0.0
0.5
training memorisation generalication score counterfactual mem.
training memorisation
genera score
counterfactua mem.
(o) Ratio of unaligned source words
(p) Word overlap
Figure 14: Visualisation of various features. We show their distribution the three memorisation metrics employed.

---

Manual annotation We uniformly sample 250 examples from the EN-NL memorisation map, with source lengths 1 such that 10 1 15. We annotate them using the following labels, where multiple labels can apply to the same example:
(Almost) Word for word: if the target is almost a word-for-word translation of the source, with very minor rephrasing or change word order. For example "In only days, without food or water, Society collapses into chaos." + "In slechts enkele dagen, zonder eten of drinken, stort de maatschappij in chaos."
Paraphrase: if the target generally expresses the same meaning the source, but using different wording, e.g. "Let me make observations concerning the Green Paper slot wil ik nog enkele opmerkingen maken over het Groenbock over havens".
ports." <> "Tot
Inaccurate:
if the target is an incorrect translation or discusses something that the source does not warrant, c.g. 'sancties' this source-target pair "We ask you to form a worldwide front against war and NATO." + "Wij vragen u om een wereldwijd front tegen de corlog en sancties te vormen." " Adds content: if the target introduces new information. "And My curse will be upon you until the Day of Judgment." < "En voorwaar, op jou rust Mijn vervloeking, tot de Dag des Oordeels." Depending on how relevant the added content is, the pair can still be word-for-word paraphrase / inaccurate.
Removes content: if the target removes content from the source, as the case in "He married his beloved wife, Penny, in 1977 and raised family." + "In 1977 trouwde hij met Penny en samen brachten ze een gezin groot." " Here 'his beloved wife' removed in the target. Depending on how relevant the removed content is, the pair can still be word-for-word paraphrase inaccurate.
Different formatting: the target changes the punctuation or the capitalisation, e.g. "It difficult to negotiate with people who CONFUSE AUSTRIA WITH AUSTRALIA. "ETTRALIA""""mmmnnn"" "S"mmenweeee met mensen die Oostenrijk verwarren met Australiē is lastig." This can co-occur with other changes.
To create the visualisation in Figure 4, we post-process the labels in the following way: (1) we create a separate label 'word for word'. This contains the examples that are only annotated with the 'almost word for word' label and others filter the examples that literally word for word, from the ones that do have slight changes, such as formatting changes or the removal of few words. (2) we restrict 'addition' and 'removal' of content cases that are not labelled as 'almost word for word', since in those cases the addition/removal is considered very minor and non-essential. This helps us to identify the cases where addition/removal actually affects the meaning difference between the source and target.

---

D Additional results
D.1 Noisy memorisation continuum
$3 and Appendix A.1 we detailed how we obtained the parallel OPUS data, that has targets for the same souree sequenees in five different languages, and filters noisy data in the process, such as data that has a lot of overlap between the source and target, data with an extreme length difference between source and target, etc. What happens if we compute memorisation measures a random OPUS subset, instead? We take the OPUS-100 subset for EN-NI. that Zhang et al. (2020) released, containing 1M randomly sampled examples.
Differences Figures 15a and show the memorisation maps for parallel OPUS and OPUS-100, re- spectively. The two most striking differences are: (1) OPUS-100 has many more datapoints with a counterfactual memorisation score close to (dark red, in the bottom right comer). (2) There are many more examples with a low generalisation score. This might appear unintuitive to the reader, but does not necessarily mean the OPUS- 100 models are worse in terms of their translations' quality; it just means the dataset is more heterogeneous, and there are more source-target pairs with unexpected tokens in the target (remember that we are computing geometric mean over the target tokens' probabilities).
These sets of 1M examples actually have some examples in common. In Figure 15c, illustrate how the scores compare between parallel OPUS and OPUS-100. These scores do strongly positively correlate, but are still quite different terms of absolute numbers. Hence, how many examples will be memorised, and what exact score is assigned to an individual example does depend on dataset composition.
Similarities Yet, what is perhaps more relevant is that when we measure the correlations between the features we assigned to datapoints and the memorisation metrics as is illustrated in Figure 16 - the same patterns emerge as we pointed out in $4, with stronger correlations across the board: (backtranslation) edit distance and unaligned words matter for training memorisation and generalisation score, and length, word frequency and segmentation features matter most for counterfactual memorisation.
C2
0.-
05
edss also 3i0 siis odso eim oáss u.
0.5
0/0
0.5 1.00
0.0
0.5 050 a 1.00
manng memonvvahon
tninnng memotiation
(c) Differences between memorisation
(a) Parallel OPUS
(b) OPUS- 100
scores for parallel OPUS and OPUS-100
Figure 15: Illustrations describing the differences between the memorisation spectrum for EN-NL. computed using the filtered parallel OPUS data vs. the OPUS-100 data (random OPUS subset).
0.5
Spearman's
GS
15 314 0.2 326-1203) 639-322
0..
CM
4444
0.00011 0.17 013
000 2.33 682 22-0222 0.200.5200 32 20 033 0.00 10 0.22
length
length-
length tokenised
trg length tokenised
length difference takenised
length ratio sre trg tokenised
length gingth difference trg-
backkrransion inttannlationn ed
edit distance sre trg
word overtap
token overlap
diglt ratio
punctuation ratio
avg src freq
avg trg freq
min src freq
min trg frea
one soe frea tokenised
svg via free dosenisse
min sre iee oikenisae
mnm tng froq tokeen tokeniisdd
Peul Aoey
nay
ri regerd
length
Figure 16: Correlations between memorisation metrics and features.

---

D.2 Performance impact
$6 we observed that examples with higher CM scores tend to be more beneficial for multiple MT performance metrics. In general, this fits with the long tail theory of Feldman (2020); Feldman and Zhang (2020) but why might this be the case? In computer vision classification tasks, Feldman and Zhang (2020) observe that examples with high CM scores are beneficial when making predictions for visually similar test examples. We would like to highlight two additional patterns observed in relation to high CM scores. Firstly, we would like examine the log-probability performance impact more closely. Figure 18 provides the prg-prbbaiility per coordinate. where darker means more relevant. Why are the bottom rows, and the examples with high CM in particular, most relevant? This metric is computed using the target tokens' probabilities, that are easily negatively affected if there are some unexpected target tokens. Coordinates in the bottom right might be relevant because they include somewhat 'noisy" data, which increases uncertainty in the model during training, which thus smooths the output probability distribution. To examine whether our data reflects that, we put tokens from the FLORES 'dev' set in buckets based on the mean token probability that they have in the predictions of all models trained in $6.1. We compare these token probabilities to those from models that leave out examples with certain CM. If examples with high CM removed during model training (e.g. row 0.7 in Figure 18), the token probabilities for buckets with a relatively low probability decrease. Vice versa, when removing examples with low CM, the token probabilities for buckets with a relatively low probability increase. This suggests that by removing examples with a high CM, the output distribution becomes less smooth.
Secondly, we would like to point out that examples with a high CM score generally have less redundancy than examples with low CM score (in particular, compared to examples with high training memorisation and generalisation score). The corpus that we constructed has 1M umique source sentences, so none of them are repeated, but, nonetheless, there are sentences that are more alike than others in terms of n-gram count, explaining that redundancy. To illustrate that, Figure 19 conveys the ratio of unique trigrams vs. all trigrams in the data from particular coordinate.
0,0075
1.0
1.0
log probability
comparison to mcan
0.0050
urique 3 g-aaam radio
-3.0:0
0.45
generalisation score
0.8
-3.015
0.0025
generalisation scare
0.60
3,000
0.75
-2
0.0000
0.6
2.970
0.6
-0.0025
CM
0.4
-0.0050
0.2
0.4
0.5
-0.0075
0.2
0.7
0.2
-0.0100
0.0
0.6 0.8 1.0 training memorisation
0.4
word probability
0.0
0.0
0.2
0.4
0.6 0.8 1.0
training memorisation
Figure 18: Differences between coor- dinates with three different CM scores, compared to the mean of all models, per group of words with certain probabil- ity.
Relevance of re- gions for the log-probability met- ric. Darker means more relevant.
Figure 17:
Figure 19: Ratio of unique tri- grams over all trigrams, per coor- dinate.

---

D.3 Replacing likelihood with BLEU
Counterfactual memorisation the difference between the train and test probability of the target. Yet, NMT, we generate a sequence, and we typically do not use the probability the target as an adequate measure of success since that metric is severely affected by the sequence length. In the main paper, we instead used the geometric mean of the target token probabilities to compute the memorisation metrics. Here, we consider generating sequences using greedy decoding and replace the train and test probabilities with BLEU scores.
Figures 20a and 20b illustrate how the memorisation map changes when we switch to BLEU-based metrics, using the EN-NI. data. The examples generally lie closer to the diagonal, and the computation of the memorisation metries is less stable across models: comparing CM scores from models with 20 seeds those of 20 other seeds leads to Pearson's r=0.84 (it was 0.94 for the LL-based scores). When comparing the two sets of LL- and BLEU-based memorisation metrics, the TM and GS metrics correlate strongly with 0.80, although the CM's correlation is substantially lower (p = 0.54). Examples that the model fully memorises (BLEU>99 or LL>0.9) do reside in the same area on the two maps, as shown by Figures 20c and 20d.
p 0.89,
generalisat laaion score (Lu)
Goneral Goncrollation (LL)
0.75
0.75
0.75
0.75
0.25
0.2
0.25
0.25
0.00
0.00
0.00
0.00 0.25
0.50 0.75 1.00
0.30 0.55
0.50
0.75 1.00
0.00
0.5
0.50
0.75 1.00
0.00
0.25 0.00
0.75 1.00
training memorisation (LL.)
training memorisation (BLEU)
training memoriattion ILL:
trainin memorisation (BLEU)
(a) LL-based map
(b) BLEU-based map
(c) Perfect TM-BLEU examples (d) Perfect TM-LL examples
Figure 20: Illustration of how the memorisation map changes when we compute the memorisation metrics using BLEU instead LL as performance metric, for Ex-Ni.. Colour represents CM scores.
In $4, we discussed how various surface-level features the source and target correlate with the different memorisation metrics. Figure 21 provides the same results, but then for the BLEU-based metrics. Although the majority of the correlations are lower in terms of absolute p, the same patterns apply in terms of the previously identified positive/negative correlations.
0.5
TM
0330 DLS 0102E1 ILST US B. BAT 3. 0.226.55 - 15 0.15 211
(033 BUM
5p opearman
0.0
CM
612 0 0.0 3.31 638
0.00. 616 22-020 1..4 1.7 17-222 6.26 6-33 0.06..2 6.3conno 5.00
src length
length-
length tokenised
trg length tokenised-
length diffrrncce takenised
length ratio are takenised- length difference length ratio trg-
backtranssation od
edit distance nce sre trg
word overiap taken overlap
digit ratio
punctuation ratio
avg src freq
avg trg freq
min are freq
min trg frea
sin ure freg takenisen
avg vi freq tokenised
min sTc freq to tokenssed nised
min tgg freeq take takenised
trg seneeatee delllggnnn med fwdd
soooona pa
unm
Figure 21: Feature correlations for the BLEU-based memorisation metries.


Reviews:
[{"review_id": "VklKJqF0Sv", "reviewer": "Reviewer_cDFT", "summary": "", "questions": "1. Line 501, what does \"we create training subsets by removing the nearest examples\" mean?\n2. Figure 8, why are the areas of log-probability and BLEU/hallucination tendency so different? It seems that the areas of log-probability are more similar to Figure 7(a).", "limitations": "", "rating": 4, "confidence": 3, "soundness": 4, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "We’re glad to hear you find the research question worth exploring!\n\n**Regarding the limited applicability of our findings**: we indeed point out that the counterfactual memorisation values we obtain are specific to our experimental setup. However, at the same time, we do demonstrate that the relation between datapoints’ characteristics and high/low training memorisation, generalisation score and counterfactual memorisation values are robust. We show that in three ways: \n1. by computing across-language generalisation of the MLPs trained to predict TM, GS and CM from datapoints’ characteristics (see Section 5)\n2. by replacing the training data with randomly selected OPUS data (as opposed to filtered data) in appendix D.1\n3. by replacing the likelihood metric (see line 187) with BLEU scores in the computation of TM, GS and CM in appendix D.3\nIn each of these scenarios, the salience of datapoints’ characteristics is similar, demonstrating robustness against changes in the experimental setup.\n\n**Regarding question 1, line 501**: In section 6.1, we train NMT models on the same training data as used in section 3, but with some examples removed (up to 750k tokens, which is approximately 3-5% of the input-output pairs). Which of the examples are removed depends on the coordinates from the memorisation map: we create such modified training sets for 55 coordinates total (all coordinates in the lower right triangle of the memorisation map) and when making the modified training set for one coordinate, we remove the 750k tokens closest to that coordinate on the memorisation map.\n\nLet me give an example to clarify how this works: To create the training subset for coordinate (0.2, 0.1), we go to the memorisation map, take the examples with a TM value of ~0.2, and a GS value of ~0.1, until we have the 750k tokens closest to coordinate (0.2, 0.1). We remove them from the training data, and train the model. The lower the performance of this system, the more beneficial those 750k tokens apparently were.\n\n**Regarding question 2**: we opted for two different areas because log-probability has, relatively speaking more `most-beneficial’ coordinates in the bottom half and the lower left corner compared to BLEU and the hallucination tendency. Log-probability’s most beneficial areas can also be seen in Figure 17, appendix D.2.", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "1. The article is well written and structured.\n\n2. The article has sufficient experimental studies to demonstrate the findings.\n\n3. The research question this work explored is useful to understand memorisation behaviour and solve hallucination problem which are widely existed in current LLMs. The work can also help reseachers to improve the model performance by manipulating used datapoints.", "reasons_to_reject": "1. As the authors stated in the Limitations, though computationally expensive experiments are conducted, the applicability of the findings this work observed is questionable as the experimental setup is limited.", "questions_to_authors": "1. Line 501, what does \"we create training subsets by removing the nearest examples\" mean?\n2. Figure 8, why are the areas of log-probability and BLEU/hallucination tendency so different? It seems that the areas of log-probability are more similar to Figure 7(a).", "paper_topic_and_main_contributions": "This work builds a memorisation map and investigates the relation among datapoint characteristics, Counterfactual Memorisation (CM) metric, and model performance in a multilingual neural machine translation scenario by large-scale experiments. The empirical results reveal new observations:\n1) the memorisation can be influenced differently by characteristics of a datapoint.\n2) the memorisation can be predicted by characteristics of a datapoint.\nthat can be instructive to researchers and other generation tasks.\nand also validate some findings:\n1) datapoints with high CM value are more beneficial to model training.\n2) examples with high CM scores lead to more hallucinations.", "excitement": 4, "reproducibility": 3, "ethical_concerns": "No", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {}, "extra_text": {}}, {"review_id": "c2t6C5gbpy", "reviewer": "Reviewer_pPXn", "summary": "", "questions": "Question A: The paper introduces the \"Memorisation Cartography\" approach to understand the continuum between memorization and generalization. How do you envision this methodology being adapted or extended to other NLP tasks beyond NMT?\nQuestion B: The experiments presented, while comprehensive, primarily focus on certain datasets and languages. Could you elaborate on how the findings might generalize to low-resource languages or more diverse domains?\nQuestion C: Given the deep dive into the memorization-generalization continuum, are there other factors or considerations in NMT that might interact with these phenomena? How might these interactions influence the overall performance and behavior of NMT systems?", "limitations": "", "rating": 3, "confidence": 3, "soundness": 4, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "Many thanks for the elaborate review; your comments encourage us to elaborate on the wider applicability of the experiments and our approach in the paper.\n\n**Regarding the depth of the article, the need for further validation and lack of generality**: We opted for an in-depth analysis of the presented memorisation maps without experimenting with training procedures and model architectures. Yet, we do present experiments for 5 languages (main paper), and also included further experimentation with changing the data (Appendix D.1) and changing the way memorisation is measured (Appendix D.3) for a subset of the experiments, which confirmed that our findings are qualitatively robust. In fact, our training data selection procedure (creating parallel corpora) was tailored to validate that the results hold within this group of languages, and the training data comes from a variety of domains (see Question A). Those parallel corpora could not be constructed for low-resource languages. While we recommend further experimentation with different languages and larger corpora, the training procedure and model architecture represent standard practices in the field. As a result, we maintain that our findings provide appealing lessons for NMT practitioners as is. Note also that our experimentation required 1.6k GPU hours, which is why we prioritised experimenting with languages and data over evaluation across architectures, which might not have been the best use of our resources.\n\n**Regarding the absence of practical implementations:** The main focus of our work is, indeed, analysing the memorisation maps. We do provide a small proof-of-concept implementation in section 6.2, and would be happy to be more explicit about further suggestions for practical implementations. To name a few thoughts that we have:\n- The training data that is easiest to translate (top right corner) can make a model overconfident in terms of its log-probabilities (section 6.1, appendix D.2). Downsample or filter data with word-for-word translations prior to model training. \n- Source-target pairs with low overlap or high backtranslation distances have low TM scores. Potentially, one could improve NMT system's ability to paraphrase by upweighing examples in the lower left corner of the memorisation map.\n- The memorisation map can be used to create a learning curriculum for NMT. To give an example: data points from the top right corner are taught first (`basic translations’), followed by examples with a high CM score.\n\n**Regarding question A:** The metrics we used to compute the memorisation map were NMT specific, but the approach as a whole could easily be applied to other tasks; one would have to redefine TM and GS, and compute task-specific datapoint features. We would be very interested in seeing more memorisation maps and explorations of which features drive data points’ positions on the maps for a range of NLP tasks. In particular, we recommend explorations that contrast sequence classification and sequence-to-sequence learning and, more generally, comparisons of multiple tasks that use the same data. After all, whether a data point requires memorisation or not depends on the task one is working on. We hope our study can provide an in-depth example of how to set up such a study and will inspire others to ask similar questions for other tasks.\n\n**Regarding question B:** Firstly, the data we used is already quite diverse since OPUS includes data from many different sources (among others, Wikipedia, books, WMT-News, OpenSubtitles, web-scraped text / ParaCrawl, Europarl). The data used was filtered (see Appendix A), and we do examine the effect of using randomly sampled data for a subset of the experiments (see Appendix D.1), in which case our results remain qualitatively similar. Our results thus do generalise when varying some parameters of the experimental setup.\n\nStill, when making substantial changes to the experimental setup, such as using very low-resource languages, the findings might differ. The memorisation scores obtained depend on the **ratio** of the dataset size and the model capacity. For lower resource languages, the model will be able to obtain higher training memorisation scores across the board, and the examples from the lower left corner of the memorisation map will move to the right. This will influence the results obtained. Examples with a high CM score may no longer be beneficial in terms of model performance when that ratio becomes too small, but further studies would be required to confirm that hypothesis.\n\n**Regarding question C:** Our approach in the paper is data-centric; we examine the role of data points in the memorisation process, but, of course, there are other factors contributing to memorisation. An obvious consideration is training time: two systems with similar validation BLEU scores can be very different in terms of memorisation (maps). Other considerations concern the computation of the loss (label smoothing explicitly discourages memorisation, whereas teacher forcing might promote memorisation) and the type of model used. We do not experiment with the training procedure in the paper but encourage future work to take a model-centric approach to memorisation since the data and model interact when enabling memorisation.\n\n**Regarding the reproducibility**: We noticed that you indicated that the results might be hard to reproduce. We would like to point out that we included the anonymised codebase in the paper, discussed GPU architecture details and training commands in Appendix A, and that the training data from OPUS is publicly available. Due to size restrictions, we cannot put the memorisation map in the codebase, but we aim to make the numbers we obtained available to interested readers for further analysis, and already provide a subset of the memorisation maps through the public demo. Could you indicate what in the paper we could improve to make our work more easily reproducible?", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "1. Innovative Exploration of Memorization vs. Generalization: The paper offers a groundbreaking perspective on the memorization-generalization continuum in NMT, supported by the novel \"Memorisation Cartography\" approach. This deep dive into a fundamental dilemma is both timely and essential for the evolving NLP landscape.\n\n2. Robust Experimental Foundation with Broad Implications: The thorough experiments presented validate the research's claims and offer insights with wide-ranging applications. These findings can guide the design of efficient NMT architectures, especially beneficial in low-resource scenarios, and pave the way for optimized model performance.\n\n3. Significant Theoretical and Practical Contributions: Beyond its immediate application to NMT, this research enriches the theoretical understanding of neural network behaviors. It also opens avenues for future work, with potential extensions to various NLP tasks and strategies", "reasons_to_reject": "1. Potential Lack of Generality: While the paper offers insights into the memorization-generalization continuum in NMT, it might be constrained in its applicability. The findings, although significant, might not extend seamlessly to other NLP tasks or even to all NMT architectures.\n\n2. Need for Further Validation: Although the paper presents various experiments, additional validation on a broader range of datasets, including more diverse languages and domains, would have strengthened the claims. The current experimental setup might not capture all nuances and complexities of real-world translation tasks.\n\n3. Absence of Practical Implementations: The paper, while rich in theoretical content, lacks a clear bridge to practical implementation. Concrete strategies or tools that harness the presented insights for real-world applications would have elevated the paper's utility.\n\n4. Depth over Breadth: The paper's deep dive into the memorization-generalization continuum, while commendable, might come at the expense of exploring other equally pertinent aspects of NMT. A more holistic approach, considering other factors influencing NMT performance, could have provided a rounded perspective.", "questions_to_authors": "Question A: The paper introduces the \"Memorisation Cartography\" approach to understand the continuum between memorization and generalization. How do you envision this methodology being adapted or extended to other NLP tasks beyond NMT?\nQuestion B: The experiments presented, while comprehensive, primarily focus on certain datasets and languages. Could you elaborate on how the findings might generalize to low-resource languages or more diverse domains?\nQuestion C: Given the deep dive into the memorization-generalization continuum, are there other factors or considerations in NMT that might interact with these phenomena? How might these interactions influence the overall performance and behavior of NMT systems?", "paper_topic_and_main_contributions": "This paper delves into the intricate dynamics of memorization and generalization in the realm of Neural Machine Translation (NMT). By mapping out the continuum between these two facets, the authors aim to unravel the nuanced trade-offs that models make when faced with the dual challenges of retaining specific training examples and generalizing to unseen data. It also presents a meticulous exploration of the intricacies of NMT models, shedding light on their behaviors and propensities. Such insights are invaluable in advancing the field of machine translation and in optimizing model performance by striking the right balance between memorization and generalization.", "excitement": 3, "reproducibility": 3, "ethical_concerns": "No", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {}, "extra_text": {}}, {"review_id": "oa1pXLh5Ms", "reviewer": "Reviewer_Jciq", "summary": "", "questions": "", "limitations": "", "rating": 4, "confidence": 4, "soundness": 4, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "Thanks for the acknowledgement of our detailed experimentation; we hope our results can contribute in a positive way to memorisation studies in general but can also benefit machine translation and sequence-to-sequence learning communities.\n\nTo reiterate our novelty and motivation from a fresh perspective:\n- Our main **motivation** is the need to understand memorisation – what leads a model to memorise some data points, without ever learning some others – and the need to understand whether or not this memorisation is problematic, i.e. whether it negatively influences the model’s performance.\n- The **novelty** lies in approaching memorisation as a graded phenomenon, examining datapoints characteristics across the entire training dataset, instead of focusing on a small percentage of the data that has been marked as problematic, as is the predominant approach in the related work (see Section 2 of our paper).", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "(1) Overall, the paper is written well and easy to follow; (2) It shows detailed experimental results and discussions about memorization in Neural Machine Translation; (3) The experiments are sufficient and the results are comprehensive and convincing.", "reasons_to_reject": "The novelty is limited and the motivation is not strong.", "questions_to_authors": "", "paper_topic_and_main_contributions": "This paper addresses two questions for neural machine translation models, which are what determines a datapoint’s position on that spectrum, and how does that spectrum influence neural models’ performance. The experimental results illustrate that memorization is not one-dimensional.", "excitement": 4, "reproducibility": 4, "ethical_concerns": "No", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {}, "extra_text": {}}, {"review_id": "JRxanE2WsS", "reviewer": "Reviewer_5nce", "summary": "", "questions": "A. You present the correlation across seeds when using half the data. How would the CM scores change if you included larger subsets of the data for training (as overall model performance would likely improve)?\n\nB. In figure 6b, why is it harder to predict CM than TM or GS?", "limitations": "", "rating": 4, "confidence": 4, "soundness": 4, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "Great to hear that you think our work is thorough and well-motivated. We hope to contribute to a better understanding of NMT systems and encourage more research into memorisation for other NLP tasks, as well.\n\nDuring the discussion period, we hope that you could elaborate on your curiosity about English on the target side. Our own expectation is that that would lead to results that are quantitatively different but qualitatively largely the same.\n\n**Regarding question A:** When computing CM scores using models trained with larger training sets, the main difference we expect to see in the CM scores is that data points receive scores that are closer together. As you point out, the overall model performance would improve (and thus the GS score would improve), but if the model capacity doesn’t change, the TM score might slightly decrease, thus bringing the CM scores closer together across the board.\n\n**Regarding question B:** CM is a combination of TM and GS (where the latter is subtracted from the former). If the model inaccurately predicts TM and inaccurately predicts GS, those errors compound when the model predicts CM. As a result, it is not unexpected that that metric is harder to predict using an MLP.\n\nThanks for your suggestion regarding improving Figure 3, we can definitely implement that!", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "This paper could help researchers better understand the behavior of neural machine translation systems.\n\nThe approach is well-motivated and simple. The analysis is quite thorough and presented logically.\n\nThis paper could potentially lead to data up/downsampling techniques to improve machine translation quality (although this is not yet established).", "reasons_to_reject": "The findings would be more convincing if there were models with English as the target language (i.e. opposite direction).\n\nThe datasets are quite small, although this is understandable given the high compute requirements. The relationship between TM/GS vs model/dataset sizes is unclear.", "questions_to_authors": "A. You present the correlation across seeds when using half the data. How would the CM scores change if you included larger subsets of the data for training (as overall model performance would likely improve)?\n\nB. In figure 6b, why is it harder to predict CM than TM or GS?", "paper_topic_and_main_contributions": "This paper analyses neural machine translation models by mapping source/target pairs onto a memorisation/generalisation map. To create this map, the authors train multiple systems on overlapping subsets of the data, and then compare the length-normalized probabilities P(Y|X), averaged on subsets where (X,Y) is part of the training data (training memorization (TM)) or not (generalisation score (GS)). The different of these scores is known as the counterfactual memorisation (CM).\n\nThe authors identify correlations between these scores and surface-level features of the input pairs, and then build models to predict TM/GS/CM. They also analyze how example from different regions of the map impact performance.", "excitement": 4, "reproducibility": 3, "ethical_concerns": "No", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {"Typos_Grammar_Style_and_Presentation_Improvements": "The curve labels in figure 3 should be ranges of values, not a single scalar. If that would make the figure too cluttered, maybe expand the description of the figure."}, "extra_text": {}}]

Author rebuttal:

Meta-review: There is clear consensus amongst the reviewers that this is a strong paper both in terms of Soundness and Excitement. The weaknesses and questions raised by the reviewers were sufficiently answered by the authors, as is clear in the review scores and discussion post-rebuttal. The remaining weaknesses are mainly suggestions that would make the work even better, rather than fill in some big gap in the current work. Overall the paper is well written, proposed methodology novel and transferable, backed by sufficient experimentation and targets an important problem in the field of NLP. It has _strong soundness_ and _strong excitement_.

The following is a summary of the strengths, weaknesses and scores across the four reviews:

**Strengths:**

- Well written and well structured (**cDFT**, **Jciq**, **5nce**)
- Claims made in the paper are backed by sufficient experimental results (**cDFT**, **pPXn**, **Jciq**, **5nce**)
- Paper works towards an important problem of generalization/memorization/hallucinations, which are very pertinent to the current state of NLP (**cDFT**, **pPXn**)
- Provides actionable suggestions on how one can improve models by modifying their data (**cDFT**, **pPXn**, **5nce**)
- Suggested methodology of memorization mapping is novel and brings a new perspective to knowledge within a network (**pPXn**, **cDFT**, **Jciq**)

**Weaknesses:**

- Application to low resource settings is not obvious (**pPXn**)
- Reviewers agreed that the following are *not major weaknesses* after rebuttal:
	- Experimental design limits the findings to the specific models and settings (**cDFT**, **pPXn**, **5nce**)
		- Rebuttal: Authors agree, but also note (and reviewers allude to as well) that the experimentation targets a few languages and datasets, and goes into depth in each of these instead of covering a breadth of models. However, the depth of experiments already uncovers a new perspective sufficiently.
	- Breadth of the experimentation is limited in lieu of depth (**pPXn**, **5nce**)
		- Rebuttal: The depth of experiments already reveals a lot of insights, and the breadth was inherently limited by computational cost


**Scores in decreasing order of confidence:**

|      | Soundness | Excitement | Reproducibility | Confidence | Additional Notes             |
|------|-----------|------------|-----------------|------------|------------------------------|
| 5nce | 4         | 4          | 3               | 4          |                              |
| Jciq | 4         | 4          | 4               | 4          | Did not acknowledge rebuttal |
| cDFT | 4         | 4          | 3               | 3          |                              |
| pPXn | 4         | 3          | 3               | 3          |                              |
Decision: Accept-Main