Memorisation Cartography: Mapping out the Memorisation-Generalisation Continuum in Neural Machine Translation (2023)

Abstract: When training a neural network, it will quickly memorise some source-target mappings from your dataset but never learn some others. Yet, memorisation is not easily expressed as a binary feature that is good or bad: individual datapoints lie on a memorisation-generalisation continuum. What determines a datapoint’s position on that spectrum, and how does that spectrum influence neural models’ performance? We address these two questions for neural machine translation (NMT) models. We use the counterfactual memorisation metric to (1) build a resource that places 5M NMT datapoints on a memorisation-generalisation map, (2) illustrate how the datapoints’ surface-level characteristics and a models’ per-datum training signals are predictive of memorisation in NMT, (3) and describe the influence that subsets of that map have on NMT systems’ performance.

## introduction when training neural networks, we aim for them to learn a generic input-output mapping, that does not overfit too much on the examples in the training set and provides correct outputs for new inputs. in other words, we expect models to generalise without memorising too much. yet, adequately fitting a training dataset that contains natural language data inevitably means that models will have to memorise the idiosyncracies of that data @xcite @xcite . the resulting memorisation patterns are both concerning and elusive: both memorisation and task performance increase with model size figure 1 : illustrative summary of findings for different areas of the memorisation map. counterfactual memorisation subtracts the y-coordinate from the x-coordinate. @xcite ; chatgpt can recall detailed information from its training data, such as named entities from books @xcite ; gpt-2 memorises personally identifiable information, including phone numbers @xcite ; gpt-2 and transformer nmt systems fail to memorise certain idioms @xcite ; and in the presence of some source prefixes, nmt systems have memorised always to emit the same translation @xcite . these examples illustrate the multi-faceted relation between memorisation and generalisation. in machine learning, memorisation has traditionally been associated with overfitting and overparameterisation; @xcite ) already discusses the concern of "fit @xcite the noise in the data by memorizing various peculiarities". yet, for deep learning, overfitting has even been referred to as benign when models overfit the training data but still have a low generalisation error @xcite . besides, memorisation is not just considered detrimental; for instance, it is needed if the natural data distribution is long-tailed @xcite or if factual information needs to be stored @xcite . at the same time, benign overfitting is still concerning if it makes models less robust @xcite and introduces privacy concerns @xcite . in this work, we get a step closer to understanding the relation between memorisation and datapoints' characteristics for neural machine translation (nmt). instead of focusing on cases that are memorised verbatim, we take 5m examples from five language pairs, put them on a memorisationgeneralisation map, learn to predict examples' places on the map and analyse the relation between the map and model performance. the map is centred around the counterfactual memorisation (cm) metric @xcite (see §3), and available to readers via our repository. using the map, we address the following research questions, for which we illustrate takeaways and interesting findings in figure 1: 1. how do characteristics of datapoints relate to their position on the memorisationgeneralisation map? in §4, we compute 28 quantitative features and annotate a data subset manually using 7 additional features. we discuss how source-target similarity, input and output length, token frequency and tokens' segmentation relate to the memorisation map. 2. can we approximate memorisation metrics using datapoints' characteristics? in §5, we use datapoints' characteristics to predict memorisation values using multi-layer perceptrons (mlps) to consolidate findings from §4, to compare different languages to one another and to understand whether resource-intensive memorisation computation has cheaper approximates. we find that the mlps generalise cross-lingually: characteristics' relation to memorisation is largely languageindependent. 3. how does training on examples from different regions of the memorisation map change models' performance? finally, we relate different parts of the map to the quality of nmt systems in terms of bleu, targets' logprobability and hallucination tendency (see §6). our results confirm previous work from other tasks -examples with high cm are most relevant for models' performance -yet there are caveats worth mentioning, in particular for the hallucination tendency. ## related work how has memorisation been measured in nlp, and what have we learnt as a result? in this section, we first dive into memorisation metrics from nlp generically and then report on the limited set of related work that exists for nmt. memorisation metrics in nlp in nlp, memorisation has most often been quantified via binary metrics that identify examples that are memorised verbatim. @xcite measure k-eidetic memorisation (i.e. a string appears at most k times and can be extracted from the model). other studies omit the constraints on k and simply examine whether, after feeding the context, parts of a sentence can be extracted verbatim @xcite @xcite @xcite . sometimes, the training data is known or even modified to include 'canaries' (e.g. @xcite while in other cases, the training data is unknown (e.g. for @xcite . these studies have raised privacy concerns -by pointing out that personally identifiable information and copyright-protected text is memorised -and have identified potential causes of memorisation, such as repetition, large vocabulary sizes, large model sizes and particular fine-tuning techniques. a second approach has been to rate memorisation on a scale @xcite . zhang et al. measure counterfactual memorisation (cm), a metric from computer vision @xcite ) that assigns high values to examples a model can only predict correctly if they are in the training set. zhang et al. identify sources of cm in smaller language models (trained with 2m examples), such as the presence of non-english tokens in english sentences. inspired by the cm metric, zheng and jiang (2022) use self-influence to quantify the change in parameters when down-weighting a training example, to measure memorisation in sentiment analysis, natural language inference and question answering. memorisation metrics in nmt for nmt, memorisation is less well explored, but we still observe a similar divide of metrics. @xcite propose extractive memorisation, a binary metric that identifies source sentences with a prefix for which models generate the same translation, independent of the prefix's ending. @xcite compute cm scores in a low-resource nmt setup to show that hallucinations are more prominent among examples with higher cm values. we, too, treat memorisation as a graded phenomenon by using cm-based metrics. whereas @xcite solely explore cm in the context of hallucinations, we build a multilingual resource of memorisation metrics, examine the characteristics of datapoints that influence their position on the memorisation map, and investigate the relation to models' performance. ## experimental setup this section details the memorisation metrics employed and the experimental setup for the model training that is required to compute those metrics. memorisation metrics to obtain a graded notion of memorisation, we employ the counterfactual memorisation (cm) metric of @xcite and @xcite . 2 assume an example with input x and target y, and a model with the parameters θ tr trained on all training data, and θ tst trained on all examples except (x, y). cm can be computed as follows: as leaving out individual datapoints is too expensive, computationally, we leave out data subsets, similar to @xcite . to then compute the cm of an individual datapoint, one can collect models and average the target probability p θ m n (y|x) over all models θ m @xmath0, where θ tr and θ tst represent sets of models that did and did not train on (x, y), respectively. since we consider the generation of sequences, we also aggregate probabilities over tokens in the target sequence of length ℓ using the geometric mean. we combine averaging over models and averaging over tokens in a likelihood metric (ll), that is computed for θ tr and θ tst to obtain the cm value of an example: in appendix d.3, we replace the probability-based measure with bleu scores for greedily decoded hypotheses and reproduce a subset of the findings with those alternative maps. cm thus consists of two components which we refer to as training memorisation (tm, that expresses how well a model performs on training examples) and the generalisation score (gs, that expresses how well a model performs on unseen examples). cm -the difference between the two -is thus high for examples that can be predicted correctly if they are in the training set but that a model cannot generalise to if they are not. instead of approaching cm as a one-dimensional metric, we examine patterns that underlie all three metrics. data even when leaving out data subsets, computing the memorisation metrics is still resourceintensive. to balance the efficiency of the computation with the quality of the nmt systems, we use corpora with 1m examples for five language pairs: english is the source language, and the target languages are german, dutch, french, spanish, and italian. to enable direct comparison between languages, we collect parallel data by taking sentence pairs from the intersection of the opus corpora for these languages @xcite . using multiple languages aids in assuring that the conclusions are not language-specific. appendix a.1 details how the corpora were filtered and the 1m examples were selected. the resulting data is relatively 'clean'. what happens when using a random opus subset with much more noisy data? appendix d.1 elaborates on this. ## data characterisation: what lies where on the memorisation map? we now have values for our memorisation metrics for 5m source-target pairs across five language pairs. we can view each source-target pair as a coordinate on a map based on the train and test performance associated with that example; the offset of the diagonal indicates the cm. figure 2 illustrates the coordinate system for en-es. it represents datapoints using scattered dots, coloured according to cm. as is to be expected, the tm values exceed the gs values, meaning that generating an input's translation is easiest when that example is in the training set. examples with high cm are rare: few examples are very easily memorised during training while also having a very low generalisation score. our interactive demo can be used to examine individual instances on the map. to better understand which characteristics influence a datapoint's position on this map, we next analyse the correlation between datapoints' features (automatically computed and manually annotated ones) and different regions of this landscape. ## approximating memorisation measures having examined correlations between datapoints' features and memorisation values, we now go one step further and treat this as a regression problem: given the characteristics of a datapoint, can we predict memorisation values? we include the previously mentioned features and additional ones obtained from an nmt system during training. we examine the performance of our feedforward predictors and explore how well the predictors generalise across languages. the analysis aids in consolidating findings from §4 and improves our understanding of how language-independent our findings are. since computing cm is resource-intensive, the predictors can also serve as memorisation approximators (we circle back to this in §6.2). experimental setup to extract training signals, we train one transformer model per language pair on the full dataset, acting as our diagnostic run from which we extract the following signals: 1) the confidence and 2) variability of the target likelihood @xcite , mean and standard deviation computed over epochs), 3) the target likelihood in the final epoch, 4) a forgetting metric (a counter that accumulates drops in target likelihood during training, @xcite , 5) the hypotheses' likelihood in the final epoch, and 6) metric 1 subtracted from 3, since initial experiments suggested those two correlated most strongly with gs and tm. next, we train a shallow mlp to predict the memorisation metrics. we train one mlp on the datapoints' features from §4, and one on the features and the training signals, and report their performance using pearson's correlation and the absolute difference between predictions and memorisation scores. appendix b details the experimental setup of training the mlps. ## conclusion learning the input-output mapping that is represented by nmt training data involves so much more than simply learning a function that translates words from one language into another and rearranges words. it requires understanding which words form a phrase and should be translated together, which words from the source should be ignored, which words can be copied from source to target, and in which contexts "eggs in a basket" are no typical eggs. nmt systems need memorisation of patterns that are out of the ordinary. there are, however, many open questions regarding what memorisation is, when it is desirable and how to measure it. in this work, we took a step toward answering those by creating a map of the memorisation landscape for 5m datapoints. we used graded metrics based on cm to position each example on the memorisation map. we identified salient features for each of the metrics ( §4), illustrated that we can approximate memorisation metrics using surface-level features ( §5) and drew connections between models' performance and regions of the memorisation map ( §6). we found that findings from other tasks and domains about cm transfer to nmt: cm highlights examples that contribute most to models' performance. furthermore, our results illustrate that memorisation is not one-dimensional: cm assigns similar scores to paraphrases and slightly inaccurate translations, examples with high cm scores can be beneficial and introduce vulnerabilities at the same time, and there are nuances about which region of the map is most beneficial depending on the performance metric used. we recommend caution when discussing different phenomena under the umbrella term of 'memorisation'. instead, we encourage future work examining more memorisation maps to further our understanding of the intricacies of task-specific memorisation patterns.