ICML 2025

How Much Can We Forget about Data Contamination?
Sebastian Bordt 1 Suraj Srinivas 2 Valentyn Boreiko 1 Ulrike von Luxburg 1
Abstract
The leakage of benchmark data into the training
data has emerged as a significant challenge for
evaluating the capabilities of large language mod-
els (LLMs). In this work, we challenge the com-
mon assumption that small-scale contamination
renders benchmark evaluations invalid. First, we
experimentally quantify the magnitude of bench-
mark overfitting based on scaling along three di-
mensions: The number of model parameters (up
to 1.6B), the number of times an example is seen
(up to 144), and the number of training tokens (up
to 40B). If model and data follow the Chinchilla
scaling laws, minor contamination indeed leads to
overfitting. At the same time, even 144 times of
contamination can be forgotten if the training data
is scaled beyond five times Chinchilla, a regime
characteristic of many modern LLMs. Continual
pre-training of OLMo-7B corroborates these re-
sults. Next, we study the impact of the weight
decay parameter on example forgetting, showing
that empirical forgetting occurs faster than the cu-
mulative weight decay. This allows us to gauge
the degree of example forgetting in large-scale
training runs, indicating that many LLMs, includ-
ing Llama 3 405B, have forgotten the data seen at
the beginning of training.
1. Introduction
A core principle of machine learning is that a model should
not be trained on the test set used for evaluation (Donoho,
2017). For foundation models trained on Internet-scale
data, there are increasing concerns that this principle is
violated due to the leakage of benchmark evaluation data
into the training data (Xu et al., 2024; Oren et al., 2024).
Indeed, many LLM developers have found overlap between
1University of T¨ubingen, T¨ubingen AI Center, Germany 2Bosch
Research North America & Bosch Center for Artificial Intelligence
(BCAI), Sunnyvale, USA. Correspondence to: Sebastian Bordt
<sebastian.bo...