A Comprehensive Survey of Contamination Detection Methods in Large Language Models
Authors: Mathieu Ravaut
Bosheng Ding
Fangkai Jiao
Hailin Chen
Xingxuan Li
Ruochen Zhao
Chengwei Qin
Caiming Xiong
Shafiq Joty
Venue: Accepted by TMLR ()

Abstract: With the rise of Large Language Models (LLMs) in recent years, abundant new opportunities are emerging, but also new challenges, among which contamination is quickly becoming critical. Business applications and fundraising in Artificial Intelligence (AI) have reached a scale at which a few percentage points gained on popular question-answering benchmarks could translate into dozens of millions of dollars, placing high pressure on model integrity. At the same time, it is becoming harder and harder to keep track of the data that LLMs have seen; if not impossible with closed-source models like GPT-4 and Claude-3 not divulging any information on the training set. As a result, contamination becomes a major issue: LLMs’ performance may not be reliable anymore, as the high performance may be at least partly due to their previous exposure to the data. This limitation jeopardizes real capability improvement in the field of NLP, yet, there remains a lack of methods on how to efficiently detect contamination. In this paper, we survey all recent work on contamination detection with LLMs, analyzing their methodologies and use cases to shed light on the appropriate usage of contamination detection methods. Our work calls the NLP research community’s attention into systematically taking into account contamination bias in LLM evaluation.

Published in Transactions on Machine Learning Research (07 (025))
A Comprehensive Survey of Contamination Detection Meth- ods in Large Language Models
Mathieu Ravaut
mathienj001 &e. itn..uuuuu...
Nanyong Technologionl University Institute of Infocoman Research (12R), ASTAR
Bosheng Ding
bosheng001 Ge.ntu.
ne..ntu.edd.... edu.sy
Nanyang Technological University
Fangkai Jiao
fangkai002Ge. www.:/eeennntteeeeeeeee edu.sg
Nanyang Technological University
Institute of Infocomam Research (12R), A "STAR
Hailin Chen
hailin001 &e. Ge..tu. aabu..dd....
Nanyong Technological University
Xingxuan Li
singzuan001 Be.ntu.
Nanyang Technological University
Ruochen Zhao
SenchennO022..nhu..ee edu.sg
Nanyang Technologicol University
Chengwei Qin
chengwei005@e.om edu..g
Nanyang Technologicol University
Caiming Xiong
caiong@saleeforve.com
Salesforce Research
Shafiq Joty
sjoty@salesforee.com
Nanyang Technological University
Salesforce Research
Reviewed on OpenReview: /ttps://oppereeeet net/ forum? id- SwWWjbtdFn
Abstract
With the rise of Large Language Models (LLMs) in recent years, abundant new opportunities are emerging, but also new challenges, among which contaminatiin is quickly becoming critical. Business applications and fundraising in Artificial Intelligence (AI) have reached scale at which few percentage points gained on popular question-answering benchmarks could translate into dozens of millions of dollars, placing high pressure on model integrity. At the same time, it is becoming harder and harder keep track of the data that. L.LMs have seen; if not impossible with closed-source models like GPT-4 and Claude-3 not divulging any information
the training set. As a result, contamination becomes major issuc: LLMs' performance may not be reliable anymore, the high performance may be at least partly due to their previous exposure to the data. This limitation jeopardizes real capability improvement in the field of NLP, yet, there remains a lack of methods on how to efficiently detect contamination. In this paper, we survey all recent work on contamination detection with LLMs, analyzing their methodologies and use cases to shed light on the appropriate usage of contamination detection methods. Our work calls the NLP research communtty's attention into systematically taking into account contamination bias in LLM evaluation.

---

Published in Transactions on Machine Learning Research (07/2025)
1 Introduction
In the rapidly evolving landscape of artificial intelligence (AI), large language models (LLMs) have emerged as a pivotal tool, driving innovation across a wide spectrum of applications, from natural language processing (NLP) and automated content creation (Achiam et al. 2023 Betker et al., 2023) to complex decision decision-making making systems and autonomous agents (Wei ct al. 2022 Li ct al. 2023a Yang ct al. 2023b Wu et al. 2023 At their core, these models rely on extensive datasets (Raffel et al. 2020 Gao et al. 2020 to learn about language and the world, and generate responses that are increasingly indistinguishable from human-authored writing (Ouyang et al. 2022a). However, the integrity of these datasets is paramount, as any contamination can siguificantly impair the modcls' effectiveness, reliability, and generalization. Contamination refers to leakage of evaluation data within the training data, an undesirable phenomenon which will inflate the model's 's performance on the evaluation data.
We identify several primary sources of such contamination that can arise during largo-scale data collection and curation. First, unfiltered web scraping to collect pre-training data often introduces data with minimal oversight, increasing the likelihood of inadvertently capturing benchmark or test datasets that are publicly available online. Second, contamination may result from the unintentional inclusion of evaluation materi- als during preprocessing or aggregation of training corpora, particularly when datasets are not rigorously audited. Finally, the reuse of proprictary or copyrighted material without robust provenance tracking can also lead inadvertent overlap between training and evaluation sets. These factors underscore the need for meticulous dataset design and transparent documentation practices to ensure the validity of experimental results.
Contamination poses a multifaceted challenge, threatening not only the technical accuracy of LLMs but also their ethical and commercial viability. In high-stakes scenarios where imprecision can have dramatic consequences, such as medical diagnosis, legal advice, or financial services, the repercussions of relying on contaminated data can be profound. Morcover, the allure of leveraging LLM outputs attract investment underscores a pressing commercial dimension. As businesses increasingly integrate AT-driven insights into their strategic planning and operational decisions, the assurance of data purity becomes intertwined with potential market success and valuation. Lastly, in the current intense race to build the most powerful LLM
Touvron et al. 2023b; Team et al.] 2023 Jiang et al.] 2024a Bai et al. 2023 Bi et al.) 2024 Young et al. 2024 Gemma Team Google DeepMind
2024 Chen et al. 2023 the community is struggling to settle on
a fixed subset of benchmarks as LLMs performance increases fast (Clark et al. 2018 Zellers et al., 2019;
Hendryeks et al., 2020 Lin et al.
2021 Sakaguchi et al., 2021 Cobbe et al. 2021). ) an issue which is further
fueled by underlying contamination. This landscape necessitates comprehensive survey of contamination
detection in LLMs.
1.1 Contamination and Contamination Detection Types
Contamination may span only part of a (x, y) data point where x is the input and y the corresponding label. II we consider whether the contaminated data contains only or both and y, contamination can be divided into imput contamination and +label contamination.
Detecting contamination in a LLM is a broad issue that we divide into open-data contamination detection and closed-data contamination detection depending on the nature of the training set. On the one hand, open-data refers to the sccnario where the LLM pro-training data is known, enabling direct comparisons with the evaluation dataset. On the other hand, closed-data contamination refers to the more and more prevalent use case of an unknown pre-training set. This latter case is more challenging and necessitates an examination of the LLM's behavior on evaluation data points.
Detecting contamination is further complicated by varying level of access to the LLM. With while-boz access, the LLM and all weights are available, e.g., a local model. In gray-boz access, model weights are not available to the rescarcher but output token probability distributions arc. Most closed-source sctups, e.g., GPT-4, follow a black- boz setup where only the output text can be accessed from the API. We will review in this study contamination detection methods specifically tailored for each of these levels of model access.
2

---

Published in Transactions on Machine Learning Research (07/2025)
Justacce- String Matchingg
Suring Matching
at 20T Pireende
Lntoves-Lvell Sering Matchiing
Delgras miat
[bing af
Ope--Tara Contamitation
Embeddings Similarity
DetectSon
20233
Paraphrsen Desection
LLM Decontasitanoo
Yoag at przza
Train-Test Performance
Gap
Contamination Detection Methods
Performance Analysis
Controlled Tralning Setop
Datawst-lave
Dang atat
Palasalli
Performance
Change Through Time
Refermoo--nsed
Km of Dio
Closnd-Dataa
Membershipp Inderence Attecks (MLA)
MLA
2001
Contamiration
Detecctiom
Refercuee-fie MLA
1023 CAANA Claas
Guantifying
Mareenn Seecc suzes neuan
Memorization
Instancelevol
Model Memorisation
Engineetin
Promps
local onder
foantns
Cluas Tusks
CirRia LANE pan Chang Siol
Vin KSC Prati
Likelibood
Model Coulidence
Probing
deu Layers
Figure 1: Classification of contamination detection methods reviewed in this paper.
1.2 Contributions and Structure of this Survey
To push forward research in contamination detection, we make the following contributions:
of the first to thoroughly review the field of contamination detection in LLMs, unrestricted to specific type of methods or specific performance aspect. We review all reported contamination detection methods up until early 2025, totaling more than 50 detection techniques and more than 120 relevant papers. We followed an inclusive approached and kept all open-access papers relevant to the problem of contamination detection in LLMs, regardless of publication status. The vast majority of resulting papers which are included in this survey are from 2023-2025.
To our knowlodge,
We categorize contamination detection into two broad types of use cases vastly differing in the techniques involved, namely open-dala and closed--ata contamination detection, and review all existing work each category. Our taxonomy further classifies detection techniques into finer-gran grained categories, each following their own assumptions and model requirements.
Through our detailed classification and review of the types of contamination detection, we not only contribute to the academic and practical understanding of data contamination issues but we also highlight the pressing need for strategies that mitigate these risks.
We refer to Figure for an overview of our proposed classification of contamination detection methods.
The rest of this paper structured as follows. Section formally defines contamination and associated concopts. Section 3 explores methods and findings related to open-data contamination detection. Section 4 addresses the issue of closed-data contamination detection. Section discusses the current and future chal- lenges in the field, including best practices, and reviews all new datasets proposed to detect contamination.
3

---

Published in Transactions on Machine Learning Research (07 (025)
1.3 Comparison to Related Surveys
There already exists surveys the broad issue of contamination.
Deng et al. (2024) survey contamination in language models, covering the problem from detection, to its impact and methods of remodiating it. Compared to their work, our survey solely focuses on detecting contamination, and covers more exhaustive set of contamination detection techniques.
The work of |Palavalli et al. (2024) proposes a taxonomy of the different types of contamination. In Figure our second-level breakdown of closed-data contamination detection into dataset-level methods and instance- level methods echos their categories. Unlike their survey, review contamination detection only, and we do so at large, unrostricted to specific form of contamination such as particular noising then injection of test data points into the training set.
Unlike the two surveys above, Fu et al. (0024b) focus on contamination detection, like our work. They also propose a taxonomy of detection methods, which has a few similaritics with ours (like instance similarity, which we cover within open-data methods), but our classification is more fine-grained (four levels of classifi- cation, whereas their classification only has one). The authors review assumptions behind certain detection methods and challenge their true efficiency. Notably, they show that membership-inferenn attacks (MIAs) perform poorly in detecting pre-training contamination. Our study does not specifically focus on the success of contamination detection methods: we are rather interested in describing in depth the whole landscape of contamination detection, and explain how each method works, including requirements in terms of model access (Table 2) and best suited training stage (Table 3
2 Formal Definitions
Formally, let M be model (e.g. Llama-3 (Dubey et al. 2024 and D be data distribution on input space X. We denote by DM the dataset that M was pre-trained on (e.g, The Pile (Gao et al. 2020)), and Deval an evaluation dataset of interest (e.g, MMLU (Hendrycks et al. 2020))
Definition 1 (Contamination) We say that the dataset Devat is contaminated by the dataset Da if:
3x E Devat 3e' E Daa (( (=== >=
(1)
where f is a similarity function, and similarity threshold. T = 1 corresponds to verbatim or strict contamination, but we do not restrict ourselves to this setup in this paper.
Contamination signs often include inflated performance on evaluation benchmarks, and model memorization of entire substrings. Contamination detection refers to techniques assessing whether Equation (1) holds true or not.
Definition 2 (Membership Inference Attack) membership inference attack (MIA) a func- tion allempting to assess whether a data point was part (a member of) the training set Das and can be noted as:
A:Xx - 1)
(2)
where
(c,) =1 J
A(c, M(x)) =0 otherwise
Both concepts of contamination detection and membership inference are closely related yet fundamentally different: a membership inference attack can be used to detect contamination, but membership inference attacks also be defined and succeed without the existence of contamination.
4

---

Published in Transactions on Machine Learning Research (07 (07/2025)
Definition 3 (Model Memorization) Let us note G(p) the model's natural language output natural language prompl p. We say that M has memorized a data point a E DM if:
Pr[G(p=)= = DM1 Pr[G(p=) = 1 D]
for some prompt Pz that elicits x (typically, Ps is a truncation of x)-
In other words, memorization refers to the LLM's capability to recreate a training data verbatim when prompted accordingly.
3 Open-Data Contamination Detection
In the open-data setup, we have access to the (model-specific) pre-training set DM In this case, contamina- tion detection falls back to investigating the intersection DMs n Deval for any evaluation dataset Deval. In the following, we review cxisting open-data contamination detection methods, which directly compare ensembles of data points.
3.1 String Matching
The most straightforward method to compute overlap between textual datasets is string matching ana- lyzing the set of words intersecting two texts.
3.1.1 Instance-Level String Matching
Several LLM pre-training reports assess contamination levels by using string matching techniques. GPT-2 [Raffrdd et al. 2019) calculates contamination as the percentage of 8-grams from a particular evaluation set that are also found in the WebText training set. An average 3.2% overlap between common LM datasets' test sets and the WebText dataset used for pre-training is discovered, raising the first concerns that LLM performance may be affected by memorization of pre-training data. GPT-3 (Brown al. 2020) scans data points having 13-gram collision with anything in the pre-training Common Crawl (C4) dataset. (Raffel ct al. 2020). Some carlier benchmarks sourced from Wikipedia, the Children's Book Test dataset (Hill et al. 2015 Quac (Choi et al. 20188) and SQuAD 2.0 (Rajpurkar el. al. 2018). are nearly entirely con- taminated. PaLM (Chowdhery et al. 2023) identifies 10 evaluation datasets at risk of contamination per their construction process, and for each datasct, partitions into a clean and a contaminated subset basod on whether at least 70% of the 8-grams of the data point can be found at least once in the training set. Both GPT-3 and PaTM authors conclude that the performance gap between clean sets and contaminated mostly small across of various contamination degrees. GPT-4 (Achiam et al., 2023) measures cross-contamination between evaluation sets and pre-training data by computing 50-characters substring collisions. After examining GPT-4 performance gap between contaminated and clean subsets, the authors conclude that contamination overall has very little effect on the reported zero-shot results. Llama-2 (Tou- vron et al. 2023b) refines the methods above by using tokens instead of words to measure contamination, and asserts that a token is contaminated the evaluation sample and the training set. The formatting of prompts used for actual evaluation also considered as it affects the performance more significantly. From there, the contamination level of sample is defined as its percentage of contaminated tokens. The authors also introduce the usage of skip-gram budget allowing matched spans between the evaluation sample and the training set to differ in at most four positions. Llama-3 (Dubey et al. 2024) follows similar technique to Llama-2, but using 8-gram tokens instead of 10-gram. The ratio of contaminated tokens is varied to find the value with highest performance gain across models. Authors conclude that performance on PiQA (Bisk et al. 2020) and HellaSwag (Zellers) ct al. 2019) is affected by contamination. Recently, Swen-2.5-Codee (Hui al. 2024 removes all training data points with 10-gram collision with the test set. Popular benchmarks HumanEval
appears in any token n-gram longer than 10 tokens in both
Chen al.) 2021a)
MBPP (Austin et al. 2021 GSM8K (Cobbe al. 2021 and MATH (Hendrycks et. al. 2021 affected.
5

---

Published in Transactions on Machine Learning Research (07/2025)
String Matching
Contamirated
GPT-3
CuQA
Robyy
LAMBADA
Panssnes al 2200
15-gmmm
wt..
WitlTest-2
Menity
Walll
PoLM
s-aam wonds
Witograd
SQuAD Balporkar at
WSC, BeCoRD,
50gnuu clarasters
LorYen of
Wegmam tohvms
HollaSearr
1Jmaa33
Segram volons
PIQA
Zhump at
BiG-Beech Hunl
Sriessaan
BooQ
Oprollokys
2015
Qua LL OF
SIQA
500 ct 200
Qwnn-2.-Collr Bui ct d. 2021
10gna words
3021a
MBPP
202
CSMSK
Coliee 4t
204
MATH
2021
Table 1: Datasots flagged with a high-level of contamination for several LLM pro-training studies, assessing contam- ination through string matching between the pre-training corpus and the evaluation samples.
Table 1 summarizes contaminated datasets found through all string-matching contamination detection de- scribed above. Notably, PIQA (Bisk et. al. 2020 Winograd (Levesque et. al. 2012). HumanEval (Chen et al. 2021a) and HellaSwag (Zellers et al. 2019 are flagged by at least two contamination detection techniques.
Singh et al. (2024) review several string- matching contamination detection techniques, including the ones used by Brown et al. (2020) (Token-Match), Chowdhery et al. (2023) (Ngram-Match) and [Tovvron et al. (2023b) "Token-Extend). They introduce new method called Longest-
which measures the fraction of tokens part of the longest contaminated token span, avoiding the issue of frequent token spans artificially inflating contamination levels with other methods. Similarly to (Touvron et al. 2023b), Longest-Match also a allows for skipgram. Longest-Match finds the highest. Estimated Performance Gain (EPG) between contaminated and clean evaluation subsets. Through Llama-1 (Touvron et al. 2023a) and Pythia (Biderman et al., 2023) series of LLMs, the Llama pre-training corpus and The Pile (Gao et al.
2020) are found to be contaminated
for many popular evaluation benchmarks, notably BigBench (Srivastava et al. 2022). HellaSwag (Zellers let al. 2019 PiQA (Bisk al.) 2020 MMLU (Henndyycks et al. 2020) and TriviaQA (Joshi et al., 2017).
3.1.2
Dataset-Level String Matching
Other studics analyze overlap between the training split and the evaluation or test split of the same datasct to assess contamination levels. While standard practice in machine learning assumes that training, validation and test splits of the same dataset are disjoint sets sampled from the same distribution, in practice, for very large datasets it may happen that the training set still overlaps with the other splits. Dodge et al. (2021) investigate the C4 (Raffel et al. 2020) dataset and study benchmark data contamination, which measures how much training or test datasets from downstream NL.P tasks appear in the C4 pre-training corpus. After studying both input-label and input contamination, they discover varied contamination, ranging from less than 2% over 50%. Li (2023b) compute the METEOR (Banerjee & Lavie 2005)- score between matched pages from CommonCraw! and queries from Bing API and consider those with scores over 0.75 as contaminated. Detected contamination levels range from 1% on Winogrande (Sakaguchi et al. 2021 to 47% C-Eval (Huang et al.) 2023). Deng et al. (2023) propose to detect contamination by retrieving top-10 documents from pre-training datasets The Pile (Gao et al.] (020) or C4 (Raffel et al.) 22020). then splitting retrieved documents into 13-gram chunks, and computing overlap metrics between these chunks and the evaluation data point's chunks. TruthfulQA [Ln et al. 2021 cxhibits high overlap with the pro-training datasets.
3.2 Embeddings Similarity
Boyond simple string matching, computing cosine similarity between embeddings offers an attractive alter- native, as this process is more robust to surface-form vocabulary changes like paraphrases.
Lee et al. (2023 prevent contamination in their Open-Platyy dataset by removing test questions which have cosine similarity (using SentenceTransforme
embeddings (Reimers & Gurevych 2019)) greater than 80% against any training item. Phi-1 (Gunasekar et al. 2023) runs a data contamination analysis between their
CodeExercises dataset and the evaluation set HumanE Eval (Chen et al. 2021a). They show that embeddings-
basod retricval between code snippots using I.2 distance between normalized CodeGen-Mono 350M (Nijkamp) et al. 2022) embeddings is effective, whereas n- n-gram overlap fails in the coding domain as it cannot capture the similarities in the logic between two coding functions.
6

---

Published in Transactions on Machine Learning Research (07/2025)
The work from Riddell et al. (2244) provides hybrid approach combining string matching and embeddings similarity: an aggregated score is designed as the maximum between the Levenshtein edit distance and the similarity from the Dolos toolkit (Maertens et al. 2022 Widely used code generation benchmarks MBPP 2021a are contaminated with pre-training sets The Pile (Austin et al., 2021) and HumanEval (Chen et al.
(Gao et al. 2020) and especially The Stack kK el al. 2022). which significantly inflates performance of models like CodeGen-NT. NNjjaamp al., 2022).
3.3 Paraphrase Detection
Embeddiings similarity is limited by its need to choose appropriate similarity threshold. To improve robustness, some methods attempt to use LLMs themselves to detect contamination in LLMs, leveraging their high performance in zero-shot paraphrase detection (Witteveen & Andrews 2019 Abaskohi et al. 2023
Vang et al. 2023cp argue that contamination checks become challenging because of the presence of "rephrased samples", which have the same semantics but different surface form the original sample. Therefore, they first use embeddings similarity scarch to get the top-k similar samples with a given test sample, then prompt strong LLM (namely, GPT-4) to examine whether any of the top-k samples is too close to the test case. Results show that this LLM Decontaminator algorithm works significantly better than existing methods at tagging contaminated samples.
4 Closed-Data Contamination Detection
In closed-data contamination detection, the pre-training set D is unknown. Since Das not accessible, researchers have to use tools analyzing the model's behavior the evaluation dataset Deval-
There two broad ways to assess closed-data contamination: examining model behavior aggregated on the whole dataset, and focusing on instance-level outputs. For the former, the main method, which we explore in Section 4.1 consists in tracking model performance on several datasets, sampled from different distributions, notably through time. For the latter, we review methods ranging from the strongest assessment of contamination to the weakest: membership-iiffernnce attacks, which assess the presence of an entire data point the pro-training set (Section 4.2) then memorization of spans of text by the model (Section 4.3) and lastly methods analyzing model confidence on specific lokens (Section 4.4
4.1 Performance Analysis
We first review line of research assessing model contamination through simple performance analysis. M is applied to different evaluation datasets, and patterns in evaluation scores may indicate contamination.
4.1.1 Test Performance Gap
On a dataset where the training set and the test set are sampled from the same distribution, a machine learning system is expected to perform on the test set slightly worse, or at best similar to, the training set. Zhao et al. (2024) leverage this fact distinguish between standard fine-tuning and contamination, focusing domain-speeific setups such financial data. They introduce Performance Consistency Ratio (PCR), equal to the LLM performance
the test set divided by its consistency, where the latter measures the LLM's ability to produce the same output across several reasoning paths. A PCR on the training set significantly greater than on the test set is typical of aue-tuning: the opposite case points out at contamination. Findings
include potential contamination of Baichuan- (Yang et al. 2023a on FinEval (Zhang et al. 2023 and
of the FinMA model (Xie el al. 2023 on the FinQA dalasel (Chen el. al. 2021b)
4.1.2 Controlled Training Setup
Jiang et al. (2024b) delve deep into the impact of evaluation data leakage within the pre-training set by running series of controlled pre-training experiments with GPT-2 (Radford et. al.] 2019). They pre-train
7

---

Published in Transactions on Machine Learning Research (07 (07/2025)
from scratch GPT-2 in three different ways: with the original pre-training set, with additional input texts from four ion datasets (SST-2, MMLU (Hendrycks et al. 2020). CNN/DM (Hermann et al. 2015) and SQuAD-v1 Rajpurrar et al.) 2016)), and with input texts and labels from these same evaluation datasets. Contaminated pre-training clearly boosts performance on all downstream tasks. This was also shown by the work of Palavalli el. al. (2024) their Verbatim selup when pre-training GPT-2. Findings also reveal a U-shaped relationship between the number of times that contaminated data points are included in the pre-training, and model performance. Authors also show that the n-gram contamination detection method from Llama-2 Touvron et al. (2023b) is not effective at detecting contaminated data points, calling for more robust methods.
4.1.3 Performance Change Through Time
The most straightforward performance analysis consists in plotting performance against time, which lever- ages the fact that evaluation datasets released after LLM is pre-trained are new and should not be con- taminated.
Aiyappa et al. (023) are among the first to question the non contamination of ChatGPT training data by popular evaluation benchmarks. Their run of the January 30th, 2023 version of ChatGPT on the SemEval 2016 dataset shows a 12.5 macro-F1 score improvement compared to a prior experiment with the vl of ChatGPT, released two months prior in November 2022. In the code domain, Roberts et al. (2023) investigate the performance of GPT-3.5 and GPT-4 on datasets released before and after the training cutoff for both models. The analysis underscores statistically significant positive correlation for problems catalogued GilHub prior to the GPT models' training cut-off, suggesting that their inclusion in the training data enhanced their performance on related tasks. Concurrent to their work, Li & Flanigan (2023) also plot the performance of several LLMs including GPT-3 (Brown et al.) 2020) and open-source LLMs on general natural language understanding lasks against time, specifically belore and after each LLM's training data cutoff date. They find that LLMs are much more likely to improve on a trivial majority baseline for datasets released before their cutoff date, suggesting some contamination. While evaluating on fresh data guarantees contamination-free performance evaluation, it is hard to scale this method in time it requires frequent update of evaluation sets. Besides, the work of Cao et al. (2024) shows that evaluating on data posterior to the LLM training cutoff time may not always show a decrease in performance: in the code domain, the opposite may happen, complicating the usage of training cutoffs as a contamination detection method.
4.2 Membership-Inference Attacks
Extracting knowledge from language model has traditionally been addressed in the field of privacy- preserving machine learning. Adversarial attacks were crafted as membership inference attacks (MIA) to access training data points memorized by a black-box language model, potentially compromising sensitive privale information (Carlini et al.] 2021 2022). If there access to data points which are known not to be present in the training set, membership inference attach is referred to as reference-based MIA ; other cases belong to the sctup of referenco-free MIA.
4.2.1 deference-based Membership-Ilnfrrnce Attacks
Xie al. 2024) derive a MIA named RECALL based on the conditional language capacity of LLMs. RECALL conditions with non-member prefixes (for instance, sampled after pre-training cutoff date): log- likelihood drops more average for member data points than for non-members, yielding simple yet powerful MIA. The method reaches state-of-the-art ROC-AUC on the WikiMIA dataset (Shi et al. 2024). Meeus et al. (2024a) revisit MIA in the era of LLMs and define document-level MIA, which is the task of detecting whether an entire document (book, scientific paper, elc) has been trained on. A reference set is built by sampling documents known to be part of the pre-training set (e.g. Common Crawl), and data posterior to the pre-training cutoff date, which guaranteed to be uncontaminated. Then, the authors derive features from predicted token probabilities by the LLM, which are normalized in several fashions. A meta-classifier predicts membership inference, reaching ROC-AUC as high as 0.86.
8

---

Published in Transactions on Machine Learning Research (07/2025)
DPO (Rafailov et al. 2023) is usually employed to train LLM on preference data. Feng et al. [2024] show that LLMS are vulnerable to MIA on the preference data, especially when using DPO. LLMs themselves can be instruction-tuned to detect if a data point is a member of their pre-training data. MIA-Tuner (Fu et al. 2024a) fine-tunes a soft prompt (Lester) et al. 2021 on several larget open-source LLMs to classify membership inference, and outperfomm all other methods on WikiMTA. Even though at inference time, non-members are not needed, the methodology should still be assessed as a reference-based MIA due to the usage of non-members during the LLM fine-tuning.
In the last stage of LLM post-training, PPO [Shullann et al.
2017
4.2.2 Reference-free Membership-Inferee Attacks
Fu et al. 2023 argue that both reference-based and reference-fres MIA are unsatisfying with LLMs in practical scenarios. The latter rely on overfitting to assess membership inference. In the referenco-free setup, they propose to prompt the target LLM to generate pseudo reference data. Semantically similar and semantically different paraphrases of target records are generated, and membership signal is designed by measuring and comparing probabilistic variations between the target and reference models. This method reaches a ROC-AUC of 0.95 with Llama on the language modelling dataset WikiText-100 Chang et al.) (2024) analyze the dynamics of next-token prediction loss for open-source Pythia and GPT-Neo LLM series. They observe that for members, the loss tends to be smoother and to decrease as context increases, whereas non-members loss curves show a flat slope as well as quite frequent outlier values. From this observation the authors derive state-of-the-art MIA technique called CAMIA. EM-MIA (Kim et al.) 2024) introduces MIA method iteratively refining the membership score used to assess MIA and the prefix score through expectatoon-maxiiiiztion algorithm. The prefix score mcasures how discriminative the data point is in classilying members and non-members when used a prefix. EM-MIA reaches state-of-the-arl ROC-AUC scores on WikiMIA, however on a custom benchmark where members and non-members distributions are almost identical, performance drops back to random range, as for all other reported MIA methods.
4.2.3 Limitations of Membership-IIferennce Attacks
Gembershhip-nfeereec attacks are hard in practice with neural networks Duan et al. (2024) evaluate scries of MIAs on the Pythia LLMs (from 70M to 12B paramcters), pre-trained on the Pilc. Across types of attacks and domains, performance hardly surpasses random guessing (ROC-AUC inferior to 0.6). This failure is attributed to the common practice in LLM development of pre-training only for a single epoch and on a too large dataset, and the fact that the border between members and non-memberss is not clear. By splitting both groups based on time, where non-members are drawn from datasets posterior to the pre- training cutoff date, MTA accuracy drastically improves. This finding motivates the monitoring of LLMs performance through time described above. Meeus et al. (2024b) echo these findings from Duan et al. (2024) they train a trivial bag-of-words model on MIA datascts constructed by splitting on time and reach a high ROC-AUC. Authors argue that proper MIA evaluation should be done on randomized train-test split. Concurrent work from Maini et al. (025)) also question the validity of membership inference attacks on a time-split of Wikipedia. An experiment replacing member sentences by sentences from the validation set of The Pile (Gao al.] 2020) using Pythia models (Biderman et al.] 2023) shows membership inference tests still reaching an ROC-AUC of 0.7 proving that they merely capture temporal distribution shift rather than actual membership to the pre-training set. Despite this lack of success of MIA pro-training sets of LLMS, Li el al. (2024b) show that similarity between the larget sample and the generated output of a LLM- powered RAG system yields a powerful MIA on the RAG database. On Natural Questions (Kwiatkowski ct al. 2019) and TriviaQA (Joshi et al. 2017), the method reaches more than 0.85 in both ROC-AUC and PR-AUC.
4.3
Model Memorization
An intuitive approach to detect contamination in LLMs is to frame it as a memorization detection problem. While MIAs assess the presence of an entire data point into the training set, here memorization is more gran- ular and detects memorized sequences of tokens which may not be full data points (documents), but rather subsets of them. The rate of memorized data points can be used to assess the presence of contamination. For this endeavor, it is important to first be able to quantify memorization itself.
9

---

Published in Transactions on Machine Learning Research (07/ (07/2025)
4.3.1 Quantifying Memorization
Elangovan et al. (2021) highlight that some LLMs may benefit from an overlap between training and test sets, and merely memorize data points in this intersection, inflating their test set performance. This overlap is due to the common practice of simple random data shufiling in NLP tasks. By employing a bag-of- wordds approach to assess text similarity, the study provides foundational method for identifying and mitigating leakage, though it acknowledges the complexity of semantic similarity measurement and the potential for more sophisticated methodologies to refine overlap detection. Magar & Schwartz (2022) find evidence of
contamination across several benchmarks, impacting the performance of models like GPT-3. They propose to quantify the effect of contamination by training on a mix of general and task-specific data, then comparing performance on seen versus unseen instances to measure memorization and exploitation. This methodology highlights the nuanced relationship between memorization and exploitation. [Carlini et al. (2023) propose simple method to detect memorization by measuring the completion rates of randomly sampled prefixes from the training set, and discuss the relations between memorization with model capacity, data repetition, context size. Memorization is more widespread than previously thought, and model size, data frequency and prompt length are factors leading to an increase in memorization.
The team from Google developing the Gemma model series (Gemma Team Google DeepMind 2024 empha- sizes the potential vulnerabilities of aligned models to adversarial attacks inducing rocitation of memorizod training data. Through comparable methods to Anil et al. (2023), they analyze discoverable memorization across 10k documents from the Gemma pre-trained models' corpora. The analysis distinguishes between exact memorization - where the model's output precisely matches the source and approzimate memo- rization, gauged by a 10% edit distance threshold. Gemma models exhibit memorization rates comparable to those of the PaLM and PaLM-2 models.
As we have seen in Definition 3 detecting memorization sums to finding a (prefix, suffix) mapping such that the corresponding suffix can be perfectly completed when conditioning the model on the prefix. For example, Ozdayi et al. (2023) append learnable soft prompt penalized through gradient ascent to prevent memorizing the original suffix. Wang al. 2024) employs transformer-based generator to approximate such mapping of prefixes, and use it to estimate the memorization rate unseen corpus.
4.3.2 Prompt Engineering
Recently, researchers have engineered elaborated prompting techniques that elicit data point completion by the LLM, where an output suspiciously close to the actual training data point indicates memorization, which in turn means contamination. Standard prompting asking the LLM to complete dała point may fail due to the LLM's alignment during RLHF (Ouyang et al. 2022b
By prompting with an off- (training)-distribttie input, Nast et al. 2023) show that ChatGPT can regurgitate entire chunks of training data, including sensitive personal-level information. The study illustraats that even models trained with alignment techniques, aimed at reducing the emission of memorized data, can be induced to divulge substantial amounts of sensitive information, calling for the development of more robust defenses. The implications of this research are far-reaching, emphasizing the importance of addressing the privacy vulnerabilities inherent in the deployment of LLMs. WWller et al. (2023) highlight the potential for steering LLMs towards generating more factual and grounded content through effective prompting strategies. By appending instructional phrases such as Accoding to that encourage quoting from specific corpora, this study observes improvements grounding as measured by the Quoted Information Precision (QUTP) Score. This mothod shows promise across various domains and corpora, indicating its versatility and effectiveness in leveraging LLMs' memorized knowledge for generating more accurate and reliable responses.
Golchin & Surdeanu (023b) make use of GPT-4's abilities in guided prompting, an enhanced prompting process where the completion prompt includes extra information such as the dataset name. Contamination is Ihen assessed based on the averaged difference in performance between standard and guided prompting, or if GPT-4 with in-context learning finds an exact match or two near-exact matches in the guided comple- tions. This latter method with GPT-4 is highly accurate identifying contamination (92%-100% accuracy). Morcover, the investigation highlights the prevalence of contamination in datasets such as AG News (Zhang et al. 2015), WNLI (eeessque et al. 2012). and XSum (Narayan et al. 2018), emphasizing the critical need
10

---

Published in Transactions on Machine Learning Research (07 (07/2025)
to address data integrity in LLM applications. The same authors follow up with the Data Contamination Quiz (DCQ) evaluation framework for contamination detection in black-box L.MSs (Golchin & Surdeanu, 2023a). LLMs are prompted with five completion options, where one option is the ground-truth text from the original dataset, three other options are paraphrases from GPT-4, and the last choice Nonc. The assumption is that. the LLM picks the exact answer, it doing so out of memorization, and the authors show that this DCQ framework finds more contamination cases than the guided prompting method. [Samul] let al. (2024) also propose a quiz framework to detect contamination. Their Local Order Quiz method consists in prompting the LLM with data point and asking it to select the subsequent data point in the original data set, among four possibilities shown in the prompt (the correct next data point, and three other random ones). A higher accuracy that the baseline 25% chance indicates that the LLM has memorized the data set. In practice, the IMDB (Maas et al. 2011) aset shows high signs of contamination (subsequent data point selection accuracy over 80%) for all three LLMs under study: GPT-4 (Achiam et al. 2023), Claude-3 and Llama-3 (Dbbey et al. 2024
4.3.3 Cloze Tasks
Cloze tasks constitute a special type of prompting where parts of the input masked. Chang et al. (2023) design cloze task in which the LLM is prompted with a book's passage, from which the character name has been replaced by [MASK] token. Correctly predicting the character's 's name out of several passages per book assessos whether the LLM has seen this book during pre-training. The name cloze accuracy is very high for GPT-4 (Achiam et al.) 2023) on several popular books such as Alice's 's Adventures in Wonderland, and shows a striking difference with BERT's (Devlin et al. 2019) performance on the same task. Deng et al. (2023) present the TS- TS-Guessing method: one of the options question-answering benchmarks test sets hidden, and the LLM is askod prodict it. Exact match rates above 50% for OpenAI models suggest contamination the MMLU dataset (Hendryeks et al.) 2020). The work of Ranaldi al. (2024) also uses cloze task: they prompt the LLM to reconstruct columns names in SQL dumps, and compare the performance
older and more recent datasct to assess contamination. GPT-3.5 accuracy falls from 33.42% to 13.2 13.21%, indicating contamination on the older Spider dataset Yu et al. 2018
4.4 Model Confidence
Going beyond simple recitation of memorized token sequences, researchers may now analyze the LLM's confidence to detect contamination: (too) high confidence some tokens may signify prior exposure to the data. Confidence analysis yields several powerful contamination detection techniques, which we highlight require gray-box to white-box access lo the LLM, as researchers need to obtain the output. probability distribution for each token during inference.
4.4.1 Likelihood
Next-token likelihood and perplexity (by extension to a whole dataset) widely used in contamination detection.
Given a test tokens sequence X, the Min-K% Prob technique (Shi et al. 2024 consists in rumning the LLM through all tokens of X, and keeping track of the K% tokens with the smallest predicted probability. Then, the average between such bottom probabilities is computed and X is deemed contaminated this average is too high. In mathematical terms, noting (z)...., 2.) the sequence of evaluation tokens under study, Min-K% Prob is computed as
Min-K% Prob(r)= = [Min-K%(>)
Σ
log(p(15(122))
(4)
A significant drawback is the choice of value for the hyper-parameter K, which authors recommend to set to 20 as default. On WikiMIA, Min Prob reaches an average ROC-AUC of 0.72, and is the highest for
http:://ww..aatrrrt....caeeeesss f-mmll
11

---

Published in Transactions on Machine Learning Research (07/2025)
every LLM tested. Notably, it outperforms other methods such as perplexity or the zlib compression entropy membership-infereene attack. Due to its simplicity and effectiveness, Min-K% Prob is gaining traction as one of the most popular contamination detection methods. Zhhu et al. (2024) use Min-K% Prob as a first contami- nation detection step to identify lcaked test samples before re-writing them, in order to build clcaner versions of popular evaluation datasets GSM8K (Cobbe et al. 2021 and MMLUU (Hendrycks et al. 2020). Min-K% Prob performance very high: yet the method has a major limitation: it does not account for the token frequency distribution. LLMs put a higher probability on more frequent tokens, thus a non-contaminated se- quence made of only frequent tokens might get assigned a score higher than sequence from the training set. To fix this, DC-PDD (Zhang et al. 2024b) calibrates token probabilities and computes the cross-entropy between the token probability distribution and the token frequency distribution. The latter is estimated through a large-scale publicly available reference corpus. The tokens sequence is scored by averaging these token-level cross-entropy scores over the first occurence of each token. Contamination is assessed when this final score is above a pre-defined threshold. Min-K%++ (Zhang et al. (2024)) proposes another extension of the idea of Min-K% Prob: rather than simply getting next-token predicted probabilities, the authors propose score which subtracts the expected log ogg-robability and divides the score by the variance; in other words, normalizing the initial log-probability. The sequence scoring mechanism is then identical to the one of Min- K% Prob. Formally, noting the expected probability over the next token Marei = 12- [log(p(=( and its corresponding standard deviation azei = ( - )=
1
log(p(x;1(<e)) - Musi
Min-K%++(1)- [Min-K%(* K%6(2)
Ozei
The motivation of the score built by Min-K%++ to assess whether an input form a mode, after observing that because of maximum likelihood training, training samples often become local maxima in the modeled distribution along each input dimension (here in the context of LLM, dimensions are tokens). Min-K%++ reaches state-of-tho-art on WikiMIA, outperforming Min-K% Prob by up to 10 points.
Li 2023a also work on token-level probabilities and compare perplexity on benchmark samples against memorized and clean baselines. The study finds significant memorization in recent models on popular reading comprehension and summarization benchmarks, while multiple-choice benchmarks show less evidence of contamination. This method provides a tool for the community to conduct rigorous contamination analysis, enabling more accurate and reliable model evaluation. Dong et al. (2024) propose two novel likelihood- based contamination detection methodologies: CDD (Contamination Detection via output Distribution) and TED (Trustworthy Evaluation via output Distribution). CDD detects data contamination by observing the peakedness in the LLM's output distribution in a black-box manner. It represents a significant improvement over cxisting approaches, offering average relative improvements of 21.8%-30.2%, in terms of Accuracy F1 Score, and AUC metrics. TED corrects the LLM's output distribution to mitigate the impact of data contamination on evaluation metrics, significantly reducing performance improvements attributed to data contamination across various scenarios and contamination degrees. The work of Xu et al. (2024b) proposes to use perplexity and accuracy next n-gram prediction. They compute the relative loss in performance on these metrics on paraphrased versions of the dataset created by ChatGPT compared to the original dataset. The difference of relative loss between training and test set is later used to assess contamination or benchmark leakage. Experiments on GSMSK (Cobbe et al.] 2021) hint at contamination for Qwen (Bai et al. 2023), Aquila and InternLM (Cai et al., 2024) series of L.MSs. Zhhng Wu (2024) leverage the concept of surprising tokens: these are tokens where the LLM is very confident yet wrong in its prediction of the next token. The authors define a score named SURP which is the average predicted log-probability the ground-truth token for tokens of the sequence where the model shows low entropy (and therefore, high confidence) and where this log- log-probability is relatively smaller. A higher SURP score is thresholded to assess contamination. The method slightly outperforms Min-K% Prob on WikiMIA.
Oren et al. 2023) take an original approach to analyze the LLM's likelihood on an evaluation dataset DE- They run inference on Dg and on shuffled versions of Dg- LLM log probabílitics being statistically different on the non-shuffted dataset compared to the shuffled versions signifies contamination. The method relies on the assumption that if they are included in the pre-training set, evaluation datasets tend to be present with
12

---

Published in Transactions on Machine Learning Research (07 (07/2025)
Closed-Data Method Type
Black-box
Gray-box
White-box
CAP Zhao
Alyappa ot
Performance Analysi
Roberts mu
3123
2123
Ce et m
2024
RECALL
MaxNorm
Meets al.
2024n
PREMIA
Feng et al.
21/44
Membenship Artacles
MIA-Taner SPV-MIA
2024
Al
T22
CAMIA Chunn
2024
82-MIA La ot al.
2024h
EM-MIA
Kim al.
2024
Mnnga & Schwartz
2022
Carlini el ml
2023
Team Google DeepMlin
2021
Dynamic Soft Prompting Wang et al.
2024
2023
Model Memorization
Aocordung prompping Weller al.
2025
DCQ Sanmel ot nl. 2024
Local
Chang el al 2023
Giesssinng Nong
1023
Rauaidi 2021
Min-K% Prob
Shi et al.
2024
DC-PDD
Zbabz a.
2216
Min Mine-K%r
Zhang et
2024n
20/3k
Model Confidence
MD
TED
Dong et
2024
X1 ct al.
20144b
Zhhang
2024
Orea et al. 2023
DICE Th al. 2024
Liu et 5t 2023
Table 2: Classification of closed-data contamination detection methods as per the level of access to the LLM required. Black-box methods simply need to prompt the model, while gray-box methods require access to the output tokens distributions and white-box techniques need full access to model weights.
the same default ordering. We point out that detecting the LLM's memory of a dataset canonical order is an idea also explored by Samuel et al. (2024) for a different contamination detection method.
4.4.2 Probing Hidden Layers
Through the use case of fine-tuning LLMs on mathernatical reasoning datasets like GSMSK (Cobbe et al. 2021), Tu et al.] (2024) introduce new paradigm in (closed-data) contamination detection. Their DICE approach consists in finding the layer of the fine-tuned LLM furthest away from its counterpart layer in the not fine-tuned same LLM. Then, MLP binary classifier is inferred the isolated layer's weights to predict contamination. This method is particularly effective for in-distribution contamination detection, setup defined by the authors as fine-tuning on a dataset from the same distribution as the evaluation dataset. In this setup, DICE reaches near-perfect ROC-AUC, outperforming all other methods, including notably Min-K% Prob.
Liu et al. (2024) also make use of the intermediate layers of a fine-tuned LLM. Their method first gathers dataset of non-members and members where the former pulled from after the pre-training cutoff synthetically generated from ChatGPT. Then, the LLM is fine-tuned to classify membership between these two groups. Activations of intermediate layers of the finc-tuned LLM on both members and non-members are used as input of a probe classifier assessing membership. At inference time, the whole system feeds a target text to the fine-tuned LLM, extract its activations and feeds them to the probe classifier to check membership. The best layer to extract activation from determined on validation set. Results show greater ROC-AUC on WikiMIA (Shi al.] (024) and ArxivMIA (Liu et al. (224).. and indicate that this probing system works better with larger models.
13

---

Published in Transactions on Machine Learning Research (07 (07/2025)
Closed-Data Method Type Contamination Detection Method
Pre-training Instruction-tuning RLHF
CAP Zhao al.
2024
Aiyappa et al.
2023
Performance Analysis
Roberts et 2023
Lt & F Clanigan
2023
Cao et al.
2024
RECALL
Xie el al.
2024
MaxNorm
Meeus et al.
2024a
PREMIA
Fong
2024
MIA-Tumer
Membership Inference Attacks
SPV-MIA
[u al.
2023
CAMIA (C Chang ct al
EM-MIA [Kim al.
2024
S2-MTA
ml.
2024h
Magar & Schwartz (2022
Carlini et sl.
2023
Cemma Team Google DecpMind 2024
Dynamic Soft Prompting Wang et 2024
Nast et al. (2023
Model Memorisation
According To prompting WWeller et al.
2023
DCQ (Sanuel al. 2024
Local order quiz (Sameel et al. 2024
Chang et al.
2023
TS-Cuessinn Deng et 2024
2023
Ranoldi et al.
Min-K% Prub
SSh et al.
2024
DC-PDD
Zhaag al.
224b
Min-K%
Ziang
2024a
Tu 2023n
CDD
TED
Dong al.
2024
Model Confidence
et al.
20044
SURP
Zhang
Wn
2024
Oren cb a
2023
DICE Tu
ct al.
2024
Tlu et al.
2024
Table 3:
Classification of closed-data conlamination detection techniques according to the model training stage(s) which they are more suitable to be applied to.
4.5 Practical Usage of Closed-Date Contamination Detection Methods
In the previous subsections, we reviewed the different types contamination detection methods. We now wish to give more concrete intuition how these methods can be applied in practice.
Contamination detection techniques strongly differ in terms of model access requirement. Table summarizes closed-data detection techniques organized by the minimum level of access to the LLM which is required. We define three levels of model access: black-bor, which merely needs prompting access: gray-boz, which needs access to output tokens distributions and white-boz which needs to see model weights. All methods based on performance analysis only require a black-box access, making them convenient to use. Membership- inference attacks are equally split. between black-box and gray-box access requirements. Model memorization techniques mainly rely on prompting and thus only need black-box access. Lastly, for model confidence methods, which often analyze output. distributions, gray-box or even white-box model access is needed.
We also highlight that closed-data contamination detection techniques may naturally be applied to different model training stages. In Table 3. we indicate the most suitable training stage(s) where each method can be applied, among (i) pre-training, (ii) instruction-tuning and (iii) reinforcement learning from human feedback (RLHF). Prompt tuning is naturally tailored for RLHF models which can prompted with any user query. Cloze tasks and likelihood-basse
methods are suitable for pre-trained models, which have captured tokens distributions. We emphasize that this categorization flexible and not strictly defined: for instance, methods most suited for pre-trained models, like Min-K% Prob, in theory be applied instruction-tumed
14

---

Published in Transactions on Machine Learning Research (07/2025)
RLHF models. However, results should be cantiously interpreted RLHF often smoothens overconfident predictions to align with human preferences, weakening the contamination signal.
5 Discussion
5.1 Best Practices to Avoid Contamination
Beyond detecting contaminated evaluation data points, recent works have called the community adopt better practices to reduce contamination.
Scanning newly released evaluation datasets Sainz et al. 2023 argue to develop automatic or semi- automatic approaches to detect contamination for new benchmarks and design mechanisms to flag those works with contamination. This process goes hand in hand with creating new, non-contaminated evaluation datasets.
Evaluating on wide spectrum To comprchensively and fairly evaluate LLMs, Zhou al. 2023 sug- gest to use more benchmarks from diverse sources, covering both basic and advanced capabilities. Further- more, they recommend employing multiple task prompts for benchmark tests to derive model performance that is more stable and reliable. Additionally, they highlight the necessity of conducting data decontamina- tion checks between pro-training data and any evaluation data when using benchmarks.
Encrypting evaluation datasets Jacovi et al. 2023 advocate for the encryption of publicly released test data using a public key, coupled with licensing agreements that prohibit derivative distribution. They recommend implementing training exclusion controls for closed APIs and saleguarding lest data by refusing to perform evaluations without such controls. Moreover they suggest avoiding data that appears with the solution and relcasing internet-derived data along with the corresponding web-page context.
Not leaking data closed-source APIs Aside from technical methodologies, Balloccu al. (2024) conduct systematic literature review of 225 papers and carefully detail data leakage from them to closed-source models like the GPT-series. Overall, they conclude that -42% of reviewed papers leaked data to GPT-3.5 and GPT-4 for a total of ~4.7M benchmark samples across 263 benchmarks. The authors report evaluation malpractices and propose a list of suggested practices for evaluating closed-source LLMs.
Several data contamination mitigation techniques are detailed the survey work of Xu et al. (2024a).
5.2 Towards Open-Source Pre- Training Data
As we have seen in this study, access to the pre-training data is the first fundamental characterization of contamination detection technique. When the pro-training data are publicly available, the task of defining membership to the pre-training set becomes drastically simpler. To facilitate contamination detection, it is important for the research community to move towards the general usage of open-source pre-training sets.
Duan et al. 2024) introduce the MIMIR dataset, constructed from The Pile training set Gao et al. 2020), on the which popular open-source models such the Pythia (Biderman et al. 2023) and GPT-Ne model series are pre-trained. MIMIR covers domain-specific subsets (e.g. GitHub, Wikipedia, Arxiv) and extracts members through several n-gram sizes, allowing rescarchers to work on multiple levels of membership granularity. MIMIR has become widely used MIA research with LLMs. OLMoMIA (Kim el al., 2024) introduces membership inference dataset centered around the open-source OLMo-7B LLM (Groeneveld et al. 2024). Knowing that pre-training epoch roughly consists of 450k, training data seen before 100k pre-training steps is considered as member, and training data. 400k and 450k steps is non-member. Dolma- Book (Zhang & Wi. 2024) also leverages the publicly available pre-training data from OLMo, and samples non-member books from books from the Project Gutenberg dated after January 1st, 2024.
https://github.... com/ Www.eeeeeeeetttttteeeeeeee
https://www/ htw.:/wwwwueebbrggook org/
15

---

Published in Transactions on Machine Learning Research (07/ (07/2025)
5.3 New Evaluation Benchmarks
Researchers have crafted new datasets specifically introduced with the aim to provide contaminationn-Ire LLM evaluation.
Benchmark Perturbation Several new datasets are crafted by perturbing existing benchmarks, for in- stance through rephrasing instructions or adding more complex instructions. Perturbed data points are more challenging for LLMs, and can be designed to be adversarial examples aiming to fool the LLMs. More- over, Alzahrani et al. (2024) point out that slight perturbations to existing benchmarks may disrupt model rankings on leaderboards, calling for better evaluation benchmarks. Such benchmarks include:
GSM-Plus Li et al. 2024a). Using the 1,319 questions of the grade-school level maths word problems GSM8K dataset (Cobbe al., 2021) as seed dataset, the authors apply eight perturbations of five different types: numerical variation, arithmetic variation, problem understanding, distractor insertion and critical thinking. The datasct is generated with GPT-4 (manually verified by humans) and has 10,552 questions in total. High relative performance drop rales of above 30% are observed for open-source models. Several elaborate prompting techniques such as shain-of-thought fall short of providing meaningful improvement, but a custom compositional prompting technique provides notable performance improvemente
MATH-Parturb (Huang et al. 2023) introduces two flavors of variations to 279 problems from the hardest level (level 5) of MATH dataset (Hendrycks et al. 2021): slightly different problems solvable with the same method (MATH-P-Simpley and problems needing a different approach (MATH-P-HARD) Performance degrades notably for all LLMS on MATH-P-HARD. Interestingly, a lot of errors can be traced to memorization: the model simply blindly applics the same technique used to solve the original problem.
Dynamic Evaluation Some new datasets discard the static paradigm of existing LLM evaluation datasets, and instead propose a dynamic, ever-changing or periodically updated benchmark to avoid contamination.
LatestEval (Li et al. 2023b leverages the most recent texts to create dynamic reading comprehension evaluations. The benchmark is constructed in three steps: (i) collect the latest texts; (ii) extract key information, and (iit) construct questions based on the extracted information through template-filling LLMs.
KIEval Yu et al. 2024 is an interactive ovaluation framework incorporating LLM-powered "inter- actor', which can ask follow-up questions in multi-round dialogue that leads to contaminatioon--esillenn evaluation.
LiveCodeBench (Jain et al. 2024 continuously collects new problems from LeetCode, AtCoder and CodeForces to provide a contamination-fus froe code generation benchmark.
LiveBench (White et al. 2024 is challenging benchmark made of 18 tasks from 6 categories: math, coding, reasoning, language comprehension, instruction following, and data analysis. Tasks are more complicated versions of existing benchmarks, such as BigBench-Hard (Srivastava et al., (022), and are evaluated against an objective ground-truth rather than with human or LLM preference. To avoid con- tamination, one sixth of questions are replaced every month, chosen among oldest and easiest tasks.
Chatbot Arena (Zheng et al. 2023) is a popular crowdsourced LLM evaluation platform. Humans submit queries and are shown answers from two LLMs which quality they have to rank. By construction, this benchmark is not set in stone and changes LLM rankings as more users interact with the platform. Besides, the design of Chatbot Arena limits contamination as user-generated prompts and model response rankings are less likely to be trained on.
Several dynamic benchmarks are specifically crafted for membership-inferenn attacks:
WikiMIA (Shi et al. training (i.e., after 2023).
2024] is dynamic benchmark made of Wikipedia events created after model
ChatGPT leveraged to paraphrase examples for evaluation, as also performed
in CleanEval (Zhu et al. 2023
16

---

Published in Transactions on Machine Learning Research (07/2025)
WikiMIA-24 Fu et al. (2024a] tackle WikiMIA's 's as well as any publicly released dataset issue: cutoff dates will need to be adjusted as time goes on. To address the issue, the authors extend WikiMIA's 's cutoff to March 2024.
ArxivMIA (Liu et al. 2024 gathers paper abstracts from the Computer Science and Maths sections of Arxiv. Abstracts dated alter 2024 are considered non-members of LLMs pre-training sets.
PatentMIA (Zhang et al. 2024b) constructs benchmarks where non-members are 5,000 patents from Google-Patents which are Chinese language and dated after March 1st, 2024, and members are 5,000 such patents dated from before Jamuary 1st, 2023.
Protected Evaluation Another strategy to avoid contamination of evaluation data is to protect it by locking it out of the public domain. Termite (Ranaldi al. 2024 is new text-to-SQI. dataset, locked out
of public access through search engines via an encryption key. Termite is made of handerafted databases, each paired with around five queries, designed to match propertics of the Spider dataset (Yu et al. 2018 which shows high signs of contamination on GPT-335..
Scaling datasets used for contamination detection is critical, and such datasets should follow advancements in scale and complexity of datasets used to evaluate LLMs. Samuel et al. (2024) emphasize that as late 2024, there remains a wide discrepancy between datasets used to evaluate LLM performance and datasets used to detect contamination, with the latter usually lagging the former by several years.
5.4 Future Challenges
Given the fast and ever-changing landscape of machine learning research, future directions for contamination detection in LLMs could encompass broad variety of methodologies and technological aspects. We highlight few potentially critical areas of focus:
Real-time contamination detection systems Real-time data contamination detection systems that can monitor data streams continuously and alert users to potential contamination events as they happen would be of upmost importance in critical applications like finance (Frésard et al. 2011) and healthcare (Gao ct al. 2022) where data integrity is paramount and model reliability critical. Given the sheer volume of new data uploaded daily on the Internet, solving this lask requires major technological breakthroughs. The key difficulty lies in achieving high recall, to ensure that no contaminated data sample slips through undetected. We hypothesize that compression (Han et al. 2015), distillation (Hinton et al. 2015 Sanh et al. 2019) and quantization (Banner et al. 2018 Sun et al. (020) techniques could play a key role in solving this problem, by decreasing model size, accelerating inference and scaling edge-device deployment of anomaly detection models or closed-data contamination detection techniques. Besides, leveraging federated learning and edge computing could enable docentralized, privacy-preserving detoction at the source of data generation.
Bypassing contamination detection Dekoninck et al. (2024) show very effective way to bypass several existing contamination detection methods: Evasive Augmentation Learning (EAL) consists in paraphrasing benchmarks with GPT-4 and fine-tuning the LLM on the paraphrased data. This method can bypass even oracle-access contamination detection methods, calling for the development of more robust methods, An intuitive future line of defense to such techniques is to incorporate largo-scale adversarial training (Jia & Liang. 2017 Liu al. 2020 into contamination delection models, and train these models on benchmarks which have been paraphrased or noised. We expect witness hat-and-mouse game between techniques such as EAL, which contaminate under the radar, and contamination detection techniques catching up to the finest transformations of benchmarks, such paraphrases.
Ethical and legal data frameworks Ethical and legal frameworks CChang 2021 are needed to govern the collection, usage, and management of LLM pre-training data. A strong legal framework will help prevent the inclusion of data from unethical sources or from copyrighted sources, as well as help prevent the contami- nation of evaluation benchmarks into widely used pre-training data sources (c.g., CommonCrawi] However, adopting such worldwide pre-tranning data legal framework would be challenging due several reasons. First,, the lines blurry between public and private data, and many web-crawled datasets include mix of
17

---

Published in Transactions on Machine Learning Research (07 (07/2025)
public, open-access and proprictary data. Jurisdiction surrounding data varies from country to another, complicating the adoption of global standards. Besides, the authorization to train machine learning models on copyrighted data remains unclear in many jurisdictions to this day. Another major blocker is the fact that pre-training data is nowadays treated with secrocy by large technological companics relcasing flagship LLMs, as publishing the exact pre-training data would enable their compelitors to catch up with their own LLMs. Lastly, enforcing and auditing a legal framework surrounding pre-training pipelines practice would be very complex and costly. Above all, legal frameworks on pre-training data are needed to protect per- sonal data privacy. The General Data Protection Regulation (GDPR), acted in 2018 in the European Union (EU), imposes strict rules on how organizations can collect, store and use personal data from EU citizens. With the exponential growth of pre-training corpora, it becomes harder and harder yet critical to ensure that personal data does not get included into LLM training sets. Once an LLM has seen and memorized personal data, there is no clear mechanism for an individual to exercise their right to erase their personal data from the LLM's knowledge. the near future, developing contamination detection techniques that do not compromise individual privacy will be crucial (Lin el al. 2010; Hayes & Ohrimenko 2018
6 Conclusion
As LLMs rapidly evolve and the training data become more extensive, LLMs' performances are inevitably biased by contamination. Eventually, most (if not all) target-answor-based public evaluation datasets end up in commonly used pre-training data dumps, which are updated every few months. Therefore, to help detect and quantify contamination, in this survey, we investigated, structured, and classified the current landscape of contamination detection in LLMs. Depending on researcher's level of access to the data and to the LLM parameters, survey can quickly guide them toward set of relevant tools to detect contamination.
References
Amirhossein Abaskohi, Sascha Rothe, and Yadollah Yaghoobzadeh. Lm-eppf: Paraphrasing-guidee data augmentation for contrastive prompt-based few-shot fine-tuning. arXiv preprint arXiv:2305.18100 2023.
Josh Achiam, Steven Adler, Sandhini Agarwal, Lama Ahmad, Ilge Akkaya, Florencia Leoni Aleman, Diogo Almeida, Janko Altenschmidt, Sam Altman, Shyamal Anadkat, et al. Gpt-4 technical report. arXiv preprint ar xrXi::2303.088 08774, 2023.
Rachith Aiyappa, Jisun An, Haewoon Kwak, and Yong-yeol Ahn. Can trust the evaluation on Chat- GPT? In Anaelia Ovalle, Kei-Wei Chang, Ninareh Mehrabi, Yada Pruksachatkun, Aram Galystan, Jwala Dhamala, Apury Verma, Trista Cao, Anoop Kumar, and Rahul Gupta (eds.). Proceedings of the 3rd Workshop on Trustworthy Natural Language Processing (TrustNLP 2023), pp. 47 54, Toronto, Canada, July 2023. Association for Computational Linguistics. doi: 10.18653/v1. h001///22.ttmmtt 2023.truustlpp1..5 URL https://aclanttholog. org trustalp-1.5//
Norah Abahrani, Hisham Abdullah Alyahya, Yazeed Alnumay, Sultan Alrashed, Shaykhah Alsubaie, Yusef Almushaykeh, Faisal Mirza, Nouf Alotaibi, Nora Altwairesh, Areeb Alowisheq, al. When benchmarks are targets: Revealing the sensitivity of large language model leaderboards. arXiv preprint arXiv:2402.0170 01711 2024.
Rohan Anil, Andrew M Dai, Orhan Firat, Melvin Johnson, Dmitry Lepikhin, Alexandre Passos, Siamak Shakeri, Emanuel Taropa, Paige Bailey, Zhifeng Chen, et al. Palm 2 technical report. arXiv preprint arXiv:2305. 10403, 2023.
Jacob Austin, Augustus Odena, Maxwell Nye, Maarten Bosma, Henryk Michalewski, David Dohan, Ellen Jiang, Carrie Cai, Michael Terry, Quoe Le, et al. Program synthesis with large language models. arXiv preprint ar STf::2188.07332 2021.
Jinze Bai, Shuai Bai, Yunfci Chu, Zeyu Cui, Kai Dang, Xiaodong Deng, Yang Fan, Wenbin Ge, Yu Han, Fei Huang, al. Qwen technical report. Xix preprint or orXin:2309. 16009, 2023.
18

---

Published in Transactions on Machine Learning Research (07 (07/2025)
Simone Balloccu, Patrícia Schmidtová, Mateusz Lango, and Ondicj Dušek. Leak, cheat, repeat: Data contamination and evaluation malpractices in closed-source llms. arXiv preprinl or arXiv:2402.03000 2024.
Satanjeev Banerjee and Alon Lavie. METEOR: An automatic metric for MT evaluation with improved correlation with human judgments. In Jade Goldstein, Alon Lavie, Chin- Yew Lin, and Clare Voss (eds.), Proceedings of the ACL Workshop on Intrinsic and Eatrinsic Evaluation Measures for Machine Transla- tion and/or or Summarization, pp. 65 72, Ann Arbor, Michigan, June 2005. Association for Computational Linguistics. URL
org/W05-0909
Ron Banner, Itay Hubara, Elad Hoffer, and Daniel Soudry. Scalable methods for 8-bit training of neural networks. Advances in neural information processing systems, 31, 2018.
James Betker, Gabriel Goh, Li Jing, Tim Brooks, Jianfeng Wang, Linjie Li, Long Ouyang, Juntang Zhuang, Joyce Lee, Yufei Guo, et al. Improving image generation with better captions. Computer Science. https://edn. openai. com/ com/papers httpppppp/////e-// pdf, 2(3):8, 2023.
Xiao Bi, Deli Chen, Guanting Chen, Shanhuang Chen, Damai Dai, Chengqi Deng, Honghui Ding, Kai Dong, Qiushi Du, Zhe Fu, et al. Deepseek Ilm: Scaling open-source language models with longlermism. arxiv preprint ar Xiv::2201 02954, 2024.
Stella Biderman, Hailey Schoelkopf, Quentin Gregory Anthony, Herbie Bradley, Kyle O'Brien, Eric Hallahan, Mohammad Aflah Khan, Shivanshu Purohit, USVSN Sai Prashanth, Edward Raff, et al. Pythia: A suite for analyzing large language models across training and scaling. In International Conference on Machine Learning, pp. 2397 2430. PMLR, 2023.
Yonatan Bisk, Rowan Zellers, Jianfeng Gao, Yejin Choi, et al. Piqa: Reasoning about physical commonsense in natural language. In Proceedings of the AAAI conference on artificial intelligence, volume 34, pp. 7432-7439, 2020.
Tom Brown, Benjamin Mann, Nick Ryder, Melanie Subbiah, Jared D Kaplan, Prafulla Dhariwal, Arvind Neelakantan, Pranav Shyam, Girish Sastry, Amanda Askell, et al. Language models are few-shot learners. Advances in neurol information processing systems, 33:1877 1901, 2020.
Zheng Cai, Maosong Cao, Haojiong Chen, Kai Chen, Keyu Chen, Xin Chen, Xun Chen, Zehui Chen, Zhi Chen, Pei Chu, Xiaoyi Dong, Haodong Duan, Qi Fan, Zhaoye Fei, Yang Gao, Jiaye Ge, Chenya Gu, Yuzhe Gu, Tao Gui, Aijia Guo, Qipeng Guo, Conghui He, Yingfan Hu, Ting Huang, Tao Jiang, Penglong Jiao, Zhenjiang Jin, Zhikai Lei, Jiaxing Li, Jingwen Li, Linyang Li, Shuaibin Li, Wei Li, Yining Li, Hongwei Liu, Jiangning Liu, Jiawei Hong, Kaiwen Liu, Kuikun Liu, Xiaoran Liu, Chengqi Lv, Haijun Lv, Kai Lv, Li Ma, Runyuan Ma, Zerun Ma, Wenchang Ning, Linke Ouyang, Jiantao Qiu, Yuan Qu, Fukai Shang, Yunfan Shao, Demin Song, Zifan Song, Zhihao Sui, Peng Sun, Yu Sun, Huanze Tang, Bin Wang, Guoteng Wang, Jiaqi Wang, Jiayu Wang, Rui Wang, Yudong Wang, Ziyi Wang, Xingjian Wei, Qizhen Weng, Fan Wu, Yingtong Xiong, Chao Xu, Ruiliang Xu, Hang Yan, Yirong Yan, Xiaogui Yang, Haochen Ye, Hualyuan Ying, Jia Yu, Jing Yu, Yuhang Zang, Chuyu Zhang, Li Zhang, Pan Zhang, Peng Zhang, Ruijic Zhang, Shuo Zhang, Songyang Zhang, Wenjian Zhang, Wenwei Zhang, Xingcheng Zhang, Xinyue Zhang, Hui Zhao, Qian Zhao, Xiaomeng Zhao, Fengzhe Zhou, Zaida Zhou, Jingming Zhuo, Yicheng Zou, Xipeng Qiu, Yu Qiao, and Dahua Lin. Internim2 technical report, 2024.
Jialun Cao, Wuqi Zhang, and Shing- Chi Cheung. Concerned with data contamination? assessing counter- measures in code language model. arXiv preprint arXiv:2408. 168988 2024.
Nicholas Carlini, Florian Tramer, Eric Wallace, Matthew Jagielski, Ariel Herbert- Voss, Katherine Lee, Adam Roberts, Tom Brown, Dawn Song, Ulfar Erlingsson, et al. Extracting training data from large language models. In 30th USENIX Security Symposium (USENIX Security 21), pp. 2633-2650, 2021.
Nicholas Carlini, Steve Chien, Milad Nasr, Shuang Song, Andreas Terzis, and Florian Tramer. Membership inference attacks from first principles. In 2022 IEEE Symposium on Security and Privacy (SP), pp. 1997-1914. IEFF, 2022.
19

---

Published in Transactions on Machine Learning Research (07/2025)
Nicholas Carlini, Daphne Ippolito, Matthew Jagielski, Katherine Lee, Florian Tramer, and Chiyuan Zhang. Quantifying memorization across neural language models. In The Eleventh International Conference on Learning Representations, 2023. URL https://openreviier httpp://ppppeeesssssoo//ooootttr 1cK.
Hongyan Chang, Ali Shahin Shamsabadi, Kleomenis Katevas, Hamed Haddadi, and Reza Shokri. Context-aware membership inference attacks against pre-trained large language models. arXiv preprint arXiv:2409.18720 2024.
Kent Chang, Mackenzie Cramer, Sandeep Soni, and David Bamman. Speak, memory: An archaeology of books known to ChatGPT/GPT-4, In Houda Bouamor, Juan Pino, and Kalika Bali (eds.), Proceedings of the 2023 Conference Empirical Methods in Natural Longuage Processing, pp. 7312-7322 7327, Singapore, December 2023. Association for Computational Linguistics. doi: 11111865/112002 w0w...m.lp....mmmmmmmmmmmmmm monnlp-main..e 453. URL
https://acclanthology www.gg.....oooooooo..o.........o org/oog//332 emnlp wwwpppmma.....mmmmmmmmmmmmmma..m 453
Victor Chang. An ethical framework for big data and smart cities. Technological Forecasting and Social Change, 165:120559, 2021.
Hailin Chen, Fangkai Jiao, Xingxuan Li, Chengwei Qin, Mathieu Ravaut, Ruochen Zhao, Caiming Xiong, and Shafiq Joty. Chatgpt's onc-ycar anniversary: Are open-source large language models catching up? arXiv preprint arXtw:2311.1698, 2023.
Mark Chen, Jerry Tworek, Heewoo Jun, Qiming Yuan, Henrique Ponde de Oliveira Pinto, Jared Kaplan, Harri Edwards, Yuri Burda, Nicholas Joseph, Greg Brockman, et al. Evaluating large language models trained on code. Xáv preprint arXiv:2107.0337, 033774, 2021a.
Zhiyu Chen, Wenhu Chen, Charese Smiley, Sameena Shah, Iana Borova, Dylan Langdon, Reema Moussa, Matt Beane, Ting-Hao Huang, Bryan Routledge, et al. Finqa: A dataset of numerical reasoning over financial data. arXiv preprint arXiv:2109.001 0022,, 2021b.
Eunsol Choi, He He, Mohit lyyer, Mark Yatskar Wen-tau Yih, Yejin Choi, Percy Liang, and Luke Zettle- moyer. Quac: Question answering in context. arXiv preprint arXiv: 1808. 070807, 2018
Aakanksha Chowdhery, Sharan Narang, Jacob Devlin, Maarten Bosma, Gaurav Mishra, Adam Roberts, Paul Barham, Hyung Won Chung, Charles Sutton, Sebastian Gehrmann, et al. Palm: Scaling language modeling with pathways. Journal of Machine Learning Research, 24(240):1-113, 2023.
Christopher Clark, Kenton Lee, Ming- Wei Chang, Tom Kwiatkowski, Michael Collins, and Kristina Toutanova. Boolq: Exploring the surprising difficulty of natural yes/no questions. arXiv preprint arXiv:1955.10044, 2019.
Peter Clark, Isaac Cowhey, Oren Etzioni, Tushar Khot, Ashish Sabharwal, Carissa Schocnick, and Oyvind Tafjord. Think you have solved question answering? try are, the ai2 reasoning challenge. arXiv preprint arXiv:1803. 10X50:1803.0545, 2018.
Karl Cobbe, Vineet Kosaraju, Mohammad Bavarian, Mark Chen, Heewoo Jun, Lukasz Kaiser, Matthias Plappert, Jerry Tworek, Jacob Hilton, Reichhro Nakano, et al. Training verifiers to solve math word problems. arXiv preprint arxiv:2110. 14168, 2021.
Jasper Dekoninck, Mark Niklas Müller, Maximilian Baader, Mare Fischer, and Martin Vechev. Evading data contamination detection for language models (too) easy. arXiv preprint orXiv:2402.02823, 2024.
Chunyuan Deng, Yilun Zhao, Xiangru Tang, Mark Gerstein, and Arman Cohan. Investigating data contam- ination in modern benchmarks for large language models. orXiv preprint ar orXiw:2911.009 007889, 2023.
Chunyuan Dong, Yilun Zhao, Yuzhao Heng, Yitong Li, Jiannan Cao, Xiangru Tang, and Arman Cohan. Unveiling the spectrum of data contamination in language models: A survey from detection remediation. Xiviv preprint ar Xi0:2406.. 14644, 2024.
20

---

Published in Transactions on Machine Learning Research (07/2025)
Jacob Devlin, Ming- Wei Chang, Kenton Lee, and Kristina Toutanova. Bert: Pre-training of deep bidirectional transformers for language understanding. In Proceedings of the 2019 conference of the North American chapter of the association for computational linguistics: human language technologies, volume 1 (long and short papers), pp. 4171-4186, 2019.
Jesse Dodge, Maarten Sap, Ana Marasovié, William Agnew, Gabriel Ilharco, Dirk Groeneveld, Margaret Mitchell, and Matt Gardner. Documenting large webtext corpora: A case study on the colossal clean crawled corpus. Xiw preprint orXiv:2104.08758, 2021.
Yihong Dong, Xue Jiang, Huanyu Liu, Zhi Jin, and Ge Li. Generalization memorization: Data contami- nation and trustworthy evaluation for large language models. arXiv preprint arXiv:2402. 15938, 2024.
Michael Duan, Anshuman Suri, Niloofar Mireshghallah, Sewon Min, Weijia Shi, Luke Zettlemoyer, Yulia Tsvetkov, Yejin Choi, David Evans, and Hannanch Hajishirzi. Do membership inference attacks work on large language models? Xiv preprint Xi2:2402.007441, 2024.
Abhimanyu Dubey, Abhinav Jauhri, Abhinav Pandey, Abhishck Kadian, Ahmad Al-Dahle, Aiesha Letman, Akhil Mathur, Alan Schelten, Amy Yang, Angela Fan, et al. The llama 3 herd of models. arXiv preprinl arXiv:2407.21788 2024.
A Aparna Elangovan; Jiayuan He, and Karin Verspoor. Memorization vs. generalization: quantifying data leakage in nlp performance evaluation. arXie preprint or Xiv:2102.01818, 2021.
Qizhang Feng, Siva Rajesh Kasa, Hyokun Yun, Choon Hui Teo, and Sravan Babu Bodapati. Exposing privacy
gaps: Membership inference attack on preference data for Ilm alignment. arXiv preprint orXiv:2407.064.3 2024.
Laurent Frésard, Christophe Pérignon, and Anders Wilhelmsson. The pernicious effects of contaminated data in risk management. Journal of Banking & Finance, 35(10):2569 2583, 2011.
Wenjie Fu, Huandong Wang, Chen Gao, Guanghua Liu, Yong Li, and Tao Jiang. Practical membership inference attacks against fine-tuned large language models via self-prompt calibration. arXiv preprinl arXiv:2311.06002 2023.
Wenjie Fu, Huandong Wang, Chen Gao, Guanghua Liu, Yong Li, and Tao Jiang. Mia-tuner: Adapting large language models pre-training text detector. arXiv preprint orXiv:2408. 000600..........1111 2024a.
Yujuan Fu, Ozlem Uzuner, Meliha Yetisgen, and Fei Xia. Does data contamination detection work (well) for lims? survey and evaluation on detection assumptions. arXie preprint orXiv:2410.18966 2024b.
Leo Gao, Stella Biderman, Sid Black, Laurence Golding, Travis Hoppe, Charles Foster, Jason Phang, Horace He, Anish Thite, Noa Nabeshima, et al. The pile: An 800gb dataset of diverse text for language modeling. arXiv preprint XrXix:2101.0 0.00007, 2020.
Xuehong Gao, Guozhong Huang, Qiuhong Zhao, Cejun Cao, and Huiling Jiang, Robust optimization model for medical staff rebalancing problem with data contamination during covid-19 pandemic. International Journal of Production Research, 1075 2022.
Gemma Team Google DeepMind. Gemma: Open Models Based on Gemini Research and Technology. Tochnical report, Google DeepMind, 2024. Available online at gemma-l-report gemma-l-eprrt@ggggllcooom @google.com.
Shahriar Golchin and Mihai Surdeamu. Data contamination quiz: A tool to detect and estimate contamination in large language models. arXiv preprint arXiu:2311.06233 2023a.
Shahriar Golchin and Mihai Surdeanu. Time travel in lmns: Tracing data contamination in large language models. ar preprint arXiv:2308.08403 2023b.
Dirk Grocneveld, Beltagy, Pete Walsh, Akshita Bhagia, Rodney Kinncy, Oyvind Tafjord, Ananya Harsh Jha, Hamish Ivison, Ian Magnusson, Yizhong Wang, et al. Olmo: Accelerating the science of language models. Xiv preprint or 0020022402.00888, 2024.
21

---

Published in Transactions on Machine Learning Research (07/)025)
Suriya Gunasekar, Yi Zhang, Jyoti Aneja, Caio César Teodoro Mendes, Allie Del Giorno, Sivakanth Gopi, Mojan Javaheripi, Piero Kaulfmann, Gustavo de Rosa, Olli Saarikivi, et al. Textbooks are all you need. arXiv preprint arXic:2306.11644 2023.
Song Han, Huizi Mao, and William J Dally. Deep compression: Compressing deep neural networks with pruning, trained quantization and huffman coding. areie preprint orXiv:1510.0014,, 2015.
Jamic Hayes and Olga Ohrimenko. Contamination attacks and mitigation in multi-party machine learning, Advances in neural informalion processing systems, 31, 2018.
Dan Hendrycks, Collin Burns, Steven Basart, Andy Zou, Mantas Mazeika, Dawn Song, and Jacob Steinhardt. Measuring massive multitask language understanding. arXic preprint arXiu:200900330 2020.
Dan Hendrycks, Collin Burns, Saurav Kadavath, Akul Arora, Steven Basart, Eric Tang, Dawn Song, and Jacob Steinhardt. Mcasuring mathematical problem solving with the math dataset. arXiv preprint arXiv:2103.0385 2021.
Karl Moritz Hermann, Tomas Kocisky, Edward Grefenstette, Lasse Espcholt, Will Kay, Mustafa Suleyman, and Phil Blunsom. Teaching machines to read and comprehend. Advances in newral information processing systems, 28, 2015.
Felix Hill, Antoine Bordes, Sumit. Chopra, and Jason Weston. The goldilocks principle: Reading children's books with explicit memory representations. arXiv preprint arX1:15511.0201,, 2015.
Geoffrey Hinton, Oriol Vinyals, and Jeff Dean. Distilling the knowledge in a neural network. arXiv preprint arXiv:1503.0255 2015.
Yuzhen Huang, Yuzhuo Bai, Zhihao Zhu, Junlei Zhang, Jinghan Zhang, Tangjun Su, Junteng Liu, Chuancheng Lv, Yikai Zhang, Yao Fu, et al. C-eval: multi-level mutti-discipline chinese evaluation suite for foundation models. Advances in Neural Information Processing Systems, 36:62991-63010, 2023.
Binyuan Hui, Jian Yang, Zeyu Cui, Jiaxi Yang, Dayiheng Liu, Lei Zhang, Tianyu Liu, Jiajun Zhang, Bowen Yu, Keming Lu, et al. Qwen2. 5-coder technical report. arXiv preprint orXiv:2409.1211 12186, 2024.
Alon Jacovi, Avi Caciularu, Omer Goldman, and Yoav Goldberg. Stop uploading test data in plain text: Practical strategies for mitigating data contamination by evaluation benchmarks. arXiv preprint arXiv:2305.1010 2023.
Naman Jain, King Han, Alex Gu, Wen-Ding Li, Fanjia Yan, Tianjun Zhang, Sida Wang, Armando Solar- Lezama, Koushik Sen, and Ion Stoica. Livecodcbench: Holistic and contamination free evaluation of large language models for code. arXie preprinl ar xxx::2403.07974, 2024.
Robin Jia and Percy Liang. Adversarial examples for evaluating reading comprehension systems. arXiv preprint 17i::177707322 2017.
Albert Q Jiang, Alexandre Sablayrolles, Antoine Roux, Arthur Mensch, Blanche Savary, Chris Bamford, Devendra Singh Chaplot, Diego de las Casas, Emma Bou Hanna, Florian Bressand, et al. Mixtral of experts. Xiv preprint arXiv:2401.0400 2024a.
Minhao Jiang, Ken Ziyu Liu, Ming Zhong, Rylan Schaeffer, Siru Ouyang, Jiawei Han, and Sanmi Koyejo. Investigating data contamination for pre-training language models. arXiv preprint #rXiv2240100055 2024b.
Mandar Joshi, Funsol Choi, Daniel S Weld, and Luke Zettlemoyer. Triviaqa: A large scale distantly super- vised challenge datasct for reading comprehension. arXiv preprint 1xXiv:1755.033 2017.
Gyuwan Kim, Yang Li, Evangelia Spiliopoulou, Jie Ma, Miguel Ballesteros, and William Yang Wang. Detoct- ing training data of large language models via expectation maximization. arxi preprint or 210.:22410007582, 2024.
22

---

Published in Transactions on Machine Learning Research (07 2205))
Denis Kocetkov, Raymond Li, Loubna Ben Allal, Jia Li, Chenghao Mou, Carlos Muñoz Ferrandis, Yacine Jernite, Margaret Mitchell, Sean Hughes, Thomas Wolf, et al. The stack: 3 tb of permissively licensed source code. arXiv preprint vrXic:2211.155333 2022.
Tom Kwiatkowski, Jennimaria Palomaki, Olivia Redfield, Michael Collins, Ankur Parikh, Chris Alberti, Danielle Epstein, Illia Polosukhin, Jacob Devlin, Kenton Lee, et al. Natural questions: benchmark for question answering research. Transactions of the Association for Computational Linguislics, 7:453 466, 2019.
Ariel N Lee, Cole J Hunter, and Nataniel Ruiz. Platypus: Quick, cheap, and powerful refinement of llms. arXiv preprinl orx::22008.077 07317, 2023.
Brian Lester, Rami Al-Rfou, and Noah Constant. The power of scale for parameter-cffiient prompt tuning, arXiv preprint Xiv:21004 08691, 2021.
Hector Levesque, Ernest Davis, and Leora Morgenstern. The winograd schema challenge. In Thirleenth international conference on the principles of knowledge representation and reasoning, 2012.
Changmao Li and Jeffrey Flanigan. Task contamination: Language models may not be few-shot anymore. arXiv preprint ar Xiv:2331 16337, 2023.
Qintong Li, Leyang Cui, Xueliang Zhao, Lingpeng Kong, and Wei Bi. Gsm-plus: A comprehensive benchmark for evaluating the robustness of Ilms as mathematical problem solvers. arXiv preprint arXiv:2402. 19255, 2024a.
Xingxuan Li, Ruochen Zhao, Yew Ken Chia, Bosheng Ding, Lidong Bing, Shafiq Joty, and Soujanya Poria. Chain of knowledge: A framework for grounding large language models with structured knowledge bases. arXiv preprint XrXin:2305. 18269, 2023a.
Yucheng Li. Estimating contamination via perplexity: Quantifying memorisation in language model evalu- ation. arXiv preprint arXiv:2309.10677 2023a.
Yuchong Li. An open source data contamination report for llama series models. arXio preprint
arXiv:2310. 17589, 2023b.
Yucheng Li, Frank Geurin, and Chenghua Lin. Avoiding data contamination in language model evaluation: Dynamic test construction with latest materials. arXiv preprint arXhw:231. 2023b.
Yuying Li, Gaoyang Liu, Chen Wang, and Yang Yang. Generating is believing: Membership inference attacks against regmenedl-nnmmented generation. arXiv preprint arXiv:2406. 19234, 2024b.
Stephanic Lin, Jacob Hilton, and Owain Evans. Truthfulqa: Mcasuring how models mimic human falschoods.
arXiv preprint ar Xix::22100 07958, 2021.
Xiaodong Lin, Rongxing Lu, and Xuemin Shen. Mdpa: multidimensional privacy-preservine aggregation scheme for wireless sensor networks. Wireless Communications and Mobile Computing, 10(6):843 856, 2010.
Xiaodong Liu, Hao Cheng, Pengcheng He, Weizhu Chen, Yu Wang, Hoifung Poon, and Jianfeng Gao. Adversarial training for large neural language models. ar preprint Xiw:2004008994, 2020.
Zhenhua Liu, Tong Zhu, Chuanyuan Tan, Haonan Lu, Bing Liu, and Wenliang Chen. Probing language models for pre-training data detection. arXiv preprint ar Xrx::22406.0 01333, 2024.
Andrew Maas, Raymond E Daly, Peter Pham, Dan Huang, Andrew Y Ng, and Christopher Potts. Learning word vectors for sentiment analysis. In Proceedings of the 49th annual meeting of the association for compulational linguisties: Human language lechnologies, pp. 142 150, 2011.
Rien Maertens, Charlotte Van Petegem, Niko Strijbol, Toon Bacyens, Arne Carla Jacobs, Peter Dawyndt, and Bart Mesuere. Dolos: Language-agnostic plagiarism detection in source code. Journal of Computer Assisted Learning, 38(4):1046-1061 2022.
23

---

Published in Transactions on Machine Learning Research (07/2025)
Inbal Magar and Roy Schwartz. Data contamination: From memorization to exploitation. arXie preprint arXiv:22030082) 08242 2022.
Pratyush Maini, Hengrui Jia, Nicolas Papernot, and Adam Dziedzic. Lim dataset inference: Did you train on my dataset? Advances in Neural Information Processing Systems, 37:124069-124092, 2025.
Matthicu Meeus, Shubham Jain, Marek Rei, and "ves-Alexandrs de Montjoye. Did the neurons read your book? document-level membership inference for large language models. In 33rl USENIX Security Sym- posium (USENIX Security 24); pp. 2369-2385, 2024a.
Matthicu Moeus, Igor Shilov, Shubham Jain, Manuel Faysse, Marck Rei, and Ales-nlex de Montjoye. Sok: Membership inference atlacks on 1lms are rushing nowhere (and how to fix it). aiv preprinl arXiv:2406. 17975, 2024b.
Stephen Mcrity, Caiming Xiong, James Bradbury, and Richard Socher. Pointer sentinel mixture models. arXiv preprinl X60: 07843,, 2016.
Todor Mihaylov, Peter Clark, Tushar Khot, and Ashish Sabharwal. Can a suit of armor conduct electricity? dataset for open book question answering. arXiv preprint orXii:1800) 1800:02787 2018.
Shashi Narayan, Shay B Cohen, and Mirella Lapata. Don't give me the details, just the summary! topic- aware convolutional neural networks for extreme summarization. arXiv preprint arXiv:1888.08 2018.
Milad Nasr, Nicholas Carlini, Jonathan Hayase, Matthew Jagielski, A Feder Cooper, Daphne Ippolito, Christopher A Choquette-Choo Eric Wallace, Florian Tramer, and Katherine Lee. Scalable extraction of training data from (production) language models. arXiv preprint arXiv:2811.1000 2023.
Erik Nijkamp, Bo Pang, Hiroaki Hayashi, Lifu Tu, Huan Wang, Yingbo Zhou, Silvio Savarese, and Caiming Xiong. Codegen: An open large language model for code with multi-turn program synthesis. arXiv preprint arXiv:2208.1344 13474, 2022.
Yonatan Oren, Nicole Meister, Niladri Chatterji, Faisal Ladhak, and Tatsunori B Hashimoto. Proving test set contamination in black box language models. arXiv preprint arXiv:2310.1766 17622, 2023.
Long Ouyang, Jeffrey Wu, Xu Jiang, Diogo Almeida, Carroll Wainwright, Pamela Mishkin, Chong Zhang, Sandhini Agarwal, Katarina Slama, Alex Ray, et al. Training language models to follow instructions with human feedback. Advances in neurol information processing systems, 35:27730-27744 2022a.
Long Ouyang, Jeffrey Wu, Xu Jiang, Diogo Almeida, Carroll Wainwright, Pamela Mishkin, Chong Zhang, Sandhini Agarwal, Katarina Slama, Alex Ray, et al. Training language models follow instructions with human feodback. Advances in neural information processing systems, 35:27730-27744 2022b.
Mustafa Özdayi, Charith Peris, Jack FitzGerald, Christophe Dupuy, Jimit Majmudar, Haidar Khan, Rahil Parikh, and Rahul Gupta. Controlling the extraction of memorized data from large language models via prompt-tuning, In Proceedings of the 61st Annual Meeting of the Association for Computational Linguistics (Volume 2: Shorl Papers), ACL 2023, Toronlo, Canada, July 9-14, 2023, pp. 1512 1521. Association for Computational Linguistics, 2023.
Medha Palavalli, Amanda Bertsch, and Matthow R Gormley. taxonomy for data contamination in large language models. arXiv preprint ar Xit::447.008110 2024.
Denis Paperno, Germán Kruszewski, Angeliki Lazaridou, Quan Ngoc Pham, Raffaella Bernardi, Sandro Pezzelle, Marco Baroni, Gemma Boleda, and Raquel Fernández. The lambada dataset: Word prediction requiring broad discourse context. arXiv preprint Xrx:::1666 06031, 2016.
Alec Radford, Jeffrey Wu, Rewon Child, David Luan, Dario Amodei, Ilya Sutskever, et al. Language models are unsupervised multitask learners. OpenAl blog, 188):9, 2019.
24

---

Published in Transactions on Machine Learning Research (07/2025)
Rafael Rafailov, Archit Sharma, Eric Mitchell, Christopher D Manning, Stefano Ermon, and Chelsea Finn. Direct preference optimization: Your language model is secretly a reward model. Advances in Neural Information Processing Systems, 36:53728 53741, 2023.
Colin Raffel, Noam Shazeer, Adam Roberts, Katherine Lee, Sharan Narang, Michael Matena, Yanqi Zhou, Wei Li, and Peter J. Liu. Exploring the limits of transfer learning with a unified text-to-text transformer. J. Mach. Learn. Res,, 21(1), jan 2020. ISSN 1532-4435.
Pranav Rajpurkar, Jian Zhang, Konstantin Lopyrev, and Percy Liang. Squad: 100,000+ questions for machine comprehension of text. arXiv preprint XrXiv:1666 05250, 2016.
Pranav Rajpurkar, Robin Jia, and Percy Liang. Know what you don't know: Unanswerable questions for squad. erXiv preprint 106.:2200.08822 2018.
Federico Ranaldi, Elena Sofia. Ruzzetti, Dario Onorati, Loonardo Ranaldi, Cristina Giannone, Andrea Favalli, Raniero Romagnoli, and Fabio Massimo Zanzotto. Investigating the impact of data contamination of large language models in text-to-sql translation. arXic preprint 0rXiv:2402.08100, 2024.
Siva Reddy, Danqi Chen, and Christopher D Manning. Coqa: A conversational question answering challenge. Transactions of the Association for Computational Linguistics, 7:249-266 2019.
Nils Reimers and Iryna Gurevych. Sentence-bert: Sentence embeddings using siamese bert-networks. arXiv preprint Xrxi::1905 10084, 2019.
Martin Riddell, Ansong Ni, and Arman Cohan. Quantifying contamination in evaluating code generation capabilities of language models. orXiv preprint arXiv:2408.0480 04811, 2024.
Manley Roberts, Himanshu Thakur, Christine Herlihy, Colin White, and Samuel Dooley. Data contamination through the lens of time. Xixiv preprint arXiv:2310.1 2023.
Oscar Sainz, Jon Ander Campos, Iker Carcia-Ferrero Julen Etxaniz, Oier Lopez de Lacalle, and Eneko Agirre. Nlp evaluation in trouble: On the need to measure Ilm data contamination for each benchmark. arXiv preprint ar XrX:::2300. 18018, 2023.
Keisuke Sakaguchi, Ronan Le Bras, Chandra Bhagavatula, and Yejin Choi. Winogrande: An adversarial winograd schema challenge at scale. Communications of the ACM, 64(9):90-106, 2021.
Vinay Samuel, Yue Zhou, and Henry Peng Zou. Towards data contamination detection for modern large language models: Limitations, inconsistencies, and oracle challenges. arXie preprint #xXi::240900927, 2024.
Victor Sanh, Lysandre Debut, Julien Chaumond, and Thomas Wolf. Distilbert, a distilled version of bert: smaller, faster, cheaper and lighter. arXiu preprint 1rXix:1910.0110 2019.
Maarten Sap, Hannah Rashkin, Derek Chen, Ronan LeBras, and Yejin Choi. Socialiqa: Commonsense
reasoning about social interactions. arXiv preprint orXiv:1904.09798, 2019.
John Schulman, Filip Wolski, Prafulla Dhariwal, Alec Radford, and Oleg Klimov, Proximal policy optimiza- tion algorithms. arXiv preprint arXiv: 707.06347, 2017.
Weijia Shi, Anirudh Ajith, Mengzhou Xia, Yangsibo Huang, Daogao Liu, Terra Blevins, Danqi Chen, and Luke Zettlemoyer. Detecting pretraining data from large language models. Tn The Twelfth International Conference on Learning Representations, 2024. URL
Aaditya K Singh, Muhammed Yusuf Kocyigit, Andrew Poulton, David Esiobu, Maria Lomeli, Gergely Szil- vasy, and Dicuwke Hupkes. Evaluation data contamination in 1lms: how do we measure and (when)
does it matter? Xiv preprint 1r10022411.03923, 2024.
Aarohi Srivastava, Abhinav Rastogi, Abhishek Rao, Abu Awal Md Shoob, Abubakar Abid, Adam Fisch, Adam R Brown, Adam Santoro, Aditya Gupta, Adríà Garriga-Alonso, al. Beyond the imitation game: Quantifying and extrapolating the capabilities of language models. Xiv preprint or arXiv:2206.04615 2022.
25

---

Published in Transactions on Machine Learning Research (07/2025)
Xiao Sun, Naigang Wang, Chia-Yu Yu Chen, Jiamin Ni, Ankur Agrawal, Xiaodong Cui, Swagath Venkatara- mani, Kaoutar El Maghraoui, Vijayalakshmi Viji Srinivasan, and Kailash Gopalakrishnan. Ultra-low precision 4-bit training of deep neural networks. Advances in Neural Information Processing Systems, 33: 1796-1807, 2020.
Gemini Team, Rohan Anil, Scbastian Borgeaud, Yonghui Wu, Ceaan-Baptiste Alayrac, Jiahui Yu, Radu Sori- cut, Johan Schalkwyk, Andrew M Dai, Anja Hauth, el al. Gemini: family of highly capable multimodal models. arXiv preprint arXiv:2312 11t0:2222.11105, 2023.
Hugo Touvron, Thibaut Lavril, Gautier Izacard, Xavier Martinet, Marie-Anne Lachaux, Timothée Lacroix, Baptiste Rozière, Naman Goyal, Eric Hambro, Faisal Azhar, et al. Llama: Open and efficient foundation language models. orXiv preprint urXiv:2302.11 139711 2023a.
Hugo Touvron, Louis Martin, Kevin Stone, Peter Albert, Amjad Almahairi, Yasmine Babaci, Nikolay Bash- lykov, Soumya Batra, Prajiwal Bhargava, Shruti Bhosale, al. Llama Open foundation and fine-tuned chat models. Xiv preprint orXiv:2307.09288 2023b.
Shangqing Tu, Kejian Zhu, Yushi Bai, Zijun Yao, Lei Hou, and Juanzi Li. Dice: Detecting in-distribution contamination I1m's fine-tuning phase for math reasoning. arXw preprint arxiv:240. S5.:2406.041999 04197, 2024.
Zhepeng Wang, Runxue Bao, Yawen Wi,, Jackson Taylor, Cao Xiao, Feng Zheng, Weiwen Jiang, Shangqian Gao, and Yanfu Zhang. Unlocking memorization in large language models with dynamic soft prompting. In Yaser Al-Onaizan, Mohit Bansal, and Yun-Nung Chen (eds.), Proceedings of the 2024 Conference on Empirical Methods Natural Language Processing, FMNLP 2024, Miami, FI., USA, November 12-16, 2024. pp. org/ 2024. mamlp-mainn.5e
Association for Computational Linguistics, 2024. URL httas://aaclanthooogg.
Jason Wei, Xuczhi Wang, Dale Schuurmans, Maarten Bosma, Fci Xia, Ed Chi, Quoc V Le, Denny Zhou, et al. Chain-of- thought prompting elicits reasoning in large language models. Advances in neural information processing systems, 35:24824 -48837, 2022.
Orion Weller, Mare Marone, Nathaniel Weir, Dawn Lawrie, Daniel Khashabi, and Benjamin Van Durme, according to...' prompting language models improves quoting from pre-training data. arXiv preprint arXiv:2305.1322 2023.
Colin White, Samuel Dooley, Manley Roberts, Arka Pal, Ben Feuer, Siddhartha Jain, Ravid Shwartz-Ziv. Neel Jain, Khalid Saifullah, Siddartha Naidu, et al. Livebench: A challenging, contamination free Ilm benchnark. arXiv preprint 0rx::2200.1100 19314, 2024.
Sam Witteveen and Martin Andrews. Paraphrasing with large language models. arXio preprinl arXiv:1911.09661 2019.
Qingyun Wu, Gagan Bansal, Jieyu Zhang, Yiran Wu, Shaokun Zhang, Erkang Zhu, Beibin Li, Li Jiang, Xiaoyun Zhang, and Chi Wang. Autogen: Enabling next-gen Ilm applications via multi-agent conversation framework. aixiv preprint arXiv:2308. 200.033000 2023.
Qianqian Xie, Weiguang Han, Xiao Zhang, Yanzhao Lai, Min Peng, Alejandro Lopez-Lira, and Jimin Huang. Pixiu: A large language model, instruction data and evaluation benchmark for finance. arXiv preprint arXiv:2306. arX4::2306.05443, 2023.
Roy Xie, Junlin Wang, Ruomin Huang, Minxing Zhang, Rong Ge, Jian Pei, Neil Zhenqiang Gong, and Bhuwan Dhingra. Recall: Membership inference via relative conditional log-likelihoods. arXio preprint arXiv:2406.1560 15968, 2024.
Cheng Xu, Shuhao Guan, Derek Greene, M Kechadi, et al. Benchmark data contamination of large language models: A survey. arXiv preprint arXiv:2406. 04244, 2024a.
Ruijie Xu, Zengzhi Wang, Ze Fan, and Pengfei Liu. Benchmarking benchmark leakage in large language models. Xiv preprint orxiv:2404. 18824, 2024b.
26

---

Published in Transactions on Machine Learning Research (07 (025))
Aiyua Yang, Bin Xiao, Bingning Wang, Borong Zhang, Ce Bian, Chao Yin, Chenxu Lv, Da Pan, Dian Wang, Dong Yan, et al. Baichuan 2: Open large-scale language models. arv preprint orXin:3309 10305, 2023a.
Hui Yang, Sifu Yuc, and Yunzhong Hc. Auto-gpt for online decision making: Benchmarks and additional opinions. arXiv preprint ar orXiv2330.0022 2023b.
Shuo Yang, Wei-Lin Chiang, Lianmin Zheng, Joseph E. Gonzalez, and Ion Stoica. Rethinking benchmark and contamination for language models with rephrased samples. arXiv preprint (tfi::331.048850 2023c.
Alex Young, Bei Chen, Chao Li, Chengen Huang, Ge Zhang, Guanwei Zhang, Heng Li, Jiangchong Zhu, Jianqun Chen, Jing Chang, et al. Yi: Open foundation models by 01. al. arXiv preprint (tl:2403.046555 2024.
Tao Yu, Rui Zhang, Kai Yang, Michihiro Yasunaga, Dongxu Wang, Zifan Li, James Ma, Irene Li, Qingning Yao, Shanelle Roman, et al. Spider: A large-scale human-labeled dataset for complex and cross-domain semantic parsing and text-to-sql task. arXic preprint arXi::1809.008: 088887 2018.
Zhuohao Yu, Chang Gao, Wenjin Yao, Yidong Wang, Wei Ye, Jindong Wang, Xing Xie, Yue Zhang, and Shikun Zhang. Kieval: A knowledge-grounded interactive evaluation framework for large language models, 2024.
Rowan Zellers, Ari Holtzman, Yonatan Bisk, Ali Farhadi, and Yejin Choi. Hellaswag: Can a machine really finish your sentence? arXiv preprint XrXiv:1905.074 07830, 2019.
Anqi Zhang and Chaofeng Wu. Adaptive pro-training data detection for large language models via surprising tokens. Xiv preprint or arx:::2477.000 21248, 2024.
Jingyang Zhang, Jingwei Sun, Eric Yeats, Yang Ouyang, Martin Kuo, Jianyi Zhang, Hao Frank Yang, and Hai Li. +in-k%++: Improved bascline for detecting pro-training data from large language models. arXiv preprint ar Xix::2204..0 02936, 2024a.
Liwen Zhang, Weige Cai, Zhaowei Liu, Zhi Yang, Wei Dai, Yujie Liao, Qianru Qin, Yifei Li, Xingyu Liu, Zhiqiang Liu, et al. Fineval: A chinese financial domain knowledge evaluation benchmark for large language models. Xiv preprint arXiv:2308. 20922900.0 2023.
Weichao Zhang, Ruqing Zhang, Jiafeng Guo, Maarten de Rijke, Yixing Fan, and Xueqi Cheng. Pretrain- ing data detection for large language models: A divergence-based calibration method. arXie preprint arXiv:2409. 14 781, 2024b.
Xiang Zhang, Junbo Zhao, and Yann LeCun. Character-level convolutional networks for text classification. Advances in neural information processing systems, 28, 2015.
Yi Zhao, Jing Li, and Linyi Yang. Cap: Data contamination detection via consistency amplification. ar.ivv preprint XrXi::24 15005, 2024.
Lianmin Zheng, Wei-Lin Chiang, Ying Sheng, Siyuan Zhuang, Zhanghao Wn, Yonghao Zhnang, Zi Lin, Zhuohan Li, Dacheng Li, Eric Xing, et al. Judging llm-as-a-judge with mt-bench and chatbot arena. Advances in Newral Information Processing Systems, 36:46595-46623, 2023.
Wanjun Zhong, Ruixiang Cui, Yiduo Guo, Yaobo Liang, Shuai Lu, Yanlin Wang, Amin Saied, Weizhu Chen, and Nan Duan. Agieval: A human-centric benchmark for evaluating foundation models. arXiv preprint arXiw:2304.06:0 2023.
Kun Zhou, Yutao Zhu, Zhipeng Chen, Wentong Chen, Wayne Xin Zhao, Xu Chen, Yankai Lin, Ji-Rong Wen, and Jiawei Han. Don't 't make your Ilm an evaluation benchmark cheater. ar preprint orXiv:2311.0100 2023.
27

---

Published in Transactions on Machine Learning Research (07 (07/2025)
Qin Zhu, Qingyuan Cheng, Runyu Peng, Xiaonan Li, Tengxiao Liu, Ru Peng, Xipeng Qiu, and Xuanjing Huang. Inference-time decontamination: Reusing leaked benchmarks for large language model evaluation. arXiv preprint ar.ie:2406. 13990, 2024.
Wenhong Zhu, Hongkun Hao, Zhiwei He, Yunze Song, Yumeng Zhang, Hanxu Hu, Yiran Wei, Rui Wang, and Hongyuan Lu. Clean-oval: Clean ovaluation contaminated large language models. or preprint arXiv:231.001164 2023.
28


Reviews:
[{"review_id": "f4qo1vzZ94", "reviewer": "Reviewer_fdZF", "summary": "", "questions": "", "limitations": "", "rating": null, "confidence": null, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "**Strengths**\n1. This survey paper demonstrates timeliness by including the most recent works. The proposed taxonomy based on data accessibility provides clear coarse-grained categorization.\n2. The authors provide insights that go beyond a mere systematic summary of existing works. The discussion on contamination mitigation strategies and detection challenges is valuable for the research community.\n\n**Weaknesses**\n1. Based on my literature research, several relevant survey works on data contamination appear missing from the discussion: [1] (accepted in 2024), [2] (2025 preprint). Furthermore, the designation of Xu et al. (2024a) [3] as “concurrent work” raises questions given its publication date. While I am not familiar with the review iterations of this paper in TMLR, the timing of this version suggests that classifying [3] as “concurrent work” may be inappropriate.\n\n2. The literature search and selection methodology lacks documentation. Critical details about selection criteria, sources, and search strategies remain unspecified, which are essential components for a survey paper.\n\n3. Although the introduction provides background about data contamination , the paper would benefit from an extra section formally defining the contamination detection task and establishing quantitative evaluation metrics prior to discussing current techniques. \n4. Presentation issues require attention:\n\n- Page 8: Hyperlink misalignment. Current hyperlink of literature incorporates the next page header.\n\n- Page 9: Inconsistent notation (italicized $K$ vs. regular K) in Min-K detection methodology.\n\n---\n\n**References**\n- [1] Deng, C., Zhao, Y., Heng, Y., Li, Y., Cao, J., Tang, X., & Cohan, A. (2024). Unveiling the Spectrum of Data Contamination in Language Model: A Survey from Detection to Remediation. In Findings of the Association for Computational Linguistics ACL 2024 (pp. 16078-16092).\n- [2] Cheng, Y., Chang, Y., & Wu, Y. (2025). A Survey on Data Contamination for Large Language Models. arXiv preprint arXiv:2502.14425.\n- [3] Xu, C., Guan, S., Greene, D., & Kechadi, M. (2024). Benchmark data contamination of large language models: A survey. arXiv preprint arXiv:2406.04244.", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "No", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "paper_topic_and_main_contributions": "", "excitement": null, "reproducibility": null, "ethical_concerns": "", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "summary_of_contributions": "Data contamination refers to the phenomenon where improper data (i.e., test data leakage or unethical data) has been mixed into model training data. This survey defines contamination as “any leakage of the evaluation data within the training data”, and conducts a systematic literature review on current data contamination detection techniques. The authors organize existing works through a taxonomy distinguishing between open-data scenarios (where model pre-training data is known) and closed-data scenarios (where pre-training data remains unknown). The review encompasses over 100 papers up to early 2025. Additionally, the authors provide practical recommendations for dataset construction and release to prevent contamination, along with discussions of future challenges.", "claims_and_evidence_yn": "No", "claims_explanation": "", "audience_yn": "Yes", "audience_explanation": "", "requested_changes": "1. Please consider supplementing discussions of missing survey works ([1], [2]) with analysis of their distinct contributions.\n2. A detailed methodology section documenting literature search protocols and selection criteria should be added. \n3. I recommend inserting a formal task definition section after the introduction part, specifying mathematical formulations and evaluation frameworks for contamination detection. This would enhance conceptual clarity for readers.\n4. Technical presentation issues (hyperlink errors, typographical consistency in mathematical notation) require correction.", "broader_impact_concerns": "Since this is a survey paper, there may not be any ethical concerns to address.\nHowever, the authors could include a Broader Impact section discussing the positive impact that the contamination detection techniques may have.", "additional_comments": "", "extra_scores": {}, "extra_text": {}}, {"review_id": "6ZSb6n5lVi", "reviewer": "Reviewer_VDsX", "summary": "", "questions": "", "limitations": "", "rating": null, "confidence": null, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "Strength: \nThe paper summarizes the contamination detection methods for open-data contamination detection and closed-data contamination detection. The coverage seems comprehensive. \n\nWeakness: \n1. The section regarding Quantifying Memorization seems to lack additional details. For example, several papers regarding discoverable memorization are not discussed [1, 2, 3].\n2. In the section of best practices to avoid contamination, the authors did not discuss evaluation settings like chatbot arena [4]. This kind of evaluation will no doubt avoid the problem of dataset contamination. It may also have some weakness, for example the model can be specifically tuned to align the preference of humans to boost its scores. The authors should discuss this approach or similar approaches in the paper.\n\n[1] Quantifying Memorization Across Neural Language Models, ICLR 2023.\n\n[2] Controlling the extraction of memorized data from large language models via prompt-tuning, ACL 2023.\n\n[3] Unlocking Memorization in Large Language Models with Dynamic Soft Prompting, EMNLP 2024.\n\n[4] Chatbot Arena: An Open Platform for Evaluating LLMs by Human Preference", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "Yes", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "paper_topic_and_main_contributions": "", "excitement": null, "reproducibility": null, "ethical_concerns": "", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "summary_of_contributions": "The manuscript is a survey, and it covers contamination detection methods for two settings Open-Data Contamination Detection and Closed-Data Contamination Detection.", "claims_and_evidence_yn": "Yes", "claims_explanation": "", "audience_yn": "Yes", "audience_explanation": "", "requested_changes": "Incorporating the discussion from the weaknesses section would strengthen the paper and reinforce my recommendation for acceptance.", "broader_impact_concerns": "I think the authors should have a stand alone broader impact section by including discussions from the section of \"Ethical and legal data frameworks.\"", "additional_comments": "", "extra_scores": {}, "extra_text": {}}, {"review_id": "LDHvSPv6fz", "reviewer": "Reviewer_67L4", "summary": "", "questions": "", "limitations": "", "rating": null, "confidence": null, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "Strengths\n- The authors provide a comprehensive overview covering over 100 papers in this topic. They also give a structured organization for data contamination. This would help researchers to get into this area.\n- The topic selected, data contamination, is of great importance for the healthy development of LLMs\n\nWeaknesses\n- Definition of contamination and other terms\n\t- What is the formal definition of data contamination? Are there any established definitions in the context of LLMs or other fields? If not, can the authors discuss this issue, especially data contamination.\n\t- As for the proposed framework, there also lacks clear definition and proper demonstration on some lines of work. For instance, what is membership inference attack and how is it different from model memorization methods? How MIA is used in data contamination detection? Such basic questions require more demonstration.\n- As an overall survey of this field, it is important to provide an all-round look at the current status, which is missing in the manuscript. Specifically, what is the cause of data contamination? What are the potential harms and broader impact of contamination? Are there any reports or well-known conclusions about contamination of LLMs? Are there any real-world applications or services for data contamination detection? \n- Despite the overview of contamination detection of LLMs, the discussion on academic and industrial efforts on how to avoid data contamination. I feel the current version could be further improved. For example, the Dynamic Evaluation section mostly involves MIA benchmarks only, which have limited connections with data contamination. There are more dynamic benchmarks than LiveCodeBench, such as Chatbot Arena, LiveBench. Moreover, some benchmarks adopt perturbation-based manipulation of the original dataset to handle potential contamination, such as GSM-Plus, Math-Perturb etc. These works should also be included and discussed.\n- Some discussions feel lack of depth and incomplete. \n\t- For the different kinds of data contamination detection, are they suitable for all kinds of contamination? What are the specific settings of contamination detection tasks? Are there any differences between contamination in pre-training, finetuning, and RLHF?\n\t- In Section 4.2, the Open-Source Pre-training Data paragraph, the authors mainly introduced three pre-training datasets, but did not go deep on the main contributions and differences of the datasets. How do the datasets benefit the data contamination research? What are their main conclusions? It seems that they are MIA datasets and the relevance is not well-elaborated.\n\nReferences\n\n[1] Chatbot Arena: An Open Platform for Evaluating LLMs by Human Preference\n\n[2] LiveBench: A Challenging, Contamination-Limited LLM Benchmark\n\n[3] Gsm-plus: A comprehensive benchmark for evaluating the robustness of llms as mathematical problem solvers\n\n[4] MATH-Perturb: Benchmarking LLMs' Math Reasoning Abilities against Hard Perturbations", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "No", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "paper_topic_and_main_contributions": "", "excitement": null, "reproducibility": null, "ethical_concerns": "", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "summary_of_contributions": "This survey focuses on the data contamination issue of LLMs. It comprehensively reviewed the contamination detection methods for both open-data and closed-data settings. The work also discussed some practices on how to avoid data contamination and build contamination-free benchmarks.", "claims_and_evidence_yn": "No", "claims_explanation": "", "audience_yn": "Yes", "audience_explanation": "", "requested_changes": "Please see above weaknesses and questions. Overall, more appropriate organization and deeper analysis are required.\n\ntypos\n- page 2, \"enabling enabling\"", "broader_impact_concerns": "Data contamination and de-contamination is recognized as important for reliable LLM evaluation. Thus there are considerable social impacts and broader concerns about this issue. It would be nice if the authors provided additional discussions about this aspect.", "additional_comments": "", "extra_scores": {}, "extra_text": {}}]

Author rebuttal:

Decision: Accept (poster)