Unlocking Post-hoc Dataset Inference with Synthetic Data
Authors: Bihe Zhao
Pratyush Maini
Franziska Boenisch
Adam Dziedzic
Venue: ICML 2025 poster (2025)
Keywords: large language models
dataset inference
synthetic data
Primary area: deep_learning->large_language_models
TL;DR: Our method enables LLM dataset inference without any i.i.d. held-out set by generating this set and performing post-hoc calibration.

Abstract: The remarkable capabilities of Large Language Models (LLMs) can be mainly attributed to their massive training datasets, which are often scraped from the internet without respecting data owners’ intellectual property rights. Dataset Inference (DI) offers a potential remedy by identifying whether a suspect dataset was used in training, thereby enabling data owners to verify unauthorized use. However, existing DI methods require a private set—known to be absent from training—that closely matches the compromised dataset’s distribution. Such in-distribution, held-out data is rarely available in practice, severely limiting the applicability of DI. In this work, we address this challenge by synthetically generating the required held-out set. Our approach tackles two key obstacles: (1) creating high-quality, diverse synthetic data that accurately reflects the original distribution, which we achieve via a data generator trained on a carefully designed suffix-based completion task, and (2) bridging likelihood gaps between real and synthetic data, which is realized through post-hoc calibration. Extensive experiments on diverse text datasets show that using our generated data as a held-out set enables DI to detect the original training sets with high confidence, while maintaining a low false positive rate. This result empowers copyright owners to make legitimate claims on data usage and demonstrates our method’s reliability for real-world litigations. Our code is available at https://github.com/sprintml/PostHocDatasetInference.

Unlocking Post-hoc Dataset Inference with Synthetic Data
Bihe Zhao Pratyush Maini 23 Franziska Boenisch
Adam Dziedzic'
Abstract
1. Introduction
Large language models (LLMs) have recently achieved remarkable success in broad range of tasks, fueled by the availability of massive high-quality text corpora often scraped from the internet (Weber et al., 2024; Penedo et al., 2024). While this practice enables LLMs to generate high- quality text and to excel on benchmarks, it also raises seri- ous concerns related to intellectual property rights (Reuters, 2023; Gry. 2023; Sil, 2023), data privacy (Duan et al., 2023a;b; Hanke al., 2024; Hayes et al., 2025), and trans- parency (Rahman & Santacana, 2023; Wu et al., 2023). The reliance potentially unauthorized data creates an urgent need for methods that allow independent authors to verify whether given dataset has been used to train an LLM with- out the explicit consent of the model provider.
The remarkable capabilities of Large Language Models (LLMs) can be mainly attributed to their massive training datasets, which are often scraped from the internet without respecting data owners' intellectual property rights. Dataset Inference (DI) offers a potential remedy by identifying whether suspect dataset was used in training, thereby enabling data owners verify unauthorized use. However, existing DI methods require a private set-known to be absent from training- that closely matches the compromised dataset's dis- tribution. Such in-distribution, held-out data is rarely available in practice, severely limiting the applicability of DI. In this work, we address this challenge by synthetically generating the required held-out set. Our approach tackles two key obsta- cles: (I) creating high-quality, diverse synthetic data that accurately reflects the original distribu- tion, which we achieve via a data generator trained on a carefully designed suffix-based completion task, and (2) bridging likelihood gaps between real and synthetic data, which is realized through post-hoc calibration. Extensive experiments on diverse text datasets show that using gener- ated data as held-out set enables DI to detect the original training sets with high confidence, while maintaining a low false positive rate. This result empowers copyright owners to make legiti- mate claims on data usage and demonstrates our method's is reliability for real-world litigations. Our com/s s
A promising approach to addressing these concerns is dataset inference (DI) (Maini et al., 2021: Dziedzic et al., 2022a;b; Maini et al., 2024; Dubiáski al., 2025; Kowal- czuk et al., 2025), which aims to determine whether sus- pect dataset has contributed to a model's training. This puts power in the hands of data owners to monitor and exer- cise their intellectual property rights. Despite its potential, DI currently faces a critical bottleneck: it requires a held- out set-a a dataset known to absent from training that shares the same distribution the suspect dataset (Zhang et al., 2024a). In practice, however, such in inddistribution held-out set is rarely available. Data creators do not typi- cally reserve a dedicated held-out set for legal or auditing purposes, and any disclosed held-out data could itself be re- purposed for future training, further complicating the verifi- cation process. Moreover, even when dataset owner can provide some held-out samples, any slight distributional dis- crepancy from the original suspect data can undermine DI by inflating false positives (Das et al., 2024; Duan et al., 2024; Meeus et al., 2024; Maini & Suri, 2024).
code is available at htttt::ttt/ttttttt Iww.araaaaaatteeteeeeeeeeeee
To illustrate the brittleness of using seemingly IID (Indepen- dent and Identically Distributed) held-out data, we demon- strate in Section 3 that even in simple scenario-where an LLM is fine-tuned on blog posts from single author-there exists distributional shift between training data (members) and randomly held-out blog posts from the same author. This highlights how even subtle variations in held-out data can undermine DI. Malicious actors may exploit this vul-
CCIPAA Helmholtz Center for Information Security "Carnegie Mellon University 'DaLologyAL. Correspondence to: Bihe Zhao <bihe. @cispa.de>. Pratyush Maini <pratyush- maini www.emuu...ooo..o edu>, Franziska Boenisch <boenisch Sesenisch@eiispal @eiichhhiiippd..... Adam Dziedzic <adam.dziedzic@ectn cis.a.de>.
Proceedings of the 42nd International Conference Machine Learning, Vancouver, Canada. PMLR 267, 2025. Copyright 2025 by the author(s).

---

Unlocking Post-hoc Dataset Inference with Synthetic Data
Our Work
E
D0O
(a) Authors publish their articles on the internet
(b) LLM providers illegally scrape data from the web
(c) LLM providers train model on this data & deploy it
(d) Authors approach arbiter with copyright claim
(e) Arbiter generates IDD held-out synthetic data like (a)
(f) Arbiter calibrates
(g) Arbiter performs
likelihoods synthetic & real data
Dataset
Inference for
ownership
Figure 1. Dataset Inference Procedure with Synthetic Held-Out Data. This figure presents a high-level overview of how the proposed dataset inference (DI) process will take place in real-world cases. (a-d) LLM providers scrape proprietary author data from the internet, and train an LLM on it. Authors who suspect unauthorized use may approach an arbiter with a copyright claim. To resolve such a dispute, the arbiter must perform dataset inference (g). However, this requires the presence of held-out datasct that IID to the suspect set. Our Work: (e) The arbiter generates IID symthetic held-out data that mimics the author's 's original data. (I) The arbiter calibrates likelihoods between real and synthetic data to ensure fair comparison, enabling them reliably perform dataset inference.
nerability by strategically introducing shifted held-out data, falsely accusing model owners of copyright infringement and further reducing the reliability of DI methods.
which incorporates both the textual features and DI's stan- dard membership indicators (e.g., perplexity, min-k proba- bilities). The key insight is that any performance advantage of the membership-aware classifier over the text-only clas- sifier must arise from the presence of membership signals rather than distributional artifacts. This difference serves our DI signal, allowing to infer whether the suspect dataset was used in the target model's training. This cali- bration strategy enhances DI's robustness, reducing false positives while maintaining high detection accuracy.
In this work, we address these challenges by proposing to synthetically generate held-out data for DI, bypassing the need for in-distribution held-out data. This vision, however, is non-trivial to achieve. First, the generated texts must be re- alistic, high-quality, and sufficiently diverse to approximate the distribution of the original data. Second, the generation process itself may introduce a distribution shift between nat- ural and synthetic held-out data. Such a shift complicates DI: if a difference is observed between the suspect and held- out sets, it becomes unclear whether this difference arises from a genuine membership signal (i.e., the target model be- haves differently on the suspect data because it has seen it during training) or merely from the distribution shift (i.e., the model behaves differently on suspect data because it is natural data). Recent studies have extensively highlighted this issue in the context of Membership Inference Attacks (MIAs) (Shokri al., 2017), where distribution shifts lead to misleading evaluation results (Das et al., 2024; Zhang et al., 2024a; Maini al., 2024; Dubiński et al., 2025).
We demonstrate the effectiveness of our approach on diverse textual datasets, ranging from single-author datasets (e.g.. personal blog posts) large-scale, multi- aulti- collections such as Wikipedia. Our results show that using synthetic held-out data, combined with calibration, enables DI to detect unauthorized training data use with high confidence while keeping false positives low. This expands the practical applicability of DI and provides a pathway for data owners to safeguard their intellectual property in an era of L.Mss.
2. Background and Related Work
2.1. Membership Inference
To this end, we first train a carefully designed text genera- tor on the suspect dataset itself, on a suffix completion task (Section 4.1.. This approach produces high-qualiny datasets with only a small distributional shift from the suspect texts. However, even small shifts in distribution can undermine DI's reliability. To address this, we introduce a post-hoc calibration step (Section 4.2) to ensure that the generated held-out set can serve as a reliable reference for DI. Specifi- cally, we disentangle the effects of distributional shifts from the actual membership signal-a critical factor in DI. To achieve this, we propose a dual-classifier approach: (1) A text-only classifier, trained to distinguish natural (original) from generated data. (2) A membership-aware classifier,
MIAs focus on deciding if a single data point was included in a given model's training dataset and often serve as fea- tures extractors for DI. In the LLM domain, MIAs exploit different signals to distinguish between members (training data points) and non memmeerrrs (data points not used dur- ing training). For instance, LOSS exploits the perplexity or loss function of the target model (Yeom et al., 2018). Shi et al. (2024) find that the rare words in a sequence can leak more privacy information, and select K% tokens with the smallest probabilities for evaluation. Min-K K%+++ further improves upon the Min-K% approach by introducing two calibration factors (Zhang et al., 2024b). Zlib ratio (Carlini
2

---

Unlocking Post-hoc Dataset Inference with Synthetic Data
et al., 2021) uses the compression rate of z-library to nor- malize the perplexity of the target model. Neighborhood- based methods compare suspect sequence with its neigh- boring texts, which can be produced by synonym substitu- tion (Mattern et al., 2023) or paraphrasing (Duarte et al., 2024). Moreover, reference-based methods compare the out- put signals on suspect sample between the target model and a reference model (Fu et al.. 2024). Yet, many recent works have shown that the evaluation of MIAs suffers from falsified experimental setup, where distributional shift exists between the member and non-member sets (Zhang et al., 2024a; Maini et al., 2024; Das et al., 2024). Duan et al. (2024) show that most MIAs only perform slightly better than random guessing if evaluated correctly on non- biased benchmarks. Recently, Kazmi al. (2024) proposed how to de-bias MIAs from this distribution shift we use as foundation for our DI calibration.
Table I. The distributional shift (GPTZ AUC) and DI p-value between suspect set that consists of non-members and held- out blog posts. Here, p-value < 0.05 indicates DI incorrectly suggests that the suspect set is member set.
Sequences per Blog
15
20
25
GPT2 AUC (%))
52.0
55.2
53.2
58.2
58.6
DI p-valuc
0.002
<0.001
<0.001
<0.001 X
<0.001
True Membership
X
Inferred Membership
combine and weight the extracted features. We follow this approach in our evaluations. LLM DI can be formalized as follows. First, after calculating over n MIA scores with linear regression, an aggregated MIA score is obtained by W MIA(x) = Ei=1 w,MIA, (z). Here, W = -[WI,... Wn] is the weight of the linear regressor, and MIA (2) is a vec- tor concatenating n MIA scores. We label the suspect data as 0 and the held-out data as 1. Note that, MIA is calcu- lated based on (xx), but we omit ʃ for simplicity. Then, a hypothesis testing is conducted to verify if the held-out set has higher MIA score than the suspect set statistically. The null hypothesis can be formalized as follows.
2.2. Dataset Inference
To strengthen the signal from training data further beyond MIAs, Maini et al. (2021) introduced DI. DI aggregates the membership signal over multiple data points, often referred to as suspect set, to decide whether a given model was trained on this data. More formally, given a target model f. DI aims to detect whether f was trained on the suspect dataset Daus- Therefore, it needs an additional held-out dataset Daa from the same distribution as Daus- Given both sets, DI extracts membership features from the data points in DAnss and Dvat. aggregates all features per given sample, and then scores these aggregate features through a scoring model. The scores should be lower for members than for non memmees Then, DI performs statistical hypothesis testing on the scores of Daus and D The null hypothesis is that the average scores for Dau are higher than or equal to the scores for Deat- If the statistical test manages to reject this null hypothesis, this is a confident indicator that the data points from D,,s are indeed members of model f's training data. Otherwise, the test is considered inconclusive.
Ho ED., [W MIA(x.a)) S ED...W (Z (1)
If the suspect set is part of the training set of f, the null hypothesis is rejected.
3. Failure Cases of DI
In this section, we dive deeper into the difficulties that arise from DI's assumption on the availability of an additional in- distribution held-out dataset. More precisely, we show that this assumption is extremely hard to meet in practice, even in the simplest setups-limiting the applicability of stan- dard DI. Therefore, we collect blog posts written by sin- gle author on topics from the same domain and split them randomly into a training and held-out set. We finetune an I.IM on the training set, perform DI (Maini et al., 2024), and find that the method returns false positives, i.e., it ille- gitimately claims that the model was trained on blog posts that actually was not trained on (see Table 1). Our analy- sis highlights that despite the texts' homogeneity, there is a small distributional shift between the suspect and held-out sets that is not even easily distinguishable by Blind Base- lines (Das et al., 2024), which causes DI to fail. This high- lights the need to generate synthetic held-out data to benefit from DI in real-world copyright claims. We provide more details below and discuss its implications.
How to extract the best membership features from the data points varies based on the learning paradigm. For example, the original DI for supervised classification models (Maini et al., 2021) designs random walk strategy to estimate the distance between data points and the decision bound- ary of a supervised model. For self-supervised models, Dziedzic et al. (2022b) use Gaussian Mixture Model to esti- mate the representational differences between the training dataset (members) and the test data. Recent dataset infer- ence methods for LLMs (Maini et al., 2024), Diffusion Mod- els (Dubińísk et al., 2025), and Image Autoregressive Mod- els (Kowalczuk et al., 2025) build on existing membership inference attacks (MIAs) tailored to each type of generative features
3.1. DI on a Single Author's Data
We consider a practical application of DI in copyright pro- tection as detailed in Figure 1. In this scenario, an au- thor has some published texts on the internet of which
model. These methods extract membership-relate using the appropriate MIA and then apply a linear model to
3

---

Unlocking Post-hoc Dataset Inference with Synthetic Data
they believe that they were illegitimately used by an LLM provider to train their model. The author provides these published works to arbitrator, non non-published blog-poos posts as held-out set from the same dis- tribution, i.e, with the same style, topics, etc. Then, the ar- bitrator performs DI to resolve the copyright claims.
Neuber
0.00
Novan (Memberr
Meen meeemmemerr
Noun (ffet--rt)
0.06
Meun (tbild-ortt
suspect set and some
0.04
0.01
0.02
0.00
0.00
Perplasily
Perpleaity
To evaluate this setup in practice, we collect blog posts of public blogger. The blogs are split into member, non- member, and held-out sets. To avoid any potential tempo- ral or topic distributional shifts, we randomly shuffle all the collected blogs before splitting. In lack of the compu- tational capacities to train an LLM from scratch, we fine- tune a Pythia model (Biderman et al., 2023) on the member set. The Pythia model is trained on the Pile dataset (Gao et al., 2020), so we only used blogs after the release date of the Pile to ensure that none of the blogs is part of the pre- training data. Also, we only finetune the target model on the member set for one epoch. This is to evaluate the per- formance of DI and our method in the most strict scenario, as Duan et al. (2024) show that MIAs perform better with more training epochs. Finally, we run DI. More detailed ex- periment configurations can be found in Section 5.1.
Figure 2. Probability Distribution Function (PDF) of target model perplexities. We show the comparison between (a) the member and held-out, and (b) the non non-member and held-out sets.
behind this observation is that each blog has different con- tent and topics, which brings different words across the non- member and held-out documents. The gap is enlarged when we sample more sequences from each blog post. Conse- quently, this distributional shift in texts also lead to shift in the MIA score. As presented in Figure 2, the distributional shift in perplexities exists not only between member and held-out sets, but also between non-member and held-out sets. This shows that the inherent distributional shift among documents is entangled with the shift caused by member- ship signals in the MIA score, and makes DI fail to deter- mine membership by simply detecting any distributional shift in the MIA score. This observation aligns with the p- values and predictions in Table 1, where we find that even this small distributional shift causes sigmificant false posi- tive rates during DI. This means that the DI falsely accuses the LLM provider of violating the copyright of an author. What is more is that this shortcoming of DI can be mali- ciously exploited: authors could deliberately provide held- out data from different distribution than their suspect data to mislead DI and illegitimately accuse the LLM provider. As a solution to this problem, in the next section, we pro- pose our approach on generating an adequate in-distribution held-out dataset synthetically.
3.2. Metrics of Distributional Gap
Before analyzing the results, we introduce the metrics we use to quantify the distributional shift between the suspect and held-out sets. Following the approach of Blind Base- lines (Das et al., 2024), we formulate the measurement of the distribution gap between two text datasets as classifica- tion problem. In particular, the suspect set Daus is randomly split into a classifier training split sus and a test split D test sus The held-out set Dval is also split into D train and Dist test in the same vein. Then, a classifier g is optimized to distinguish the training splits Dirain and DD train Finally, we calculate the area under the curve (AUC) score of the classifier the test splits 'D test and 'D Dest which is used to measure the dis- tributional gap between Daus and Dval.
4. Synthesizing Held-out Data
Our approach consists of two subsequent steps. First, we generate high-quality held-out data, then, we perform a calibration to account for the distribution shift that such generation can introduce.
The design of the classifier decides how the texts are vec- torized and if the discrepancies between texts can be suf- ficiently captured. Das al. (2024) apply a bag-of-worde (BoW) classifier, which can only detect the differences in terms of word frequency. Instead, we build a GPT2-based classifier with two transformer blocks to also find the dif- ferences in grammar, content, styles, etc. between two text distributions. We train the classifier from scratch to avoid the impact of any pre-training data. Using only two trans- former blocks of the GPT2 architecture avoids overfitting.
4.1. Held-out Data Generation
We explore three approaches that leverage LLMs for gen- erating held-out data based on provided suspect data with minimal distribution shift.
In-context Learning. As naive approach, we use GPT-4 models to paraphrase the suspect set with in-context learn- ing (ICL) and evaluate the distributional shift between the original suspect texts and the paraphrased texts. Specifically, each prompt includes a few data points as demonstrations (shots) and requests the model to produce paraphrases for
3.3. False Positive of DI
The AUC scores of the GPT2-based classifier in Table show that there is neglligiee distributional shift be- tween the non-member and the held-out sets. The intuition
4

---

Unlocking Post-hoc Dataset Inference with Synthetic Data
Held-out Data Generation
Post-hoe Calibration
Suspect Train Set
Suspect
Set
Suspect Test Set
① Segment
① Split
Held-out
Held-out Train
Set
Ileld-out Test Set
Provided Ducuments
2
Train
Text Classifier
Shuffle Split
Suspect
Target Model f()
Held-out
Altallon sight
Train Set
Train Sct
MIA score MIA(F(3))
Generator Training Split
Generator Inference Split
Combined Classifier Goon MIA(((H))
Evaluation
Pred
Train
Elleiing Wnliiiiin deformii shal
Generator
Text Classifier Gxve(()
Suspect
Target Model /())
Held-out
Suspect
Test Set
Test Set
Set
MIA score MIA(f(+))
Compare
Generator
Held-out
Set
Combined Classifier Soorea (r. MIA(F(+))
Inference
Predcomb
Figure 3. Held-out Data Generation (Left Panel): (1) The suspect dataset is first segmented into snippets. (2) These snippets are shuffled and split into a generator training set and an inference set. (3) A generator model is trained on the suspect dataset using a suffix completion task. The trained generator produces synthetic held-out data that mimics the suspect set. Post-hoc Calibration (Right Panel): (1) The suspect set and synthetic held-out set are split into training and test subsets. (2) A text classifier is trained to differentiate from syuthetic text. combined classifier integrates textual features with DI membership signals from the target LLM. (3) The two classifiers are evaluated: the combined classifier should only outperform the text classifier if the suspect dataset was used for training, (4) statistical comparison ensures that any signal detected due to actual membership rather than distributional shifts.
another data point in the suspect set. We test two types of prompts: 1) Paraphrasing, where we directly prompt the model to paraphrase a chosen data point, and 2) Text Com- pletion, where we prompt the model to complete trun- cated data point. For text completion, the original suffix is used as the suspect data, and the generated suffix the held-out data. Our results in Table 2 show that there is significant distribution shift between the original and para- phrased samples for both types of prompts. Even a BoW classifier obtains a significant AUC of 76.2% when distin- guishing between the original suspect vs paraphrased text and the GPT2 classifier can achieve 99.0% AUC. The rea- son is that there many words (such "remarkable" and "moreover") that appear much more frequently in the syn- thetic text than in the Human-written text. Please refer to Appendix A for more detailed explanation of GPT-4-based generation and examples of generated texts.
Table 2. Distributional shifts between the suspect set and syn- thetic held-out set. The shifts are measured by Bag-of Word (BoW) and GPT2 classifiers.
Generation Method
BoW AUC (%)
GPT2 AUC (%)
Paraphrasing ICI. Text Completion
79.2
99.2
Preference Optimization
Suffix Completion
50,0
52.2
BoW classifier is similar random guessing, which means frequent words can be greatly reduced in generated texts by this approach. However, the GPT2 classifier can still obtain an AUC of 58.9%. This shows that preference optimization still leaves distinguishable generation patterns that could be easily captured by transformer-ban based classifier, limiting the data's usefulness in-distribution held-out set.
Suffix Completion. The failure of the above methods demonstrates the difficulty of producing high-quality held- out data with small enough distributional gap to the sus- pect data. To solve this problem, we design a generator train- ing scheme that enables the generator to derive suspect set from the author's 's provided documents, together with a held-
Preference Optimization. We also adapt preference opti- mization methods (Rafailov et al., 2024; Xu et al., 2024) to the task of held-out data generation by changing from hu- man preference to natural text preference. A more detailed explanation is presented in Appendix B. The AUC of the
5

---

Unlocking Post-hoc Dataset Inference with Synthetic Data
out from the same distribution as this suspect set. As shown in Figure 3, we ! first segment the provided docu- ments into multiple short sequences.
x and the MIA signal (1A(2) based on the outputs of J. Here, MIA(x:) can also be a vector by concatenating multi- ple MIA scores. We split both the suspect set and held-out sets into training and test splits. The two classifiers are opti- mized on the suspect train split Desia and the held-out train split D train and evaluated on the suspect test split D lest and the held-out test split test val By comparing the distributional shifts quantified by the MIA classifier and the shifts identi- fied by the text classifier, we can separate the membership signals from the distribution gap caused by generation.
2 All the sequences are shuffled and randomly split into a generator training split and a generator inference split. Then, (3) low-rank adap- tion (LoRA) generator is finetuned on the training split with the cross-entropy loss for next-token prediction. Finally, (4) we segment each sequence in the generator inference split into two parts, and the generator predicts a synthetic suffix based on the prefix. Here, the original suffixes are used as the suspect set, and the synthetic suffixes as the held-out set. Note that, the training and inference sets are split on the shuf- fled text sequences rather than on the documents. This is to ensure that the text snippets from the generator training and inference splits are from the same distribution, such that the generator can achieve better generalization from the training to the inference set. Furthermore, we design a suffix comple- tion task for generator inference. In this task, both the orig- inal suffix and the synthetic suffix share a common prefix. This approach ensures that the synthetic text maintains the same position within a sentence as its original counterpart, making the two suffixes directly comparable. Another im- portant insight is that the generator can produce suffixes of higher quality when the sequence length is relatively short. Therefore, we limit the length of the sequences to no longer than 64 tokens for a smaller distributional gap. The results in Table 2 show that our method achieves a significantly small distributional shift, and even GPT2-based classifier can only achieve an AUC as low 52.2%. For examples of our generative approach, please refer to Appendix D.2.
sus
We design hypothesis test to statistically verify if the com- bined classifier quantifies a larger distributional shift be- tween the suspect and held-out data than the text classifier, namely the difference comparison t-test. The t-test is con- ducted on the test splits D Dies and D lest but we abbreviate them Daus and Dval for simplicity. During the t-test, we first sample a suspect data point Taus E 'Daus and pair it with its corresponding generated counterpart Tival 'Daal- Note that, Usus and Tval are original and generated suffixes, which are both continuations of a common prefix. For every such original/held-ou pair, we quantify the shift caused by generation with the text classifier as Gex( - Cuei (Fiss)) We also quantify the combined effects caused by generu- tion and membership signal with the combined classifier as Ceomb ( vaa - Ccomb ( In particular, Tval and Tsus are la- beled as 1 and O respectively. c(x) denotes the predicted probability of classifier c on data point x, which ranges be- tween 0 and 1. If the membership signal is present, the combined effects will be stronger than the generation ef- fect alone, and the predicted probability will be slightly more accurate for the combined classifier, i.e. Ceomb (Iiaa) Coomb (comb( > Gea (Xval) - Grer (Jas)) To this end, we for- malize the following null hypothesis for our t-test:
4.2. Post-hoc Calibration
Since the generation itself can introduce a distributional shift (natural vs generated) data, DI might yield false positives. This is because it would detect differences between suspect and held-out data also when they only differ in terms of distribution but not necessarily in membership. Therefore, we need to identify and mitigate this distribution shift.
(Coomb Ho F (Commo (Irat) - Coomt (Foos)))) 5
(2)
Een ED.. [Cust (Ival) - Cwxx (Fxus )].
The difference comparison t-test is performed multiple times with different random seeds, and the p- p-values are aggregated with Sidac correction (Šidák, 1967).
To do so, we rely on an important observation: the genera- tion shift between natural and synthesized data occurs in the textual space, while the shift caused by the potential mem- bership of the suspect set exists in the target LLM's 's output space. This allows us to disentangle the two signals. By relying on our GPT-based text-classifier from Section 3.2, we can quantify the textual distribution shift caused by the generation. We denote this classifier by (x)((z), where x is the text input for which the classifier should decide if it is original or generated data. Inspired by Kazmi et al. (2024), we also define second MIA-classifier with input signals from both the texts and the outputs of the target model, such that we can quantify the combined effects of generation and the membership signal. Concretely, we train a combined classifier Goumb (z, (z)(f())) with inputs from both text
By introducing a dual-classifier approach along with a statis- tical test, we can statistically distinguish distributional shifts caused by actual membership signals from those caused by generation. Further results in Section 5.4 show that this ap- proach can prevent false positives in dataset inference effec- tively.
4.3. Weight Constraint
In this section, we explain why and how we apply weight constraint when computing the importance of different MIA scores. In the original DI, the aggregated MIA score is compared between the held-out and the suspect sets. We define the difference in aggregated MIA score between the
6

---

Unlocking Post-hoc Dataset Inference with Synthetic Data
Text
Data
Target Model f(x
Table 3. Results single author blog posts. Here, p-value 0,05 indicates the suspect set is member set.
Combined Classifier
True Membership
AUC leat (%)
AUCCamb (%)
P-value
Inferred Membership
CPT2 based Classifier
score MIA(f (x))
53.8
55.6
0.01
V
53.8
53.9
0.13
X
text score
MIA, (f()))
(IAx(F(O)
MIA weights
empirical analysis of the weight constraint in Section 5.4 and an example for the weight constraint in Appendix G.
Wezzt
(Werre)
J:R. (0.1)
Wi
5. Experimental Evaluation
(Wgy
We start by introducing our experimental setup, further de- tailed in Appendix D. Then, we present the results of DI executed based on our generated held-out data. We also per- form ablation studies to investigate the contribution of each component in our proposed method. Finally, analyze the impact of t-test sample size and the classifier architecture.
af (M,))
crmb
Figure Weight Constraint. The weights for the MIA and text
scores are constrained to (0,1) with Sigmoid function.
5.1. Experimental Setup
two sets Yaur as follows:
Single author data. We collect 1400 blog posts from single author. All figures, tables, vidcos, and hyperlinks are removed during pre-pron pre-processing and only plain text is used for evaluation. We sample 450 posts as member data and (3) finetune a Pythia 410M deduplicated model as target model. The other posts are held out as non-member and held-out sets for the evaluation.
Udiff
wi W,MIA, (Ivvl)]
E
w. M.MIA, (Ians)
i=1
wi (E[MIA, E[MIA (r
> if Dus is member set, otherwise S O.
More Complicated Dataset and Model. We also evaluate our method on the Pile dataset (Gao et al., 2020), which is much more complicated and has subsets of diverse types of texts. We use the de-duplicated version of Pythia 1B model as the target model. The training split of the Pile dataset used as member data, and the held-out and test split is used as non-member data. Here, we only evaluate Pile subsets that are free from copyright issues. Please also refer to Appendix C for detailed configuration on the Pile.
Here, wa E I is the weight for the MIA score MIAG Assum- ing that Dan. and Drai are i...d., we have ME[MIA, (ara)) >
E[MIA, (fsss on member set and (EMMIA(( E (MIA((3rms)) on non-member set for each MIA score. Therefore, we have Ydiff close to 0 on the non-member set, regardless of the weights w; However, when we synthesize the held-out set with a generator, there can be a small dis- tributional shift between Dau. and D With this shift, we can have E (Iva)) < E[MIA, on non-member set. This is often the case for generated text, because the generator usually produces held-out texts that simpler than human-writted texts, therefore causing the generated held-out texts to have smaller perplexity. This affects most perplexity-base based methods, such as LOSS, Min-K% and Zlib ratio. Consequently, the linear regression algorithm can assign negative weight w; to such MIA scores, which causes Yain on the non- member set and therefore high false pos- itive rates. To ensure that this generation shift does not add up falsely high Yain constrain the weights to be posi- tive. As shown in Figure 4, the weights are projected from I to (0, 1) with Sigmoid function (x) = Thee With such weight constraint, the linear regression only assigns a small weight we for MIA, if E[MIA, < E[MIA, (Zaus), ), avoiding false positives many cases. We also present the
Implementation Details We finetune a Llama 3 8B model (Dubey et al., 2024) with LoRA as the generator. For both types of datasets, we split 2,000 sequences as the generator inference set, and the others the generator training split, Both text classifier and combined classifier are trained on 1,000 synthetic held-out data and 1,000 suspect data for each dataset. Our proposed t-test is also conducted on 1,000 syn- thetic held-out data and 1,000 suspect data. More implemen- tation details can be found in Appendix D. We also provide an analysis of hyperparameter sensitivity in Appendix K.
5.2. Results for Single Author Dataset
The experimental results on the single author dataset are pre- sented in Table 3. On the member set, the combined classi- fier Coumb outperforms the text classifier Gext, by a large mar-

---

Unlocking Post-hoc Dataset Inference with Synthetic Data
Table Results for different Pile subsets. True represents the true membership while Inferred denotes the inferred membership. Our generation successful if these two align.
Table 5. Ablation studies of our approach. Setting 1-3: replacing our generation method with baselines. Setting 4-5: removing key designs from our generation method. Setting 6: without post-hoc calibration. Setting 7: without weight constraint. Setting 8: our complete method.
AUC (%)
AuC (%)
P-value
Inferred
Pile-CC
55.3
55.6
53.3
58.2
0,002
0.99
X
Setting
Configuration
True Memhership
P-value
Membership
Inferred
Wikipedia
56.0
56.8
0.04
wlo Sutlix Completion
X
x
54.9
52.4
1.00
X
(ICL P'rooprasingg
x
1.0
X
ArXiv
53.6
59.1
<0.001
/
w/o Suttix Completion
/
1.
X
53.1
53.3
0.74
(ICL Teat Completion)
1.0
X
NTH ExPorter
56.8
$7.7
0.02
/
3 w/o Suffix Completion (Preference Optimization)
1.0
X
53.6
53.3
1.00
x
1.0
X
FreeLow
52.8
<0.001
4 w/o Segment and Shuffle
1.0
X
X
51.4
53.9
0.09
x
X
1.0
X
Ubuntu IRC
53.4
55.8
0.01
w/o Suffi Comparison
1.0
X
53.4
54,9
0.33
X
1.0
X
54.6
58.1
<0.001
w/o Post-bac Calibration (Original 7-test DI)
<00001
/
PubMed Central
<0.001
Githab
53.9
53.6
55.4
0.003
0.07
x
7
w/o Weight Constraint
0.004
0.43
X
/
EuroParl
51.0
51.4
53.9
57.0
<0.001
0.07
Ours
X
<0,001
1.0
X
PhilPapers
55.1
59.1
<0.001
/
61.1
56.0
0.99
HackerNews
56.7
60.7
<0,001
/
text, academic writing, and code using our method for gen- erating the held-out data. The results also show that our gen- eration method generalizes well to documents with different lengths, ranging from KB (Wikipedia) to 70 KB (PhilPa- pers). Moreover, our proposed method generalizes well to texts from different domains and languages, e.g., medical (PubMed Central), legal (FreeLaw), and multilingual (Eu- roParl) domains. Notably, the p-values for our difference comparison t-test are significantly lower than 0,05 on all the evaluated member sets, and higher than 0.1 on all the non- member sets. Please refer to Appendix F and Appendix for the results on different model sizes and model architectures.
58.0
56.7
0.43
X
54.5
58.0
<0.001
Enron Emails
X
54.6
52.9
0.99
x
54.3
60.1
<0.001
/
StackExchange
53.0
55.0
0.06
54.5
59.0
<0.001
/
PubMed Abstracts
53.8
53.4
0.90
X
USPTO Backgrounds
52.8
0.001
52.8
52.2
0.97
X
Mathematics
53.9
55,5
0,002
54.0
51.3
1.00
gin of 1.8% AUC score. Moreover, the observed p-value of 0.01 strongly supports the alternative hypothesis, indicating that the superior performance of Coomh over Gext is statisti- cally significant. This enables our method to correctly iden- tify that the target set is part of the training set. For the non- member set, Ccomb and Gext achieve comparable AUC scores, with p-value of 0.13 that significantly exceeds the thresh- old of 0.05. This result confirms the ability of our approach to correctly identify non non-menne member texts such, thus avoid- ing the false positives that occur with the original LLM DI approach. Here, we finetune the target model on the single author dataset with LoRA for one epoch. We also present the results with other fine-tuning setups in Appendix H.
5.4. Ablation on Post-hoc Dataset Inference
We conduct ablation studies to separately analyze the contri- bution of the three components in our held-out data genera- tion: suffix completion, calibrating, and weight constraint.
Suffix Completion. As presented in Table 2, our proposed sequence completion scheme can synthesize held-out texts with a distribution much more similar to the suspect texts when compared with the baseline methods. In addition to the AUC results, we also show that the baseline generation methods cannot produce reliable held-out sets even when combined with our post-hoc calibration and weight con- straint in Table 5. In particular, replace our generation scheme with three baselines, including ICL. paraphrasing, ICL text completion, and preference optimization. The values are presented as Setting 1-3. We also remove two key designs in our generation method, 1) Segment and Shuffle, and 2) Suffix Comparison, as shown in Setting 4-5. In all
5.3. Results for Pile Datasets
The results of different Pile subsets are shown in Table 4. We observe that DI correctly predicts the membership of datasets from diverse domains and styles, including plain
8

---

Unlocking Post-hoc Dataset Inference with Synthetic Data
10a
Table 6. The AUC of different classifier architectures.
10
10"2
Architecture
AUCC Tex
Training Time
P-value
10:
all-MimiLM-L6--22
50.8
0.3
10-4
BERT 1.ma3-88, Pre-tmined-I
51.2
2.0
53.2
65.1
10 5
GPT2, Pre-trainn--oRA
53.0
26.2
EuroPari Mediannatatt
GPT2, Tre-Irained-Full Finetuned
52.3
36.8
10-6
GPT2, 2 Inyerrs-iittalized
53.3
0.5
10-7
MaXpatautt
500
1000
1500
2000
number of the suspect and held-out set, which is also the number of queries made to the target model. The two sets are of the same size, as they are produced in a pairwise man- ner. We observe from Figure 5 that, as the number of sam- ples increases. DI exhibits improved detection capability of training data. Notably, with fewer than 1,000 samples, DI achieves statistical significance (p- 0.05) across most of the evaluated datasets. When increasing the sample size to 2k queries, the method demonstrates even stronger statisti- cal significance (p- < 0.01) consistently across all datasets.
Number of Samples
Figure 5. The p-values of member sets with change in sample size. Mediannasee denotes the median p-value of different datasets, and Meamoamer the maximum p-value of all subsets. Number of samples refers to the total size of both suspect and validation sets.
the above settings, the p-values for both member and non- member sets are 1.0, which indicates that the Cext has bet- ter or similar performance when compared with Coomh The reason behind the observation is that the distributional shift caused by the generation is much larger than the shift in- duced by the membership signal, such that Coomb does not outperform Cuxt even with extra membership inputs on the member set. Consequently, the DI predicts both sets as non- member and suffers from false negatives.
5.6. Choice of Classifier
We explore different text classifier architectures and present the results for different architectures and different parame- ter sizes in Table 6. The results show that the simple GPT2- based classifier with 2 layers and initialized weights can achieve the best AUC in our experimental settings. Addi- tionally, this lightweight classifier has significantly short training time, making the method more practical when faced with more queries. Therefore, we choose this 2-layer GPT2- based classifier as our text classifier. During our experi- ments, we consider the scenario where the author can only provide a limited number of tokens, so stronger text classi- fiers, such Llama and full GPT2 models, can be easily overfitted. In world applications, an arbitrator is sug- gested to select the most suitable text classifier based on their specific conditions regarding data size, data type, and computation resources.
Post-hoc Calibration. We replace our calibration method with the original DI without calibration, as shown in Set- ting 6 of Table 5. Specifically, only linear classifier is op- timized aggregate different MIA metrics and output the final prediction score. Furthermore, the t-test is conducted directly between the predictions on the target set and the ones on the held-out set. We observe that the p-values under this condition are extremely low for both member and non- member sets, and DI has false positive in this case. This observation aligns with results in Section 3, where we show that even a small distributional shift causes significantly small p- p-value in the original DI. Therefore, our post-hoe cal- ibration approach is crucial to evaluating the distributional shift caused only by membership signals.
6. Conclusions
Weight constraint. As explained in Section 4.3, the weight constraint avoids summing the distributional shift caused by generation to the final MIA prediction when the direc- tion of the generation shift is different from that caused by the membership signal. As shown in Setting 7 of Table 5, applying the constraint leads to a much lower p-value on the member set and much higher on non-member set, which helps our method make a more accurate prediction about the membership.
We propose how to synthetically generate an in-distribution held-out dataset to enable the real-world application of DI Therefore, we solve two critical challenges, namely (1) creating high-quality, diverse synthetic data that accurately reflects the original distribution and (2) bridging likelihood gaps between real and synthetic data. Our solution relies on designing data generator training scheme based on suffix- based completion task and post-hoc calibration to align the likelihood gaps between real and synthetic data. Through extensive experimental evaluation, we highlight that our method enables a robust DI and correctly identifies training data while achieving a low false positive rate. This shows our method's reliability to support copyright owners to make litigations.
5.5. Analysis of Sample Size
We also set out to analyze how the sample size in our pro- posed t-test affects the statistical confidence of DI with our generated held-out data, Here, the sample size the total
legitimate claims on data usage for

---

Unlocking Post-hoc Dataset Inference with Synthetic Data
Impact Statement
Carlini, N., Tramer, F., Wallace, E., Jagielski, M., Herbert-
Voss, A., Lee, K., Roberts, A., Brown, T., Song, D., Erlingsson, U., et al. Extracting training data from large language models. In 30th USENIX Security Symposium (USENIX Security 2/), pp. 2633-2650, 2021.
Crawling of data for training LLMs is becoming pervasive, with model training companies scraping vast spans of the internet in order to find high-quality data. Given the strong correlation between data quality and model performance, many content creators want to protect their work from be- ing trained on, without their consent. Claiming that one's data has been trained on, only with access to the suspect LLM, has thus far stayed near impossible. Our work takes leap forward by allowing content creators to 'post-hoc' infer their data on the internet was trained on by leverag- ing synthetic data. This means that authors, bloggers, and columnists with decades of internet data can now leverage our method in order to claim their rightful ownership. Our work aims to serve as an important tool in future copyright litigations, in particular in their 'discovery' period.
Das, D., Zhang, J., and Tramèr, F. Blind baselines beat mem- bership inference attacks for foundation models. arXiv preprint arXiv:2406. 16201, 2024.
Duan, H., Dziedzic, A., Papernot, N., and Boenisch, F. Flocks stochastic parrots: Differentially private prompt learning for large language models. In Thirty-seventh Conference on Neural Information Processing Systems (NeurIPS), 2023a.
20233..
Duan, H., Dziedzic, A., Yaghini, M., Papernot, N., and Boenisch, F. On the privacy risk of in-context learning. In The 6l Annual Meeting The Association For Com- putational Linguistics, 2023b.
Acknowledgment
This work was supported by the German Research Founda- tion (DFG) within the framework of the Weave Programme under the project titled "Protecting Creativity: On the Way to Safe Generative Models" with number 545047250. Re- sponsibility for the content of this publication lies with the authors.
Duan, M., Suri, A., Mireshghallah, N., Min, S., Shi, W., Zettlemoyer, L.,, Tsvetkov, Y., Choi, Y., Evans, D., and Hajishirzi, H. Do membership inference attacks work on large language models? urXiv preprint urXiv:2402.0788 078411, 2024.
Duarte, A. V., Zhao, X., Oliveira, A. L., and Li, L. De- cop: Detecting copyrighted content in language models training data. arXiv preprint 2021v:2402.099100 2024.
References
The times sues a.i.
openai and microsoft use of
over http://wwwwwyyyii....mm222222221
copyrighted work
Dubey, A., Jauhri, A., Pandey, A., Kadian, A., Al-Dahle,
A., Letman, A., Mathur, A., Schelten, A., Yang, A., Fan, A., et al. The llama 3 herd of models. arXiv preprint arXiv:2407.2178. 2024.
voteriinss-pppaaammmmcoootttaaav 2023, URL
1-microsoftt-Lawait..rom
Dubiński, J., Kowalezuk, A., Boenisch, F., and Dziedzic, A. CDI: Copyrighted Data Identification in Diffusion Models. In The IEEE CVF Computer Vision and Pattern Recognition Conference (CVPR), 2025.
Sarah silverman and authors sue openai and meta over copy- right infringement. 2023. URL https:/ /tww://wwwww/// nyti mes coms/coo//e wtw.c//.m22222 2023 /0/////0/11/// art wwwtrrattaaaaeeeeeeeeeeeeeeeeeee parrhh-silvens an- nonllassuit. wwwpsuuit-ppeeaae -meta. html.
Dziedzic, A., Dhawan, N., Kaleem, M. A., Guan, J., and Pa- pernot, N. On the difficulty defending self-supervised learning against model extraction. In ICML (Interna- tional Conference Machine Learning), 2022a.
Balloceu, S., Schmidtová, P., Lango, M., and Dušek, O. Leak, cheat, repeat: Data contamination and evaluation malpractices in closed-source liss. In Proceedings of the 18th Conference of the European Chapter of the Associa- tion for Computational Linguistics (Volume I: Long Pa- pers). pp. 67-93, 2024.
Dziedzie, A., Duan, H., Kaleem, M. A., Dhawan, N., Guan, J., Cattan, Y., Boenisch, F,, and Papernot, N. Dataset infer- ence for self-supervised models. Advances in Neural In- formation Processing Systems, 35:12058-1200 12070, 2022b.
Biderman, S., Schoelkopf, H., Anthony, Q. G., Bradley, H., O' B'Brien, K., Hallahan, E., Khan, M. A., Purohit, S., Prashanth, U. S., Raff, E., et al. Pythia: A suite for an- alyzing large language models across training and seal- ing. In International Conference on Machine Learning, pp. 2397-2430, PMLR, 2023.
Fu, W., Wang, H., Gao, C., Liu, G., Li, Y., and Jiang, T. Membership inference attacks against fine-tuned large language models via self- seff-prompt calibration. In The eighth Annual Conference on Neural Information Processing Systems, 2024.
10

---

Unlocking Post-hoc Dataset Inference with Synthetic Data
Gao, L., Biderman, S., Black, S., Golding, L., Hoppe, T., Foster, C., Phang, J., He, H., Thite, A., Nabeshima, N., et al. The pile: An 800gb dataset of diverse text for language modeling. arXiv preprint arXiv:2101.00027, 2020.
Maini, P., Yaghini, M., and Papernot, N. Dataset inference: Ownership resolution in machine learning. In 9th Inter- national Conference Learning Representations, ICLR 2021, Virtual Event, Austria, May 3-7, 2021. OpenRe- view.net, 2021.
Golchin, S. and Surdeanu, M. Time travel in Ilms: Trac- ing data contamination in large language models. arXiv preprint Xrviv22308.00 08493, 2023.
Maini, P., Jia, H., Papernot, N., and Dziedzie, A. LLM dataset inference: Did you train on my dataset? In The Thirty-eighth Annual Conference on Neural Information Processing Systems, 2024.
Groeneveld, D., Beltagy, I., Walsh, E.. Bhagia, A., Kinney, R., Tafjord, O., Jha, A., Ivison, H., Magnusson, I., Wang, Y., et al. Olmo: Accelerating the science of language models. Proceedings of the 62nd Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers). pp. 15789- 15809, 2024.
Mattern, J., Mireshghallah, F., Jin, Z., Schoelkopf, B., Sachan, M., and Berg- Kirkpatrikk,,, T. Membership infer- ence attacks against language models via neighbourhood comparison. In Findings of the Association for Computa- tional Linguistics: ACL 2023, pp. 11330- 11343, 2023.
Hanke. V., Blanchard, T., Boenisch, F., Olatunji, E., Backes, M., and Dziedzic, A. Open lims are necessary for current private adaptations and outperform their closed alternatives. In Tiihth-Eighth Conference on Neural In- formation Processing Systems (NeurIPS), 2024.
Meeus, M., Shilov, I., Jain, S., Faysse, M., Rei, M., and de Montjoye, Y.-A. Sok: Membership inference attacks on lims are rushing nowhere (and how to fix it). arXiv preprint arXiv:2406. 17975, 2024.
Meng, Y., Xia, M., and Chen, D. SimPO: Simple prefer- ence optimization with a reference-free reward. In The Thirty-eighth Annual Conference on Neural Information Processing Systems, 2024.
Hayes, J., Shumailov, I., Choquette-Choo C. A., Jagielski, M., Kaissis, G., Lee, K., Nasr, M., Ghalebikesabi, S., Mireshghallah, N., Annamalai, M. S. M. S., Shilov, I., Meeus, M., de Montjoye, Y.-A., Boenisch, F., Dziedzic, A., and Cooper, A. F. Strong membership inference at- tacks on massive datasets and (moderately) large lan- guage models. 2025.
Oren, Y., Meister, N., Chatterji, N. S., Ladhak, F,, and Hashimoto, Proving test set contamination in black- box language models. In The Twelfth International Con- ference on Learning Representations, 2024.
Kazmi, M., Lautraite, H., Akbari, A., Tang, Q. Soroco, M., Wang, T., Gambs, S., and Lécuyer, M. PANORAMIA: Privacy auditing of machine learning models without retraining. In The Thirty-egghth Annual Conference on Neural Information Processing Systems, 2024.
Penedo, G., Kydlíček, H., Lozhkov, A., Mitchell, M., Raffel, C., Von Werra, L., Wolf, T., al. The fineweb datasets: Decanting the web for the finest text data at scale. arXiv preprint arXiv:2406. 002406..7555 /7557, 2024.
Kowalczuk, A., Dubiński, J., Boenisch, F., and Dziedzic, A. Privacy attacks on image autoregressive models. In Forty- Second International Conference on Machine Learning (ICML), 2025.
Rafailov, R., Sharma, A., Mitchell, E., Manning, C. Er- mon, S., and Finn, C. Direct preference optimization: Your language model is secretly reward model. Ad- vances in Neural Information Processing Systems, 36, 2024.
Li, C. and Flanigan, Task contamination: Language mod- els may not be few-shot anymore. In Proceedings of the AAAI Conference on Artificial Intelligence, volume 38, pp. 18471-18480, 2024.
Rahman, N. and Santacana, E. Beyond fair use: Legal risk evaluation for training llms on copyrighted text. 2023. URL htt.t:tp///tm //tp://///aw..mm//e hatr/CCameraaaaae pdf.
Magnusson, I.. Bhagia, A., Hofmann, V., Soldaini, L., Jha, A. H., Tafjord, O., Schwenk, D., Walsh, E., Elazar, Y., Lo, K., et al. Paloma: A benchmark for evaluating language model fit. Advances in Neural Information Processing Systems, 37:64338-64374 2024.
Reuters. Getty images lawsuit says stability ai misused photos to train AI, 2023. URL http:://// reut ers com crssscco///ellllgtttt--geegggg wwwg--immmesslwssiiiee ays-stabil wttssttaaiittttataaaaaaaa -tl.-iyy--------ss--------aeoi ai- 2i-2023-000 02-06/
Maini, P. and Suri, A. Reassessing emnlp 2024's best paper: Does divergence- based calibration for membership infer- ence attacks hold up? 2024. URL https: www. an shumansuri. com/ htom/blog///// ///o/////////////// htte4//albbratedrr
Roberts, M., Thakur, H., Herlihy, C., White, C., and Dooley, S. To the cutoff... and beyond? a longitudinal perspective on 1lm data contamination. In The Twelfth International Conference on Learning Representations, 2024.
1a/. Accessed January 29, 2025.

---

Unlocking Post-hoc Dataset Inference with Synthetic Data
Shi, W., Ajith, A., Xia, M., Huang, Y., Liu, D., Blevins, T., Chen, D., and Zettlemoyer, L. Detecting pretraining data from large language models. In The Twelfth International Conference on Learning Representations, 2024.
Shokri, R., Stronati, M., Song, C., and Shmatikov, V. Mem- bership inference attacks against machine learning mod- cls. In 2017 IEEE symposium security and privacy (SP), pp. 18. IEEE, 2017.
Sidák, Z. Rectangular confidence regions for the means of multivariate normal distributions. Journal of the Ameri- can statistical association, 62(318):626 633-2 1967.
Soldaini, L.,, Kinney, R., Bhagia, A., Schwenk, D., Atkin- son, D., Authur, R., Bogin, Chandu, K., Dumas, J., Elazar, Y., et al. Dolma: open corpus of three trillion tokens for language model pretraining research. In Pro- ceedings of the 62nd Annual Meeting of the Association for Computational Linguistics ( (Volume 1: Long Papers), pp. 15725-15788, 2024.
Weber, M., Fu, D. Y., Anthony, Q., Oren, Y., Adams, S., Alexandrov, A., Lyu, X., Nguyen, H., Yao, X., Adams, V., Athiwaratkum, B., Chalamala, R., Chen, K., Ryabinin, M., Dao, T., Liang, P., Ré, C., Rish, I., and Zhang, C. Redpajama: an open dataset for training large language models. NeurlPS Datasets and Benchmarks Track, 2024.
Wu, X., Duan, R., and Ni, J. Unveiling security, privacy, and ethical concerns chatgpt. Journal of Information and Intelligence, 2023.
Xu, H., Sharaf, A., Chen, Y., Tan, W., Shen, L., Durme, B. V., Murray, K., and Kim, Y. J. Contrastive preference optimization: Pushing the boundaries of LLM perfor- mance in machine translation. In Forty-first International Conference on Machine Leurning, 2024. URL https: //openreview net. formm iididdd510 Sinniiiii55lipppp
Yeom, S., Giacomelli, I., Fredrikson, M., and Jha, S. Pri- vacy risk in machine learning: Analyzing the connection to overfitting. In 2018 IEEE 31st computer security foun- dations symposium (CSF), pp. 268-282. IEEE, 2018.
Zhang, J., Das, D., Kamath, G., and Tramèr, F. Membership inference attacks cannot prove that a model was trained on your data. arXiv preprint arXiv:2409.
2024a. Zhang, J., Sun, J., Yeats, F., Ouyang. Y., Kuo, M., Zhang, J., Yang, H. F., and Li, H. Min-k%++: Improved baseline for detecting pre-training data from large language models. arXiv preprint arXiv:2404.0295: 2024b.
12

---

Unlocking Post-hoc Dataset Inference with Synthetic Data
A. Details of ICL Generation
experiment with GPT-4-Turbo and prompt it using in-context-leaming learning to paraphrase the suspect data. Each prompt includes few data points as demonstrations (shots) and requests the model produce paraphrases for the suspect set. We test types of templates to prompt GPT-4-Turbo model with in-context-learming (ICL). One is to paraphrase the given sample based on the examples, and the other to complete the text with part of the sentence.
A.1. Text Completion Prompt
In the text completion prompt. we give ten examples from the author provided documents, and prompt the GPT-4-Turbo model complete the given sample, which is shown the end of the prompt. Here is example of this kind of queries:
Inpul:
have some text samples. Please help complete the last sample based my example samples.
device may be provided to sense an envi ronment's present temperature. For example, [...] a thermistor that is robust and reliable and that can be fabricated at a reasonable cost.
[...]
This invention relates to method of synthesizing musical tores.A musical tone natural tone produced by acoustic or natural musical instrument can [...] With the music synthesizer it is considerably difficult to obtain dynamic audio
Radio communications receivers commonly receive and decode messages which contain information. A part of this information may be [...]a single radio will transmit messages the remainder the group. These radio
Output:
messages are selectively received by the grouped radios which decode the identity information within the [...] and secure group communications in diverse and often challenging operational environments.
A.2. Paraphrasing Prompt
In the paraphrasing prompt, we give ten examples from the author provided documents, and prompt the GPT-4-Turbo model paraphrase the last. Here an example this kind of queries:
Input:
Paraphrase the last sample according to the other samples.
1. Field of the Invention This invention relates generally to fluid injectors.
More particularly, this invention relates to proportional needle control fuel injector having [...] The needle valve is moveable between a first position closing the injection fuel outlet port and a second position to open to injection fuel outlet.
[...]
The present invention relates the structure of a casing of a miniature
portable [...] is forcibly urged away from the casing by a tool, it is likely that the casing is scratched or the base plate is deformed.
This invention relates to fuel cells and, in particular, to improvements in the catalyst used in internal [...] fuel cell in which the catalyst assembly has minimum displacement and permits vertical and horizontal orientations of the fuel cell.
Output:
13

---

Unlocking Post-hoc Dataset Inference with Synthetic Data
This invention pertains to fuel cells, specifically addressing enhancements in catalysts utilized [...] and efficiency of internal reforming fuel cells by
mitigating previous limitations and improving overall system functionality.
A.3. Distribution Shift Analysis
The distributional shifts quantified in BoW AUC and GPT2 AUC shown in Table A1. As observed, both types of prompt templates produces synthetic texts with large distributional shifts to the suspect sets. Notably, the GPT2-based classfier can achieve as much as an AUC of 99.2%.
Table AI. Distributional shifts between the suspect set and GPT-4-Turbo generated validation set.
Template Type
BoW AUC(S)
GPT2 AUC(K)
ICL Text Completion
79.2
99.2
Paraphrasing
99.0
B. Details of Preference Optimization Generation
Preference optimization methods focus on optimizing a pre-trained LLM based on human preference (Rafailov et al., 2024; Xu et al., 2024). Particularly, LLMs iteratively produce random generations, then human annotators are requested to label the generations as chosen or rejected, and the L.LMs are further optimized according to this human feedback. We note that, we can leverage preference optimization approaches to make our generator model prefer the human-written texts over synthetic data, thus producing texts with more similar distribution natural texts. Here, we instantiate the preference optimization scheme with state-of-the-ae art method, the simple preference optimization (SimPO) (Meng et al., 2024). During each training iteration, the human-writee suspect data are always labeled as chosen and the generations from the last iteration are marked as rejected. As noted in Section 4.1, this approach improves significantly upon prompted paraphrasing, but still causes a large distributional shift between the suspect set and the generated held-out set.
Table A2. Segmentation configurations for different Pile subsets.
Number of
Chosen Split Size
Max. Snippets per Document
Number of Tokens per Snippet
Subset
Test Set in Pile
Pile CC
> 3000
3000
30
64
StackExcaange
4000
5
64
PubMed Abstracts
56000
6000
5
64
Wikipedia (en)
> >-000
30
64
USPTO Backgrounds
56000
6000
10
32
PubMed Contral
-500
500
100
64
FreeLaw
>-4000
4000
5
32
ArXiv
200
200
100
32
ExPorter
64
TackerNews
> 3000
3000
10
64
Github
> 2000
20
64
Enron Emails
1957
1957
50
32
DM Mathematics
>150
1500
50
32
EuroParl
290
290
200
32
PhilPapers
132
132
100
32
Ubuntu
2000
C. Pile Dataset Segmentation
We present the details for the configurations of Pile subset in Table A2. We note that, it is claimed that the following Pile subsets may have copyright issues and cannot be included for evaluation: Books3, OpenWebTTer Gutenberg (PG-19), OpenSubtitles. BookCorpus2, and YoutubeSubtitles. For most subset there are documents that are much longer than the other documents, which causes that too many snippets are sample from these documents if all snippets are used. Therefore, we set a maximum snippet for each document on each subset according to the median lengths of the documents. Also, we note that our approach can achieve good performance on most subsets with only 32 tokens. For certain subsets, we use
14

---

Unlocking Post-hoc Dataset Inference with Synthetic Data
token length of 64 for stronger membership signal. The average token number for the Pile subset is 45. In practical applications, suggest choosing the token numbers when the text-classifier has the minimal AUC avoid distributional shift much as possible. As more conservative approach, this avoids making false accusations of copyright violation.
D. Implementation Details
D.I. Generator
The LoRA rank for the generator is 32. The generator is trained for 100 epochs, and the learning rate is set to 2 x 10-4. set warm-up ratio of 0.03, and a linear scheduler is used to dynamically adjust the learning rate.
D.2. Text and Combined Classifiers
For both the text and the combined classifier, we leverage the basic architecture the GPT2 classfier with an extra linear layer. Specifically, the classifier has only two layers, with an embedding dimension of 1600 and attention head number of 25. As explained in Section 4.3,, apply weight constraint to the linear layer. The GPT2-based classifier is optimized for 20 epochs, and the linear layer is further optimized for 200 epochs.
E. Examples of Synthetic Texts
In this section, we provide some examples of the synthetic texts on the Pile dataset. Here, prefix denotes the first half of generated text, real suffix refers the original suffix the natural text, and generated suffix refers the synthetic completion based on the prefix. We observe that, the generated suffixes are reasonable continuation of the prefixes. The generated suffixes also align with the style of each dataset and do not overfit the content of the real suffixes.
E.1. Pile-CC Prefix:
are excited about and also what we hoped see from this years E3!
Real suffix:
From the surprising new Spider-Mo Man PS4 game the bizarre Happy Few and Generated suffix:
Let us know your thoughts this monologue we are preparing for next
E.2. Stackxchaange
Prefix:
var rKentityyisttw:hhaatttoo new debiteur Get () GetProperty "N"
Real suffix:
schakeling\") GetValue (dbEntry) (elaennnrd as List<FKnttity::/Duu
Generated suffix:
PKEntityListN . GetVValue (instance, null); An
foreach (var in FKEntity
E.3. PubMed Abstracts
Prefix:
were calculated using the Kaplan-Meier method. of the 117 patients in
15

---

Unlocking Post-hoc Dataset Inference with Synthetic Data
Real suffix:
whom data were analyzed, 103 had follow-up MR or CT images and 14 patients were
Generated suffix:
the study (76 with UC and 41 with DC), 45 patients required proctocolic resection
E.4. Wikipedia (en)
Prefix:
Em is going away for a while. While it's s not up to the standard
Real suffix:
"Mockingbird," it is more fully realized than the two other new Generated suffix:
of their three previous albums, cattle call is still enjoyable romp,
E.5. USPTO Backgrounds
Prefix:
1. Field of the InventionlnTtis invention relates to storage device for athletic equipment and, in particular, to portable storage device for transporting and retaining
Real suffix:
elongate items of athletic equipment such as hockey sticks and related athletic equipment. \n2. Discussion of Related AnNumerous team athletic activities require individual players on the
Generated suffix:
multiple pairs of basketballs. \n2. Description of the Related ArtAnDuring the summer and other periods when there is extended break from athletic school or program
E.6. PubMed Central
Prefix:
example, both cycles apply Lewis acidic metal centers to bind the monomers (ep
Real suffix:
oxide or lactone), , and both invoke labile metal alkoxide intermedi ates
Generated suffix:
oxides or cyclic carbonates), but the axes of the metal detallan lacycle in
16

---

Unlocking Post-hoc Dataset Inference with Synthetic Data
E.7. FreeLaw
Prefix:
Court, 638 P.2d 65 (Colo.1981
Real suffix:
Here, the juvenile court denied the GAL's motions because it did not want
Generated suffix:
) cert. denied, 454 U.S. 1146, 102
E.8. Arxiv
Prefix:
up and vice versa. In contrast, fundamentalists expect the price to track its
Real suffix:
fundamental value. Orders from this type of agent may be written as\n\n$sD
Generated suffix:
underlying fundamentals up and down, but given sufficient acceleration the price might \u20lcrun away
E.9. NIH ExPorter
Prefix:
attachment and growth, respectively. Together with an industrial sponsor, Vaxiron
Real suffix:
Inc., we will develop quality control tools and metrics for assessing vaccine antigen formulations,
Generated suffix:
the applicant has carried out clinical trials of different vaccine candidates based on different viruses for
E.10. Github
Prefix:
.string \"reach only by using BIKE technique. & MAw.tttellll _ext____
Real suffix:
16BEF6:: @ 6162EF61n(t.string
\Which
Generated suffix:
16F381:: c16F381\nkt.stae
"RUUEE (ROAD
17

---

Unlocking Post-hoc Dataset Inference with Synthetic Data
E.11. Enron Emails
Prefix:
Lay. He went on to say that Kenneth was Dewayne Re
Real suffix:
es' cousin and started telling about all of your fine attributes and what
Generated suffix:
ams' direct Anreport and that it would be extremely difficult for Kenneth to get
E.12. EuroParl
Prefix:
het mondeling amendement op schrift heeft gekregen. AnIk st
Real suffix:
el voor om niet te spreken over \'de Raad de lidstat Generated suffix:
akk voor de uitnodiging tijdens uw volgende bij
E.13. PhilPapers
Prefix:
distribute well among [the gods who fought with him] their titles and privileges
Real suffix:
(885, cf. 66\u201367 and 74); to swallow
Generated suffix:
(17.1). Orderly distribution praise for the victory is re
E.14. Ubuntu IRC
Prefix:
about setting up reoccuring status meetings?\acddarriinng should we start
Real suffix:
holding those or is it too soonfnnndfa Luke will be joining Generated suffix:
with a status meeting or a design mettingg/aammnshheii dfarning
18

---

Unlocking Post-hoc Dataset Inference with Synthetic Data
E.15. HackerNews
Prefix:
Angular (work just uses Dojo) An'nPeople don't seem to
Real suffix:
be hungry here. An -neeewiisspllllrrdlkkorrre for IBM as a software engineer on one of
Generated suffix:
care that i adding yet another -2CKB per page. We' re\nsaying no
F. Generalization to Different Model Sizes
Here, we evaluate the performance of our method on three different model sizes of the Pythia model: 1.4B, 2.8B, and 6.9B. We outlier removal ratio 0.01 for Pythia-6.9B, Pythia- IB, 0.2 for Pythia-410M, All the tested models are the deduplicated versions. The results demonstrate that our proposed method generalizes well across different model sizes.
0.05 for Pythia-2.8B, r 0.1 for Pythia-1.4B, 0.15 for
Table A3.. Results for different sizes of Pythia Models. True represents the true membership while Inferred denotes the inferred memhership. Our generation successful these two align.
410
Pythia
Pythia
Pythia
Subsel
True
AUC
AUC Conbs (96)
P-value Infered
Tex
Comb (9)
p-value Infered
AUC Test
AUC Coob (%)
P-salee
Infervad
AUC Coob (9)
Pasalee Inferred
Pil-CC
$4.9
57.4
0,009
1,00
/
X
55.5
57.5
0.007
0.89
/
56.2
57.8
0.02
55.9
58.2
D.06
X
AcXiv
54.3
60.5
.0..00
0.87
/
x
$4.0
59.3
<0.000
0.64
53.8
57.7
<0.001
0.67
/
53.7
52.4
$7.0
<0.001
x
/
FiceLaw
/
52.9
58.3
<00,00
/
52.6
57.3
<0..00
52.2
57.4
<0.001
52.2
56.7
<0.001
/
PubMed
/
51.7
58.0
<00000
54.1
58.4
<00.001
55.1
57.9
0.002
54.7
57.2
0.002
/
Earo-
55.7
0,002
55.6
0.004
$1.0
53.9
0.02
50.7
54.2
0.04
0.13
0.31
Phil-
59.5
<00000
58.3
<00.001
<0.001
56.8
0.02
Papers
55.6
1.00
56.1
0.98
61.4
56.2
0.98
60.2
$6.0
0.99
61.1
<00.001
<0.001
0.0e
News
58.2
55.6
0.94
X
58.2
57.1
0.24
58.1
37.7
0.17
58.7
57.7
0.24
<0.000
57.7
<00001
Emails
x
51.8
$2.8
1,00
X
54.6
53.0
0.97
54.5
34,9
0.20
34.6
54.0
0.76
<00.001
58.8
<00.001
58.1
<00.001
57.2
Exchange
x
32.3
35.0
0.06
X
52.1
34.3
0.090
x
52.1
54.1
0.24
52.5
54.0
0.29
59.7
<00.001
58.5
<0:0001
58.3
<0.0001
<0.001
Abstract
54.9
52.6
1.00
x
54.1
34,0
0.74
X
53.9
53.7
0.83
x
53.9
54.3
0.57
x
USPTO
56.8
0
56.3
0.004
52.4
35.6
0.028
52.6
55.4
0018
Rack.
x
$2.6
51.7
1.00
x
$2.9
52.2
0.99
x
52.5
53.2
0.98
x
52.6
$3.1
0.75
x
G. Example of Weight Constraint
Here, we provide the following example to illustrate the importance of weight constraint. In Table A4, we show the score of three MIAs for suspect/held-out pair on both member and non non- member sets, For each suspect/held-ou pair, the smaller MIA score is highlighted in bold in the table. We have the following observations:
1. On the member set, suspect data consistently shows smaller MIA scores. This occurs because membership signals have stronger effects than generation, causing suspect data to consistently yield lower MIA scores than held-out data.
19

---

Unlocking Post-hoc Dataset Inference with Synthetic Data
2. On the non-member set, held-out data may exhibit smaller values for certain MIAs. This happens because generation randomness introduces fluctuation in MIA scores.
For both member and non-member sets, we train a linear model / that aggregates all MIA scores to predict overall score:
(xx) Σ w. M,MIA (z)
(4)
The held-out set is labeled and the suspect set O. The model assigns positive weights w, to any MIA metrics MIA, on the member set because label 0 1 and (suppss) < (IAA(held-out). However, the non-member set, the model assigns negative weight wy for MIA3- This means smaller MIA, score in the held-out set would contribute to larger overall MIA score, which is undesirable. To address this, we constrain all weights in the linear model to be strictly positive, ensuring that lower MIA, score can only result in a lower overall MIA score.
Table A4, An example to demonstrate the importance of the weight constraint.
Membership
Split
Label
MIA
MIAg
MIAg
Suspect (natural)
0.86
0.87
0.54
Held-out (generated)
0.90
0.91
0.55
Suspect (naturaf)
0
0.88
0.89
0.58
X
Held-ou (generated)
0.90
H. Other Finetuning Configurations for Single Author Dataset
We evaluate our proposed approach the single author dataset under different finetuning settings in Table A5. The results show that, the membership signal is stronger when the model is fine-tuned with more epochs. Also, our method performs better when full finetuning is used instead of LoRA.
Table A... Results for different fine-tuning methods. True represents the true membership while Inferred denotes the inferred member- ship. Our generation is successful if these two align.
Fine tuning Method
True
AUC (%)
come (%)
Povalue
Inferred
LoRA (1 epoch)
53.8
55.6
0.01
53.8
53,9
0.13
LoRA (10 epochs)
53.7
56.2
0.005
X
53.6
53.5
0.14
X
Finetuning (I epoch)
56.8
X
53.8
53.7
0.21
X
Results on the OLMo Model
We conduct the experiments to analyze the performance with OLMo-7B model (Groeneveld et al., 2024). The OLMo-7B model is trained on the Dolma V.1.7 dataset (Soldaini et al., 2024), which has a large size of 4.5 TB. Following Duan et al. (2024), we use Dolma V.1.7 as the member set and employ Paloma (Magnusson et al., 2024) as the non-member set. The results in Table A6 demonstrate that our method successfully detects both member and non-member sets for Wikipedia and Common Crawl subsets when using the OLMo-7B model as the target model.
J. Ablation Studies on Single Author Dataset
In addition to the ablation studies on the Pile presented in Section 5.4, we also perform the ablation studies on the single author dataset. The results in Table A7 follow a similar trend to the Pile, showing the importance of each component in our framework.
20

---

Unlocking Post-hoc Dataset Inference with Synthetic Data
Table Results for OLMo-7B on different data subsets. True represents the true membership while Inferred denotes inferred membership. Our generation is successful if these two align.
Subset
True
AUC (%)
AUC (8)
P-value
Inferred
52.9
55.4
0.009
Wikipedia
52.1
50.6
1.0
53.5
55.7
0.01
Common Crawl
Table A7. Results for different configurations. True Membership represents the true membership while Inferred Membership denotes the inferred membership. Our generation is successful if these align.
Configuration
True Membership
P-value
Inferred Membership
w/o Suffix Completion
(ICL Paraphrasing)
X
1.0
w/o Post hoc Calibration
V
<0.001
(Original T-test in DI)
X
<0.001
w/o Weight
/
0.02
Constraint
Ours
X
0.13
K. Analysis of Hyperparameter Sensitivity
We conducted a comprehensive analysis of hyperparameter sensitivity, focusing on two key parameters: the number of epochs and the number of t-test samples. The number of epochs represents the training epochs for our linear model that aggregates MIA scores. The number of t-test samples indicates the total sample size used in statistical analysis, including both the suspect and synthetic held-out sets. Our experimental results in Table A8 demonstrate that our proposed method exhibits robust performance across a wide range of values for both hyperparameters, indicating low sensitivity to these configuration choices.
Table A8. Performance of our method across different numbers of epochs and T-test samples.
Hyperparameter
Value
membership
P-value
Inferred membership
<0.001
100
1.0
<0.001
Number Epochs
200
<0.001
500
1.0
1000
x
1.0
<0.001
1.0
<0.001
Number of Titest Samples
2000
-00001
3000
0.41
<0.001
4000
0.25
L. Other Related Works about Test Set Contamination Detection
Test set contamination is newly identified risk, where the public test benchmarks are involved during LLM training (Balloccu et al., 2024). For example, Roberts et al. (2024) observe that LLMs are better at generating code with more appearances on GitHub, revealing that LLMs can contaminated with open-source GitHub data and are overestimated on coding tasks. Similarly, Li Flanigan (2024) demonstrate that some L.MMs have better performance on few-shot
21

---

Unlocking Post-hoc Dataset Inference with Synthetic Data
benchmarks constructed before the model training, which indicates contamination, Golchin & Surdeanu (2023) design prompts that guide LLM to reproduce exact near-exact test instances, such that the model encloses the contaminated samples memorized during the pre-training phase. Oren et al. (2024) compare the target model predictions between test set and all of its penmutations. However, this method is based on the assumption that the test set is involved in the training set in its exact order, which could be interrupted by a random shuffle before training. Test set contamination can also be potential application our method, as the proposed approach can perform training data detection on complex datasets composed by different authors.
contamination for LLMs. To detect test
M. Algorithm of Our Work
We present the detailed algorithms for our held-out data generation in Algorithm 1, and post-hoe calibration in Algorithm 2.
Algorithm Held-out Data Generation
Require: Documents Doc (Doc1,..,, Docm
Require: Hyperparameters: Document number m, Maximum sequence each document M MarSeq
Ensure: Suspect set sus and held-out set val are nearly IID
Initialize: Seq, Suss vat 0.00
2: for each document doc; Doc do
3: Segment doc; into multiple sequences {seq) seq, mm
4: if m4 < MorSSq then
5: Seq, = {seq] ...,, .... "eq,")
6: else
7: Seq, = randomly sampled M Max sequences from {seq) seq,")
8: end if
9: Seq Seq U Seg:
10: end for
11: Randomly split Sey into generator training set Seqtrain and generator inference set Seqrest
12: Optimize generator on Sequrain with next-token prediction loss
13: for each seq, E Seg do
14: pre;, sufi Divide(seg:)
su g(pre:)
Dsus= sus (sufi, 0)}
17: D D U {(suf/ 1)}
18: end for
Algorithm 2 Post-hoc Calibration
Require: Target model f
Require: Suspect set Daus and held-out set Daal nearly IID.
1: Randomly split Daus into suspect training set Dirain and suspect test set D
2: Randomly split D.ai into held-out training set 'D train and held-out test set D
test val val
val
3: Optimize text classifier Creae on sus U DD
Irain
val
4: Optimize combined classifier Ccomb (z. MIA(f(x)) on Dtrain sus
train
5: Ddiff = 0 ext
6: Delifi 0
7: for ..lest vaall D lest Diest do
8: Deliff
Somp Stmm U CCoomb (adest (f Ccomb (2:lest (aommblateet
(f (z:tes)))
9: 'Ddift Ddiff U {Ctent (2tts
Ctenn (z.tstt
text
10: end for
11: Compare and Ddiff comb and Ddiff with t-test
22


Reviews:
[{"review_id": "eGeJTQwVOj", "reviewer": "Reviewer_thbe", "summary": "This paper introduces a framework for post-hoc dataset inference in large language models (LLMs) by synthesizing held-out data. The central motivation is to address the critical bottleneck of conventional dataset inference methods, which require an in-distribution held-out set that is rarely available in practice. To overcome this limitation, the authors propose generating synthetic held-out data using a fine-tuned data generator trained on a suffix completion task, designed to mimic the distribution of the suspect dataset. Since synthetic data may introduce distributional shifts relative to natural data, the paper further introduces a post-hoc calibration step that leverages a dual-classifier approach—one classifier distinguishes natural versus synthetic text, while a membership-aware classifier combines textual features with membership signals (e.g., perplexity and token probabilities). The method is evaluated on both single-author datasets (such as blog posts) and larger, heterogeneous collections (e.g., various subsets from the Pile), demonstrating that the synthetic held-out data, once calibrated, can reliably reveal whether a suspect dataset was used in LLM training. The authors also present ablation studies comparing different generation strategies (including in-context paraphrasing, preference optimization, and suffix completion), showing that suffix completion yields the smallest distributional gap.", "questions": "", "limitations": "", "rating": 1, "confidence": null, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": ">**The evaluation might be strengthened by testing on more recent LLM architectures and diverse model sizes.**\n\nWe provide results to demonstrate the effectiveness of our method across larger model sizes of Pythia (2.8B and 6.9B) and different model architecture (Olmo 7B). Please refer to our response to Reviewer z1aa (https://openreview.net/forum?id=a5Kgv47d2e&noteId=z18yFkoW8x) for detailed results.\n\n\n>**Weakness 1 & Question 1: How does your approach for synthesizing held-out data [...] differ from prior methods for synthetic data generation?**\n\nThe goal of our work is not to propose a novel method for synthetic data generation. Instead, our main contributions lie in combining *high-quality and nearly IID synthetic data generation* with *post-hoc calibration* within a framework of dataset inference to provide a practical tool for real-world litigations.\n\nMoreover, we note that directly applying previous generation methods only have limited performance for our task (shown in Table 2). Therefore, we introduce two simple but effective designs tailored for our framework:\n \n(1) **Segmenting and shuffling** author-provided documents: The segmented and shuffled sequences constructs IID generator training and inference splits, facilitating easier synthesis by the data generator. \n\n(2) **Suffix comparison** scheme: Both the suspect and synthetic suffixes share a common prefix, which ensures the two suffixes have the same position within a sentence. This enables direct comparison between original-synthetic suffix pairs, which also supports the difference comparison t-test in our *post-hoc calibration*.\n\nWe present the following results to demonstrate the importance of the two key designs in our held-out data synthesis:\n\n\n| **Approach** | **True Membership** | **AUC_Text (%)** | **AUC_Comb (%)** | **P-value** | **Inferred Membership** |\n|----------------------|-------------------|----------------|-----------------|-------------|------------------------|\n| **w/o segment and shuffle** | ✓ | 72.5 | 45.8 | 1.0 | ✗ |\n|  | ✗ | 72.6 | 42.2 | 1.0 | ✗ |\n| **w/o suffix comparison** | ✓ | 62.8 | 52.9 | 1.0 | ✗ |\n| | ✗ | 62.4 | 50.3 | 1.0 | ✗ |\n| **Ours (w/ segment and shuffle+suffix comparison)** | ✓ | 53.6 | 59.7 | <0.001 | ✓ |\n|  | ✗ | 53.1 | 46.6 | 1.0 | ✗ |\n\n\nWith the two proposed designs, our method has the lowest AUC_Text, indicating it can synthesize nearly IID held-out data. Moreover, the generated held-out set has a significantly higher AUC_Comb on the member set than on the non-member set, which shows our synthetic texts are effective for detecting membership signals.\n\n>**Weakness 2 & Question 2: Clarify the unique aspects of your dual-classifier t-test framework compared to the method in “LLM Dataset Inference”**\n\nIn Section 3, we show that the “LLM Dataset Inference” fails since it uses *the simple null hypothesis* that “the suspect dataset was not used for training” with a t-test that compares the MIA scores between *the suspect and the held-out sets*. \n\nBy introducing *statistical tests based on dual-classifier*, we can statistically distinguish distributional shifts caused by actual membership signals from those caused by generation, and provide more robust detection results. Specifically, our method compares the shifts between suspect and held-out sets *with and without MIA metrics*, which are measured by the text classifier and the combined classifier, respectively.\n\n>**Weakness 3 & Question 3: Sensitivity analysis on hyperparameter settings**\n\n\nIn our proposed framework, the arbitrator, who verifies the training data usage, can choose hyperparameters in both the held-out data generation and post-hoc calibration stages.\n\nIn the **held-out data generation** stage, the arbitrator can choose hyperparameters according to the AUC of the text classifier, where a low AUC_text indicates nearly IID suspect and synthetic held-out texts.\n\nIn the **post-hoc calibration** stage, there are a few important hyperparameters for our method, especially epochs for training the linear classifiers and t-test sample sizes. We present the sensitivity analysis of our method for these hyperparameters below (we use 200 epochs and 2000 sample size in our manuscript). \n\n| **Hyperparameter** | **Value** | **True membership** | **P-value (diff)** | **Inferred membership** |\n|----------------|-------|-----------------|----------------|---------------------|\n| **Epoch** | 100 | ✓ | <0.001 | ✓ |\n|  |  | ✗ | 1.0 | ✗ |\n|  | **200** | ✓ | <0.001 | ✓ |\n|  |  | ✗ | 1.0 | ✗ |\n|  | 500 | ✓ | <0.001 | ✓ |\n|  |  | ✗ | 1.0 | ✗ |\n|  | 1000 | ✓ | 0.003 | ✓ |\n|  |  | ✗ | 1.0 | ✗ |\n| **Num. t-test samples** | 1000 | ✓ | <0.001 | ✓ |\n|  |  | ✗ | 1.0 | ✗ |\n|  | **2000** | ✓ | <0.001 | ✓ |\n|  |  | ✗ | 1.0 | ✗ |\n|  | 3000 | ✓ | <0.001 | ✓ |\n|  |  | ✗ | 0.41 | ✗ |\n|  | 4000 | ✓ | <0.001 | ✓ |\n|  |  | ✗ | 0.25 | ✗ |\n\nThe results show that our method can tolerate a wide range of hyperparameters and gives true positives and true negatives.", "claims_and_evidence": "1. The paper claims to unlock reliable post-hoc dataset inference by synthetically generating an in-distribution held-out set that can serve as a surrogate for real data.\n2. The authors designed data generation pipeline based on a suffix completion task, followed by a post-hoc calibration procedure using a dual-classifier approach.", "methods_and_evaluation_criteria": "The methodology focuses on generating synthetic held-out data from the suspect set using a fine-tuned generator and then aligning the generated and natural data distributions via post-hoc calibration. The dual-classifier setup—combining a text-only classifier with a membership-aware classifier—is used to disentangle genuine membership signals from artifacts caused by distributional shifts.", "theoretical_claims": "The dual-classifier and t-test framework are theoretically motivated by prior work in membership inference attacks.", "experimental_designs_or_analyses": "Experiments are conducted on both a single-author blog dataset and multiple subsets from the Pile dataset. The target models are LLMs (e.g., fine-tuned Pythia 410M) and the synthetic data is generated via multiple strategies, with suffix completion proving most effective.\nHowever, the evaluation might be strengthened by testing on more recent LLM architectures and diverse model sizes.", "relation_to_broader_scientific_literature": "The work is well positioned within the literature on dataset inference, membership inference attacks, and test set contamination.", "essential_references_not_discussed": "Incorporating a comparison with more recent methods for LLM membership inference could strengthen the literature review.", "other_strengths_and_weaknesses": "Strengths:\n1. The paper addresses a practical bottleneck in dataset inference by eliminating the need for a reserved held-out set.\n2. The paper is easy to follow.\n\nWeaknesses:\n1. The approach for synthesizing held-out data, using techniques like suffix completion and preference optimization—does not represent a significant departure from existing methods for synthetic data generation. Similar strategies have been explored in other works.\n2. While the authors claim that their dual-classifier t-test framework uniquely disentangles genuine membership signals from mere distribution shifts, its novelty compared to the approach used in “LLM Dataset Inference: Did you train on my dataset?” remains unclear.\n3. Sensitivity to different LLM architectures and hyperparameter settings is not fully explored.", "other_comments_or_suggestions": "Refer to weaknesses", "questions_for_authors": "1. How does your approach for synthesizing held-out data—using prompted paraphrasing, suffix completion and preference optimization—improve upon or differ from prior methods for synthetic data generation?\n2. Can you clarify the unique aspects of your dual-classifier t-test framework compared to the methodology presented in “LLM Dataset Inference: Did you train on my dataset?”\n3. What is the sensitivity analyses on various LLM architectures and hyperparameter settings?", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "mraimJHhOl", "reviewer": "Reviewer_z1aa", "summary": "This paper presents a method for Unlocking Post-hoc Dataset Inference (DI) with Synthetic Data, for safeguarding intellectual property in the era of Large Language Models. The authors claim that synthetic data generation combined with post-hoc calibration can robustly enable DI, allowing data owners to verify unauthorized data usage in LLM training.", "questions": "", "limitations": "", "rating": 4, "confidence": null, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "We thank the Reviewer for the positive feedback and encouraging comments. \n\n>**However, it may struggle with highly specialized datasets (e.g., medical, legal, or technical texts) where domain-specific patterns might not be captured. This could lead to larger distributional shifts and reduced reliability in these contexts.**\n\nWe would like to note that the Pile is not a homogeneous dataset and has many different data subsets from various domains, formats, and languages. For example, the PubMed Central (PMC) is a subset of the PubMed online repository for biomedical articles (*medical*). The FreeLaw subset contains opinions from federal and state courts (*legal*). The ArXiv subset contains *technical text* mostly from papers in the fields of Math, Computer Science, and Physics. EuroParl is a *multilinguall* corpus consisting of the proceedings of the European Parliament in 21 European languages. StackExchange contains user-posted content on the Stack Exchange network in the *question-answer format* covering a wide range of subjects. We present examples of these synthetic texts from the different subsets of the Pile in the Appendix E. These samples show the diversity of the subsets in the Pile. Here, we also demonstrate that our method generalizes well across different pile subsets and model parameter sizes.\n \nWe further extend our analysis beyond the Pile and test our method on the Dolma dataset, which was built from a diverse mixture of web content, scientific papers, code, public-domain books, social media, and encyclopedic materials. We run additional experiments using Olmo 7B, with the member data from Dolma-v1_7 and the non-member data from Paloma. We observe that our method is also effective for the different data types. The results are provided in the table below (following Table 4 in our submission):\n\n| **Model** | **Subset** | **True** | **AUC Text (%)** | **AUC Comb (%)** | **P-value (Diff)** | **Inferred** |\n|-------|--------|------|--------------|--------------|----------------|----------|\n| **Pythia-2.8B** | ArXiv | ✓ | 53.0 | 59.1 | <0.001 | ✓ |\n|  |  | ✗ | 53.1 | 53.3 | 0.84 | ✗ |\n|  | StackExchange | ✓ | 53.7 | 57.7 | <0.001 | ✓ |\n|  |  | ✗ | 52.3 | 53.7 | 0.18 | ✗ |\n|  | EuroParl | ✓ | 50.5 | 54.0 | 0.009 | ✓ |\n|  |  | ✗ | 51.1 | 52.3 | 0.17 | ✗ |\n| **Pythia-6.9B** | ArXiv | ✓ | 53.6 | 60.0 | <0.001 | ✓ |\n|  |  | ✗ | 53.5 | 54.1 | 0.37 | ✗ |\n|  | StackExchange | ✓ | 53.7 | 58.4 | <0.001 | ✓ |\n|  |  | ✗ | 52.4 | 53.1 | 0.21 | ✗ |\n|  | EuroParl | ✓ | 50.8 | 55.8 | <0.001 | ✓ |\n|  |  | ✗ | 50.7 | 53.4 | 0.11 | ✗ |\n| **Olmo-7B** | Wikipedia | ✓ | 52.9 | 55.4 | 0.009 | ✓ |\n|  |  | ✗ | 52.1 | 50.6 | 1.0 | ✗ |\n|  | Common Crawl | ✓ | 53.5 | 55.7 | 0.01 | ✓ |\n|  |  | ✗ | 54.2 | 53.8 | 0.68 | ✗ |", "claims_and_evidence": "The claims made in the submission are well-supported by experimental evidence.", "methods_and_evaluation_criteria": "The proposed methods and evaluation criteria are well-designed and appropriate for the problem of post-hoc dataset inference. The authors provide clear and convincing evidence that their approach effectively addresses the challenges of DI.", "theoretical_claims": "All theoretical claims have been checked and seem valid.", "experimental_designs_or_analyses": "The authors evaluate their method on single-author blog posts and subsets of the Pile dataset, which include diverse text types (e.g., Wikipedia, ArXiv, GitHub). These datasets are representative of real-world scenarios where DI might be applied, such as copyright claims for blog posts or academic papers.", "relation_to_broader_scientific_literature": "Previous work on Dataset Inference by Maini et al. (2021) provides a method to determine whether a suspect dataset was used to train a machine-learning model. However, DI traditionally requires an in-distribution held-out dataset, which is rarely available in practice. The paper addresses the critical limitation of DI by proposing a method to synthetically generate in-distribution held-out data.", "essential_references_not_discussed": "To the best of my knowledge, there are no essential references that are not discussed.", "other_strengths_and_weaknesses": "Strengths:\nThe paper is well-written and contains extensive experimental results. \nIt touches on an important topic of safeguarding intellectual property against Large Language Models.\n\nWeaknesses:\nThe method is evaluated on single-author blog posts and subsets of the Pile dataset, which are relatively homogeneous or well-structured. However, it may struggle with highly specialized datasets (e.g., medical, legal, or technical texts) where domain-specific patterns might not be captured. This could lead to larger distributional shifts and reduced reliability in these contexts.", "other_comments_or_suggestions": "No other comments.", "questions_for_authors": "No other questions.", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "zVspNVoJPt", "reviewer": "Reviewer_hUkh", "summary": "The work presents a method to generate synthetic data for use with Dataset Inference (DI) algorithms, which require held-out examples from the dataset distribution of interest. DI algorithms are used to detect the presence of certain data distributions within the training data of a particular model.  The authors argue that a (non-synthetic) held out dataset is rarely available and that even small shifts in the distribution of a held out set can cause DI algorithms to fail. Furthermore, the authors define a \"post-hoc calibrated\" test procedure, to use in conjunction with this synthetic dataset, to identify when a suspected dataset has been included in a model's training set.", "questions": "", "limitations": "", "rating": 3, "confidence": null, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "> **Is the result in table 5 only for the single-author dataset? Do similar trends also hold for the Pile dataset?**\n\nThe results in Table 5 are for the Pile. Below, we also show that our data generation procedure is necessary for the single-author dataset.\n\n| Configuration | True Membership | P-value | Inferred Membership |\n|--------------|----------------|---------|---------------------|\n| **w/o Suffix Completion (ICL Paraphrasing)** | ✓ | 1.0 | ✗ |\n| | ✗ | 1.0 | ✗ |\n| **w/o Post-hoc Calibration (Original T-test in DI)** | ✓ | <0.001 | ✓ |\n| | ✗ | <0.001 | ✓ |\n| **w/o Weight Constraint** | ✓ | 0.02 | ✓ |\n| | ✗ | 0.08 | ✗ |\n| **Ours** | ✓ | 0.01 | ✓ |\n| | ✗ | 0.13 | ✗ |\n\nThe results for single-author data follow a similar trend as the Pile, showing the importance of each component in our framework.\n\n>**The authors do not mention what blog posts they use. Could this be clarified?**\n\nThe blog posts are from the financial domain, regarding stock investment suggestions. Due to copyright reasons, the blog post data is not available for sharing yet.\n\n>**Additional model sizes (2B, 7B) would help illustrate the scaling behavior.**\n\nWe provide the results to demonstrate the effectiveness of our method across larger model sizes of Pythia (2.8B and 6.9B) and on different model architecture (Olmo 7B). Please refer to our response to Reviewer z1aa (https://openreview.net/forum?id=a5Kgv47d2e&noteId=z18yFkoW8x) for detailed results.\n\n>**Another simple data generation baseline, which should be considered, is an in-context suffix completion approach.**\n\nWe performed this baseline in Appendix A.1 “Text Completion Prompt”. Concretely, as an approach of in-context learning, we give ten examples from the documents, and prompt the GPT-4-Turbo model to finish an incomplete document. This baseline performs slightly worse than ICL paraphrasing, and the GPT2 classifier can easily detect the distributional shift with an AUC of 99.2%.\n\n>  **How critical is the synthetic data generation approach given contribution (2)?**\n\nBesides the results in Table 2 and Appendix A, we also extended the following results in Table 5 to show the importance of our data generation approach. Concretely, we show that all the generation baselines do not work even with our self-calibration methods (Row 1-6). Moreover, we show that each component in our generation procedure is critical to our framework (row 7-10).\n\n| Configuration | True Membership | P-value | Inferred Membership |\n|--------------|----------------|---------|---------------------|\n| **w/o Suffix Completion (ICL Text Completion)** | ✓ | 1.0 | ✗ |\n| | ✗ | 1.0 | ✗ |\n| **w/o Suffix Completion (ICL Paraphrasing)** | ✓ | 1.0 | ✗ |\n| | ✗ | 1.0 | ✗ |\n| **w/o Suffix Completion (Performance Optimization)** | ✓ | 1.0 | ✗ |\n| | ✗ | 1.0 | ✗ |\n| **w/o segment and shuffle** | ✓ | 1.0 | ✗ |\n| | ✗ | 1.0 | ✗ |\n| **w/o suffix completion** | ✓ | 1.0 | ✗ |\n| | ✗ | 1.0 | ✗ |\n| **Ours** | ✓ | <0.001 | ✓ |\n| | ✗ | 1.0 | ✗ |\n\n\n\n> **should we use distribution testing to ensure that the suspect and held-out set are statistically similar as a standard part of the DI protocol?**\n\n\nAdding distributional testing as part of DI protocol only strengthens its *robustness* but not *utility*. When a distributional shift is identified, it is still hard for the author to provide held-out data required for DI (as shown in Sec. 3). This disables further DI procedures, even when there is a copyright infringement. This is exactly the problem we would like to address in this work—instead of asking the author to provide statistically similar held-out data, **the arbitrator should be able to synthesize a reliable held-out set**. We think this should be a standard protocol for dataset inference and visualize it in Figure 1.\n\n> **\"The null hypothesis is that the average scores for $D_{sus}$ is lower than for $D_{val}$\" Shouldn't the null hypothesis (eq 1) be that the scores for the two distributions are the same?**\n\nThe text should be “The **alternative** hypothesis is that the average scores for $D_{sus}$ is lower than for $D_{val}$”. If $D_{sus}$ is trained on the target model, then its MIA scores (e.g. loss) should be lower than the IID held-out set $D_{val}$. The null hypothesis, against the alternative hypothesis, is that “the average scores for $D_{sus}$ is higher than or the same as those for $D_{val}$”. This also aligns with Eq. 1.\n\n>**Does the inferred column in Table 4 need to be updated to indicate some of the errors, for example in the case of Ubuntu IRC it looks like there is a false positive?**\n\nAs pointed out in Sec. 5.3, although the **difference comparison** t-test works for all subsets, the **AUC comparison** t-test fails for three subsets: Ubuntu IRC, PubMed Central, and PhilPapers. Therefore, we suggest the difference comparison t-test as a better metric. We have also modified the table accordingly.\n\n> **Suggestions for figures and symbols**\n\nWe thank the suggestions from the reviewer, and made changes as suggested.", "claims_and_evidence": "The submission claims two contributions, the procedure for generating synthetic held-out data and a second procedure for testing whether the suspect data was included in the target model's training set via a \"post-hoc calibration\".  The impact of the post-hoc calibration seems quite well supported by the empirical evaluation of Section 5. The necessity for a more complex data generation procedure is somewhat supported in Table 5, but it is unclear from the text, is this result only for the single-author blogpost dataset? Do similar trends also hold for datasets from the Pile?", "methods_and_evaluation_criteria": "The benchmark datasets (The Pile and blog posts) are reasonable, however, the authors do not mention what blog posts they use.  Could this be clarified?\n\nThe model (Pythia 410M) is relatively small so there is some question as to how well the method performs with model scale.  Additional model sizes (2B, 7B) would help illustrate the scaling behavior.\n\nAnother simple data generation baseline, which should be considered, is an in-context suffix completion approach. One where you prompt the model to complete a suffix (selected in the same fashion as done for the proposed suffix completion approach). Furthermore, a few full examples could be included in the context to give the model some guidance on the writing style.", "theoretical_claims": "No theoretical claims were made.", "experimental_designs_or_analyses": "The empirical methodology seems sound (apart from my comments related to methods and evaluation criteria).", "relation_to_broader_scientific_literature": "The paper does a reasonable job of introducing DI concepts and related works.", "essential_references_not_discussed": "N/A", "other_strengths_and_weaknesses": "As presented, the paper presents two main contributions -- (1) how to generate synthetic data for solving the DI problem and (2) how to conduct tests that take into account the synthetic data.  The paper could be strengthened and made more clear, if there are additional experiments to support (1), i.e., how critical is the synthetic data generation approach given (2).  If it is indeed critical, it would be great to have further empirical support and if it is not important (e.g. any reasonable data generation approach works well when coupled with (2)), then the paper should be updated to emphasize (2) more, in my opinion.", "other_comments_or_suggestions": "- As motivated by sec 3.3, using a bad (i.e. out of distribution) held-out set can create false positives...this begs the question, should we use distribution testing to ensure that the suspect and held out set are statistically similar as a standard part of the DI protocol?\n\n-  \"The null hypothesis is that the average scores for D_sus is lower than for D_val\"\nShouldn't the null hypothesis (eq 1) be that the scores for the two distributions are the same, i.e. in the case that D_sus and D_val are drawn from the same distribution and neither has been included in the training set?  If we find D_sus has a much higher score, then we reject the null and conclude D_sus was included in the training set.\n\n- Please make clear earlier that larger MIA score implies the datapoint is included in the training set. At least, that is what I inferred from how MIA is being used in Eq (1).\n\n- Figure 3, left-panel, step 2: all the text snippets in the illustration have identical text. Is this intended?\n\n- This is minor, but I found the use of red checks to indicate positive and green/blue crosses to indicate negative confusing, since I usually think of red to indicate negative and green to indicate positive. I suggest using green checks and red crosses.\n\n- Does the inferred column in Table 4 need to be updated to indicate some of the errors, for example in the case of Ubuntu IRC it looks like there is a false positive?", "questions_for_authors": "(1) Can you provide some (even preliminary) empirical results for using an in-context suffix completion approach (as suggested in the Experimental Design section above)?  This would serve as an ablation to measure the effectiveness of suffix completion with LoRA vs using in-context instructions.\n\n(2) Can you provide additional experiments showing that the synthetic generation method is necessary (and that the post-hoc calibration alone is not sufficient)? Essentially, expanding the result of Table 5.", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "kU216K7ZiJ", "reviewer": "Reviewer_ZFHo", "summary": "The paper proposes a novel approach to dataset inference (DI) by addressing the challenge of requiring a held-out dataset that closely matches the suspect dataset’s distribution. The authors generate synthetic held-out data using a text generator trained on a suffix-based completion task. They further introduce a post-hoc calibration step to bridge the likelihood gaps between real and synthetic data, improving the reliability of DI. Experiments on diverse datasets demonstrate that this method enables accurate detection of whether a dataset was used in model training while minimizing false positives, making it a viable tool for copyright enforcement and data ownership verification.", "questions": "", "limitations": "", "rating": 3, "confidence": null, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "> **The limited model scale and dataset size constrain the generalizability of the conclusions.**\n\nWe performed additional experiments on larger Pythia models (2.8B, 6.9B). Moreover, we present results on Olmo 7B, which is trained on a very large training corpus of 4.5TB. The results show our method can scale to much larger models and dataset sizes. Please refer to our response to Reviewer z1aa (https://openreview.net/forum?id=a5Kgv47d2e&noteId=z18yFkoW8x) for detailed results.\n\n> **The justification for hypothesis testing, particularly Equation (3), lacks sufficient reasoning. [...] Specifically, what is the range of values for c(⋅)? Discrete or continuous?**\n\nThe difference comparison t-test in Eq. 3 is designed following our proposed suffix completion generation task. In the suffix completion task, the suspect and generated heldout suffixes have the same position within a sentence, making the two suffixes directly comparable. To this end, we propose to quantify the distributional gap in each suspect-heldout suffix pair. The gap is measured *with only text signals* (by the text classifier), and *with both text and MIA signals* (by the combined classifier). If the gaps in suspect-heldout pairs are larger with extra MIA signals, then the suspect suffixes are likely to be used for training. \n\nSpecifically, the values for c(⋅) are continuous, ranging from 0 to 1.\n\n> **There appears to be a contradiction between the first paragraph of Section 2.2 and Equation (1).**\n\nWe modified the text in Section 2.2 as follows:\n“The null hypothesis is that the average scores for $D_{sus}$ **are higher than or equal to** those for $D_{val}$.”\n\n> **The fine-tuning setup is narrow (only one recipe is tested)**\n\nWe performed additional experiments under two more fine-tuning setups, (1) finetuning with LoRA for 10 epochs and (2) full-finetuning, as follows.\n\n| Fine-tuning Method | True | AUC Text (%) | AUC Comb (%) | P-value (Diff) | Inferred |\n|-------------------|------|--------------|--------------|----------------|----------|\n| **LoRA (1 epoch)** *(in Table 3)*  | ✓ | 53.8 | 55.6 | 0.01 | ✓ |\n| | ✗ | 53.8 | 53.9 | 0.13 | ✗ |\n| **LoRA (10 epochs)** | ✓ | 53.7 | 56.2 | 0.005 | ✓ |\n| | ✗ | 53.6 | 53.5 | 0.14 | ✗ |\n| **Full-finetuning** | ✓ | 53.7 | 56.8 | 0.008 | ✓ |\n| | ✗ | 53.8 | 53.7 | 0.21 | ✗ |\n\nOur results suggest that, with more iterations or larger parameter size, the membership signal is stronger and therefore easier to detect. This means that the setup in our submission is the most challenging one.\n\n> **Only two specific classifiers are tested without exploring stronger alternatives.**\n\nWe explored a few stronger text classifier backbones and chose the simple 2-layer GPT2-based classifier.\nConsidering the limited number of tokens provided by the author in the DI scenario, stronger text classifiers can be easily overfitted, especially for SoTA LLM-based text classifiers. Here, we present the results for different architectures with different parameter sizes.\n\n\n| Architecture | AUC_text (%) | Training time (minutes) |\n|-------------|------------|------------------------|\n| **GPT2 (initialized+2 layers)** | 53.3 | 0.5 |\n| **GPT2 (pretrained+lora)** | 53.0 | 26.2 |\n| **GPT2 (pretrained+full finetuning)** | 52.3 | 36.8 |\n| **Llama3-8B (pretrained+lora)** | 53.2 | 65.1 |\n\n The results show that the simple GPT2-based classifier ((initialized+2 layers) can achieve the best AUC. Additionally, this simple classifier has a significantly shorter training time, making the method more practical when faced with more queries.\n\n> **Typo: Line 283 and Line 310**\n\nWe have modified those texts in our manuscript.\n\n> **Can you provide a more detailed explanation of the rationale in Section 4.3? A concrete example would be helpful.**\n\nWe added a more detailed explanation with an example in the manuscript for better understanding. We also added a figure to visually demonstrate the idea. \n\n> **Can your method defend against a malicious actor?**\n\nThe existence of a malicious actor is one of the important problems we would like to address for the previous LLM Dataset Inference—the author might provide a held-out set with a distributional shift to trigger a false positive result. With our approach, **the arbitrator can synthesize a reliable held-out set** to avoid false positives caused by a malicious user. We think this should be a standard pipeline for dataset inference and visualize it in Figure 1.\n\n> **In the ablation study, you included ICL paraphrasing. I am curious about the results when using preference optimization for generation.**\n\nWe extended Table 5 and provided extra results of three baselines (preference optimization, ICL text completion, ICL paraphrasing) together with two more ablation studies for each component for our generation method. Please refer to the second table in our response to reviewer hUkh (https://openreview.net/forum?id=a5Kgv47d2e&noteId=aEzlhvnQm9) for more detailed results.", "claims_and_evidence": "I am confused by Section 4.3, and I believe more details should be added to improve the clarity of the claims.", "methods_and_evaluation_criteria": "The proposed methods and evaluation criteria are reasonable (Section 5.1), but the limited model scale and dataset size constrain the generalizability of the conclusions.", "theoretical_claims": "The paper does not present theoretical claims, however: \n1. The justification for hypothesis testing, particularly Equation (3), lacks sufficient reasoning. \n2. There appears to be a contradiction between the first paragraph of Section 2.2 and Equation (1), suggesting an error in one of them. \n3. The explanation in Section 4.3 is unclear.", "experimental_designs_or_analyses": "The experimental design has limitations due to the small model scale and dataset size. The analysis is reasonable, but the necessity of the weight constraint is not clearly demonstrated based on the ablation in Section 5.5.", "relation_to_broader_scientific_literature": "The paper addresses a key limitation of prior DI methods by eliminating the need for an in-distribution held-out dataset, making DI more practical and reliable.", "essential_references_not_discussed": "I am not deeply familiar with the literature on DI, but I am well-versed in MIA research. The paper's coverage of MIA-related works is adequate.", "other_strengths_and_weaknesses": "**Strengths**:\n1. The research problem is important and addresses a clear gap in existing dataset inference methods.\n2. The approach is interesting, particularly the calibration step for handling distribution shifts.\n3. The current experimental results are generally effective.\n\n**Weaknesses**:\n1. The experimental scale is limited, involving only small models and two datasets. The fine-tuning setup is also narrow (only one recipe is tested), limiting broader MIA evaluations. Additionally, the evaluation of text classifiers is limited, as only two specific classifiers are tested without exploring stronger alternatives.\n2. The writing lacks clarity in several areas. There is a contradiction between Section 2.2’s first paragraph and Equation (1). Sections 4.2 and 4.3 are unclear, particularly the reasoning behind Equation (3) and the rationale of the weight constraint.\n\nI believe expanding the experiments would strengthen the validation of the method.", "other_comments_or_suggestions": "**Typo**: \n1. Line 283: \"we rely on an important observations...\"\n2. Line 310: A \"]\" is missing\n\n**Suggestions**:\n1. I suggest giving a concrete example of MIA(f(x)), which can help readers understand the context better.\n2. I strongly suggest improving the clarity of Section 4.2 and Section 4.3.", "questions_for_authors": "1. Can you clearly explain the relationship between $MIA(x_{val})$ and $MIA(x_{sus})$? There seems to be a contradiction in Section 2.2.  \n2. What is the rationale behind Equation (3)? Specifically, what is the range of values for $c(\\cdot)$? Discrete or continuous?  \n3. Can you provide a more detailed explanation of the rationale in Section 4.3? Why does the linear regression coefficient become negative? A concrete example would be helpful.  \n4. Have you considered using stronger text classifiers beyond the two currently used?  \n5. Can your method defend against a malicious actor? Please discuss potential vulnerabilities.  \n6. How does your method perform under different fine-tuning recipes? My understanding is that you tested only one setting.  \n7. In the ablation study, you included ICL paraphrasing. I am curious about the results when using preference optimization for generation.", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}]

Author rebuttal:

Meta-review: This paper presents a framework for dataset inference (DI) in large language models, addressing the practical limitation of needing a real in-distribution held-out set. The proposed approach combines suffix-based synthetic data generation with a post-hoc calibration procedure using a dual-classifier t-test. 

After the rebuttal, three reviewers recommended acceptance, citing the clarity of the presentation, the importance of the problem, and the expanded experimental scope. One reviewer remained negative, raising three points: (i) limited novelty in the data‑synthesis component, (ii) missing hyper‑parameter ablations, and (iii) insufficient differentiation from a related paper. The authors’ rebuttal satisfactorily addressed points (ii) and (iii); the meta‑reviewer further agreed with the authors' argument that the work’s goal is to harness - not advance - data‑synthesis methods for DI, which mitigates concern (i).

Overall, the framework is a meaningful contribution to practical DI scenarios, though the authors are encouraged to further improve the clarity of the methodological exposition as recommended by one of the reviewers.
Decision: Accept (poster)