Fine-tuning can Help Detect Pretraining Data from Large Language Models
Authors: Hengxiang Zhang
Songxin Zhang
Bingyi Jing
Hongxin Wei
Venue: ICLR 2025 Poster (2025)
Keywords: Large language models
Fine-tuning
Pretraining data detection
Primary area: foundation or frontier models, including LLMs
TL;DR: We propose Fine-tuned Score Deviation (FSD), a novel and effective approach that improves the detection capabilities of current methods for pretraining data detection.

Abstract: In the era of large language models (LLMs), detecting pretraining data has been increasingly important due to concerns about fair evaluation and ethical risks. Current methods differentiate members and non-members by designing scoring functions, like Perplexity and Min-k%. However, the diversity and complexity of training data magnifies the difficulty of distinguishing, leading to suboptimal performance in detecting pretraining data. In this paper, we first explore the benefits of unseen data, which can be easily collected after the release of the LLM. We find that the perplexities of LLMs shift differently for members and non-members, after fine-tuning with a small amount of previously unseen data. In light of this, we introduce a novel and effective method termed Fine-tuned Score Deviation (FSD), which improves the performance of current scoring functions for pretraining data detection. In particular, we propose to measure the deviation distance of current scores after fine-tuning on a small amount of unseen data within the same domain. In effect, using a few unseen data can largely decrease the scores of all non-members, leading to a larger deviation distance than members. Extensive experiments demonstrate the effectiveness of our method, significantly improving the AUC score on common benchmark datasets across various models.

Published as a conference paper at ICLR 2025
FINE-TUNING CAN HELP DETECT PRETRAINING DATA FROM LARGE LANGUAGE MODELS
Hengxiang Zhang', Songxin Zhang', Bingyi Jing', Hongxin Wei!* Department of Statistics and Data Science, Southern University of Science and Technology
ABSTRACT
In the era of large language models (LLMs), detecting pretraining data has been increasingly important due to concerns about fair evaluation and ethical risks. Current methods differentiate members and non-members by designing scoring functions, like Perplexity and Min-k%. However, the diversity and complexity training data magnifies the difficulty of distinguishing, leading to suboptimal performance in detecting pretraining data. In this paper, first explore the bene- fits of unseen data, which can be easily collected after the release the LLM. We find that the perplexities of LLMS shift differently for members and non-members, after fine-tuning with a small amount of previously unseen data. In light of this, we introduce a novel and effective method termed Fine-tuned Score Deviation (FSD), which improves the performance of current scoring functions for pretrain- ing data detection. In particular, we propose to measure the deviation distance of current scores after fine-tuning on small amount of unseen data within the same domain. In effect, using a few unseen data can largely decrease the scores of all non-members, leading to larger deviation distance than members. Ex- tensive experiments demonstrate the effectiveness of our method, significantly improving the AUC score common benchmark datasets across various mod- els. Our code available at https:/ /tts://gghhtbb com/ /mlmmmlssttt// stalsssat--ae SataatttSusehhhh Pine-tunede Seordd--ooe----eeeee
INTRODUCTION
The impressive performance of large language models (LLMs) arises from large-scale pretraining on massive datasets collected from the internet (Achiam et al., 2023; Touvron et al., 2023b). But, model developers are often reluctant to disclose detailed information about the pretraining datasets, raising significant concerns regarding fair evaluation and ethical risks. Specifically, Recent studies reveal that the pretraining corpus may inadvertently include data from evaluation benchmarks (Sainz et al.. 2023; Balloccu et al., 2024), making it difficult to assess the practical capability of LLMs. Besides, L.MS often generate text from copyrighted books (Grynbaum & Mac, 2023) and personal emails (Mozes al., 2023), which could infringe the legal rights of the original content creators and violate their privacy. Considering the vast size the pretraining dataset and the single iteration of pretraining, it has been increasingly important and challenging detect pretraining data, which determines whether a piece of text is part of the pretraining dataset.
In the literature, current works of detecting pretraining data primarily focus on designing scoring functions to differentiate members (i.e., seen data during pretraining) and non-members (unseen). For example, previous work shows that sequences from the training data tend have lower per- plexity (i.e., higher likelihood) than non-members (Li, 2023). Min-k% leverages the k% of tokens with minimum token probabilities of a text for detection, assuming that trained data tends to contain fewer outlier tokens (Shi et al., 2024). However, non-member data can obtain low perplexities by including frequent or repetitive texts, while members may contain rare tokens that result in high per- plexities. This casts significant doubt utilizing those scoring functions for detecting pretraining data. Consequently, this issue prompts us to present a preliminary attempt to enlarge the difference between members and non members for pretraining data detection.
"Corresponding author (wei hx@sustech, edu..c cn)

---

Published as a conference paper at ICLR 2025
centenary outhreak World commemoruted Europe
Pre- trsined Model
Pine- tune- Model
Non- members
Fine- turing
Fine-tuned Score Deviation
O20 2014
S(x) fa) S0)
03 C6 09
Member
Figure 1: Overview of Fine-tuned Score Deviation. Our method first fine-tunes the pre- pre-trained model with a few non-members and then measures the deviation distance of scores from the pre- trained and fine-tuned models membership inference metric. If the deviation value is smaller than the threshold value, the text is likely in the pretraining data.
In this work, propose Fine-tuned Score Deviation (FSD), novel and effective approach that improves the detection capabilities of current scoring functions in a specific domain (c.g., event data from Wikipedia, arXiv research papers). Our method is motivated by an empirical analysis of the perplexity deviation after model fine-tuning. We find that when fine-tuned with few previously unseen data from specific domain, the perplexities of L.LMs experience significantly larger de- crease for other unknown non-members in the domain compared to the members. This suggests the possibility of using the disparity to distinguish between members and non-members.
Therefore, our key idea behind FSD is to enlarge the score deviation between members and non- members by exposing the LLM to a few non members. This can accomplished by measuring the deviation distance of current scores (See Figure 1), owing to the self-supervised fine-tuning non non-members. effect, the fine-tuning largely decreases the scores non member data, resulting in more distinguishable amount of unseen data for LLMs within a specific domain. For example, we can make use of those contents (e.g., journal articles) published subsequent to the release of the LLM.
few
and unseen data. In practical applications, it is easy collect small
To validate the effectiveness of our method, we conduct extensive experiments on various datasets, including WikiMIA, BookMIA (Shi al., 2024), ArXivTection, BookTection (Duarte et al., 2024) and Pile (Maini et al., 2024). The results demonstrate that our method can significantly improve the performance of existing methods based on scoring functions. For example, our method improves the AUC score of the best baseline method Min-k%, increasing it from 0.62 to 0.91 on WikiMIA under the OPT-6.7B model. Moreover, our method can also improve the TPR @ FPR score of baseline methods. For example, our method improves the TPR @ FPR score of the detection method using perplexity, increasing it from O. 10 to 0.81 on ArXivTection under the LLaMA-7B model.
Our main contributions are follows:
We analyze the limitations of existing methods based scoring functions for pretraining data detection. The significant overlap in metric score distribution between seen data and unseen data results in the inferior performance of detection methods.
We propose Fine-tuned Score Deviation (FSD), a novel and effective method for detecting pretraining data from large language models. The core idea enlarge the gap between members and non-mem memmerrmerr by exposing the LLM to few unseen data.
We empirically show that FSD can improve the performance of existing methods based on scoring functions for pretraining data detection, through extensive experiments conducted on various benchmark datasets with diverse LLMs.
2

---

Published as a conference paper at ICLR 2025
2 BACKGROUND
In this work, we focus on detecting pretraining data, the problem of detecting whether piece of text included in the pretraining data of specific LLM. First, we formally define the problem setup and its challenges. Then, we introduce two commonly used methods for this task.
Pretraining data detection Pretraining data detection is an instance of membership inference attacks (MIAs) (Shokri et al., 2017), and can help identify data contamination in the pretraining corpus (Shi et al., 2024). Let f be an autoregressive large language model (LLM) with trainable parameters 0 (e.g.. LLaMA (Touvron et al., 2023a)) and 'D denotes the associated pretraining data, sampled from an underlying distribution P. As model developers rarely provide detailed information about the pretraining dataset, we generally desire to identify if the LLM is trained the given text for scientific and ethical concerns, Formally, the task objective is to learn a detector h that can infer the membership of an arbitrary data point æ in the dataset D: h(z, fo) {0, 1}.
Unlike the black-box box assumption in previous works (Shi et al., 2024; Oren et al., 2024), we assume the access fine-tune LLMS with custom datasets and the output probabilities of LLMs, which is realistic for spen-sourced LLMs and many commercial APIs, such as GPT-4o'. In addition, the detector can obtain few data samples {ax²] 1. that belong to the same domain as the given sample and do not present in the training set. This can be achieved by collecting those contents (e.g.. journal articles) published after the release the LLM.
The task of pretraining data detection can be formulated as a binary classification: determining whether given text æ is a member or non-member of the pretraining dataset D. Pretraining data detection can be performed by a level-set estimation:
member
if S(
x fe) < E, if Sf : fo) 2
ha:; fo)
non-member
where S(x; fo) denotes a scoring function and c is the threshold determined by validation dataset. By convention, examples with lower scores S(x; fo) are classified as members of pretraining data and vice versa. In the following, we introduce two popular scoring functions for the task.
Scoring functions For large language models, likelihood is typically used to estimate the uncer- tainty in generating new tokens. In particular, a high likelihood indicates that the model predicts tokens with high confidence. Given a piece of text æ = { I2, ..,,... }, the likelihood of the next token In+1 is pe (I (21, ).). In general, piece of text seen in pre-training tends to have more tokens with a high likelihood, whereas unseen texts have more tokens with low likelihood.
In light of this, previous studies usually design rikeiihodd-based scoring functions to detect pretrain- ing data (Shi et al., 2024; Carlini et al., 2021; Li, 2023). For example, Perplexity is proposed to distinguish members and monmmeebers, based on the observation that members tend to have lower perplexity than non-members (Li, 2023). Formally, The perplexity of æ is calculated as:
Perplexity(#; fo) = expl 1 Σ log (2, (1,.... 2;
where x = { 22,1, , vn) is a sequence of tokens and po po( "....,,;;;))) 1) is the conditional probability of x; given the preceding tokens.
Instead of using the likelihood of all tokens, Min-k% (Shi et al., 2024) computes the average prob- abilities of k% outlier tokens with the smallest predicted probability. The intuition is that non- member example is more likely to include a few outlier words with low likelihoods than members. Formally, Min-k% is computed by:
1
Min-k% fo)
log I1, I1,...; Z1-1)
E
(x))
where E is the size of the Min-k%(x) set.
"https://platffrmm.oo
3

---

Published as a conference paper at ICLR 2025
However, non-member data obtain low perplexities by including frequent repetitive texts, while members may contain rare tokens that result in high perplexities (See Figure 3a and 3b). This issue makes it challenging distinguish members and non-members using those scoring functions, leading to suboptimal performance in detecting pre-training data. Thus, we present preliminary attempt utilize extra non-member data to enlarge the gap between members and non-members.
3
METHOD
Recall the realistic assumption that detectors obtain few non-members that belong to the same domain as the given sample, we aim to explore how to utilize these extra non nemberemrrs to improve the detection. Thus, start by investigating the effects of LLM fine-tuning with unseen examples. Our analysis shows that fine-tuning exerts different impacts on members and non-members.
3.1 MOTIVATION
In the analysis, conduct experiments with WikiMIA (Shi al., 2024), evaluation benchmark that uses events added Wikipedia after specific dates as non non-member data. We use D to denote the non-member dataset that is accessible for detectors. To construct the dataset D, we randomly sample subset with 100 examples from the non-member data of WikiMIA. In addition, we construct the test set with 630 examples each for both members and non-members. Throughout this subsection, fine-tune LLaMA-7B (Touvron et al., 2023a) with LoRA (Hu al., 2022) the non mommeeeer dataset D. To illustrate the effects of fine-tuning, we compare the perplexity distribution of members and non-members from the pre-trained model and the fine-tuned model.
0.30
0.30
pre
pro trainra
0.25
fine tunedtuned
0.25
fine tuned
0.20
0.20
Density
Density
0.15
0.15
0.10
0.10
0.05
0.05
0.00
0.00
10
14
12
14
Perplexity
Perplexity
(a) Members
(b) Non-members
Figure 2: The perplexity distribution from the pre-trained model and the fine-tuned model.
Fine-tuning decreases the perplexity of non-members. Figures 2a and 2b present the deviation of perplexity distributions for members and non-members, throughout the fine-tuning on the non- member dataset D. The results show that unseen data in the pretraining tend to obtain a lower perplexity from the fine-tuned model than the pre-trained model. In contrast, we observe that the deviation of the perplexity distribution for members negligible after fine-tuning. The analysis indicates that fine-tuning with a few unseen data from specific domain can decrease the likelihood- based scores of the LLM for other unknown non-members in the domain. The contrast the score deviation resulting from fine-tuning allows for the distinction between members and non-members.
3.2 FSD: FINE-TUNED SCORE DEVIATION
Motivated by the previous analysis, we propose Fine- tuned Score Deviation (FSD), a general method that improve the detection performance of current scoring functions in specific domain. The key idea of our method is to enlarge the gap between seen and unseen data, by exposing the LLM to few unseen data. With this in mind, we present the details of approach step by step.
4

---

Published as a conference paper at ICLR 2025
Construct fine-tuning dataset Given piece of text æ, the first step of method is collect a small amount of unseen data for the LLM within the same domain. Owing to the availability of construct non-member datasets by comparing the [.M release date and data creation timestamp. For instance, we collect some events occurring post- 2023 from Wikipedia the auxiliary non-member dataset for fine-tuning LLaMA (Touvron et al., 2023a), since LLaMA was released in February 2023.
public text data in enormous quantities,
Fine-tuning with non-members To expose LLMs to unseen data, we perform fine-tuning on L.MSs with the constructed fine-tuning dataset. As our goal is to reduce the perplexity of the un- seen data, we employ self-supervised fine-tuning by predicting the next word or token given sequence. In particular, we build the loss function by decreasing the negative log likelihood of the actual next token in the sequence. Formally, the fine-tuning loss is:
L Linee tining (x)
Σ log (1.,)
(4)
Fine-tuned Score Deviation Recall that fine-tuning decreases the perplexity of non-mennen members but almost maintains those of members, we propose to exploit the score deviation for detecting pretrain- ing data. Given a new sample æ. we calculate the score difference between the pre-trained LLM fo and the fine-tuned LLM for, where O' denotes the parameters of LLM after fine-tuning. Formally, the new score of Fine-tuned Score Deviation (FSD) can be formulated as:
FSD(#; So. Jor) = S(æ; fo) - S(æ;;
(5)
where S(.) denotes an existing scoring function, such as Perplexity and Min-k%. With the proposed score, we can estimate the membership of æ through the level-set estimation (Eq. (1)). Examples with a large deviation score are considered as non-members and vice versa. In practice, we deter- mine the threshold by maximizing detection accuracy on validation set, following the previous work (Shi et al., 2024). Our method is compatible with various scoring functions and consistently enhances their performance in detecting pretraining data, as presented in Table 1.
By way of the FSD score, we can obtain a clear distinction between members and non-members, establishing excellent performance in detecting pretraining data. To provide straightforward view, we show in Figure 3 the score distribution between members and non-members using various scoring functions on WikiMIA (Shi et al., 2024). The results of ArXivTection (Duarte al., 2024) are also presented in Appendix D.I. Our experiments validate that, compared to the perplexity and Min-k% scores, our FSD score significantly increases the gap between non- members and members, and as result, enables more effective pretraining data detection.
4 EXPERIMENTS
In this section, we evaluate the effectiveness of our method for pretraining data detection across several benchmark datasets with multiple existing open-sourced models. We also apply FSD to copyrighted book detection in real-world scenarios and find it consistently effective solution.
4.1 EXPERIMENTAL SETUP
Models We conduct extensive experiments on diverse open-sourced L.MMs. For the main results, use LLaMA-7B (Touvron al., 2023a) as the L.M throughout our experiments. We also provide experiments on other models including Pythia-6.9B (Biderman et al., 2023), GPT-J-6B (Wang & Komatsuzaki, 2021), OPT-6.7B (Zhang et al., 2022), LLaMA-13B models (Touvron et al., 2023a), LLaMA-30B (Touvron et al., 2023a), and NeoX-20B (Black et al., 2022). Existing works (Shi et al., 2024: Ye et al., 2024) generally use these models LLMs for performing the studies of pretraining data detection. The models are provided by Hugging Face? ?
Datasets To verify the effectiveness of detection methods, we employ four common benchmark datasets for evaluations, including WikiMIA (Shi et al., 2024), ArXivTection (Duarte et al., 2024),
Phtpps: hhttps://hugginggfaae wo/moeel...mooooooooooooo
5

---

Published as a conference paper at ICLR 2025
0.5
member member
member
member
0.3
0.15
0.00
6
16
0.0
Perplexity
Min-k56
(a) Perplexity
(b) Min-k%
6.7
non-member member
non-member
member
0.5-
Densiy
0.4
6.15
0.3
0.19
0.1
-10
-5
15
20
2
Perplexity
Min-k5%
(c) FSD with Perplexity
(d) FSD with Min-k%
Figure 3: Distribution of scores from pre-trained model vs. FSD. We contrast the distribution of
scores from the pre-trained model using perplexity and our FSD with perplexity(a & c). Similarly, we contrast the Min-k% scores distribution from the pre-trained model and our FSD (b & d). Using FSD leads to enlarging the gap between members and non memmmes.....
BookTection (Duarte al., 2024) BookMIA (Shi et al., 2024) and Pile (Maini al., 2024). Previous works have demonstrated that model developers commonly use text content among those datasets for pre-training (Shi al., 2024; Duarte al., 2024; Ye et al., 2024). The datasets are provided by Hugging Face", and detailed information of datasets is presented in Appendix B.
Baseline methods We use four detection methods based on scoring functions as baselines to eval- uate the performance of different methods across various datasets and models. Those methods em- ploy specific metrics based on the likelihood, followed by a comparison with preset threshold to identify the membership of the given text. Specifically, baseline methods include the example perplexity (Perplexity) (Li, 2023), the ratio of example perplexity and zlib compression entropy (Zlib) (Carlini et al., 2021), the ratio of the perplexity the example before and after lowercasing (Lowercase) (Carlini et al., 2021) ) and detecting pretraining example through outlier words with low probability (Min-k%) (Shi al., 2024).
Evaluation metrics We evaluate the performance of different detection methods for detecting pre- training data from large language models, by measuring the following metries: (1) AUC, the area under the receiver operating characteristic curve; (2) the true positive rate (TPR) when the false positive rate (FPR) of the examples is 5% (TPR FPR)
Implementation details Our approach involves constructing the non-member dataset to fine-tune the base model. For constructing the non-member dataset, we randomly sample 30% of the data from the entire dataset and select all non-members from this subset as the constructed fine-tuning dataset. The remaining 70% of the dataset is used for testing. We employ LoRA (Hu et al., 2022) fine-tune the base model with epochs and batch size of 8. We set the initial learning rate as
hhttps://hugginngfacc.e.

---

Published as a conference paper at ICLR 2025
Table 1: AUC score for pretraining data detection with baselines and our method from various models on WikiMIA and ArXivTection. Base and +Ours respectively denote the baseline methods and our method. Bold shows the superior result.
GPT-J-6B
OPT-6.7B
Pythia-6.9B
LLaMA-7B
NeoX-20B
Dataset
Method
Base
+Ours
Base
+Ours
Base
+Ours
Base
+Oury
Base
+Ours
Perplexity
0.64
0.95
0.60
0.90
0.64
0.90
0.64
0.92
0.69
0.93
Lowercase
0.59
0.77
0.59
0.71
0.58
0.74
0.58
0.69
0.66
0.76
WikiMIA
Zlib
0.61
0.94
0.59
0.89
0.61
0.88
0.62
0.90
0.64
0.93
MIN-K%
0.68
0.92
0.62
0.91
0.67
0.86
0.65
0.85
0.73
0.90
Perplexity
0.79
0.96
0.68
0.89
0.77
0.95
0.68
0.92
0.79
0.95
Lowercase
0.59
0.81
0.58
0.70
0.60
0.77
0.50
0.69
0.62
0.75
ArXivTection
Zlib
0.64
0.96
0.55
0.89
0.63
0.95
0.57
0.91
0.65
0.95
MIN-K%
0.85
0.92
0.74
0.84
0.84
0.91
0.76
0.86
0.85
0.91
Table 2: The average AUC score of baselines and our method from the Pythia-6.98 over 20 subsets of the Pile dataset. Base and +Oury respectively denote the baseline methods and our method.
Perplexity
Lowercase
Zlib
MIN-K %
Method
Base
+Ours
Base
+Ours
Base
+Ours
Base
+Ours
Pile
0.503
0.625
0.519
0.566
0.507
0.624
0.515
0.600
0.001 and drop it by cosine scheduling strategy. We conduct all experiments on NVIDIA L40 GPU and implement all methods with default parameters using PyTorch (Paszke al., 2019).
4.2 MAIN RESULTS
Can FSD improve the performance of current scoring functions? We compare the perfor- mance of detection methods on WikiMIA and ArXivTection datasets across various LLMs. The details of the dataset split are shown in Appendix C.1. The results in Table 1 show that the FSD significantly improves the performance of all baselines on both datasets across diverse models. For example, our method improves the AUC score compared to the best baseline method Min-k%, in- creasing it from 0.62 to 0.91 on the WikiMIA dataset under the OPT-6.7B model. Similarly, it im- proves the AUC score from 0.76 0.86 on the ArXivTection dataset under the LLaMA-7B model. Moreover, our method improves the TPR @ 5%FPR score of all baselines, which can be found in Ap- pendix D.2. We also present the results on different subsets of the Pile dataset under the Pythia-6.98 model. Table 2 shows the average AUC scores of baselines and our method over the 20 subsets of Pile, demonstrating that our method achieves superior performance on the Pile dataset. In addition, the detailed results of all subsets of the Pile dataset are provided in Appendix D.3.
How does the fine-tuning data size affect the performance of FSD? To investigate the effect of varying the fine-tuning data size on the pretraining data detection, we compare the performance of our method using different-sized fine-tuned datasets for fine-tuning LLaMA-7B. To construct fine- tuning datasets of varying sizes, we randomly sample varying amounts of non-members (0, 30, 50, 100, 150, 200, 250, 300) from the WikiMIA dataset as fine-tuning datasets. In addition, we construct a balanced test set of 930 examples for evaluation.
Figure presents the performance of FSD with various sizes of auxiliary datasets. The results show our method achieves better performance as the size of the fine-tuning dataset increases. Notably, our method is highly data-efficient, achieving dramatic improvements with only small amount of non-members for fine-tuning. For example, FSD improves the AUC score of the perplexity-base method from 0.63 to 0.91, by leveraging only 100 non-member data for fine-tuning direct improvement of 44%. In summary, a few non-members are sufficient for FSD to improve the detection, demonstrating its practicality. In addition, we also evaluate our method on the BookC2 subset of the Pile dataset under the Pythia-6.9B model. The results show a similar trend, which can be found in Appendix D
significant
7

---

Published as a conference paper at ICLR 2025
Table AUC score for pretraining data detection with baselines and our method from the different- sized LLaMA model on WikiMIA. Base and +Ours respectively denote the baseline methods and our method. Bold shows the superior result.
LLaMA-7B
LLaMA-13B
LLaMA-30B
Method
Base
+Ours
Base
+Ours
Base
+Ours
Perplexity
0.64
0.92
0,66
0.92
0.68
0.91
Lowercase
0.58
0.69
0.60
0.70
0.60
0.75
Zlib
0,62
0.90
0,63
0.90
0.65
0.91
M
0.65
0.85
0.67
0.86
0.70
0.82
1.0
0.7
Perplexity Lowercase Zlib Min--k%
0.9
TPR@S%%FP
0.8
0.4
0.2
0.5
0.0
250
250
Sample size
Sample size
(a) AUC
(b) @
Figure 4: AUC and TPR PPFFR% of scoring functions with FSD, using auxiliary datasets with varying sizes. Notably, * represents the baseline without FSD.
Is FSD effective with different-sized models? We also verify the performance of baselines and our methods from different-sizes LLaMA models (7B, 13B, 30B) on WikiMIA. In Table 3, our re- sults demonstrate that method is effective with different-sized models, and achieves remarkable performance from different-sized models. Notably, the AUC score of Lowercase slightly rises as the parameters of the LLaMA model increase. Moreover, additional results of the TPR55%FPR score show a similar trend, which can be found in Appendix D.2.
Can our method detect copyrighted books in pretraining data? Recent works (Shi et al., 2024; Duarte al., 2024) study the problem of copyrighted book detection in training data. Following previous works, verify the effectiveness of detection methods for detecting excerpts from copy- righted books on BookMIA (Shi al., 2024) and BookTection (Duarte al., 2024). Specifically, we randomly sample 500 members and non members from datasets, constructing a balanced validation set of 1,000 examples. The detailed information datasets split is presented in Appendix C.2.
In Table 4, we compare the accuracy of our method and baselines for detecting suspicious books in pretraining data from the LLaMA-7B model. A salient observation is that our method significantly improves the accuracy of baseline methods for copyrighted book detection. For example, compared with baselines, our method reaches an accuracy of 98.66 98.6% on BookMIA using detection method Zlib, which marks significant 71.8% improvement. We also present the AUC score with method and baselines, which consistently improves the detection capabilities of baseline methods. Our extensive experiments demonstrate the effectiveness of our method for copyrighted book detection.
5 DISCUSSION
Can members be used for fine-tuning? The key step of our method is to fine-tune the pre-trained model using a few non-members. One may also ask: can a similar effect be achieved by utilizing members the fine-tuming dataset? In this ablation, members from WikiMIA to construct fine-tuning datasets(Mem, Non). In addition, we randomly sample data from WikiMIA another fine-tuning dataset (All). The details of implementation
separately sample members and non-
8

---

Published as a conference paper at ICLR 2025
Table 4: Accuracy and AUC score for copyrighted book detection with baselines and our method from LLaMA-7B on BookTection and BookMIA, Base and +Ours respectively denote baslines and our method. Bold shows the superior result.
Metric
Accuracy
AUC
Book Tection
BookMIA
Bection
BookMIA
Method
Base
+Ours
Base
+Ours
Base
+Ours
Base
+Ours
Perplexity
66.9
85.4
59.0
96.5
0.710
0.910
0.564
0.995
Lowercase
64.5
73.0
67.0
69.2
0.664
0.770
0.708
0.779
Zlib 65.3
86.4
57.4
98.6
0.568
0.920
0.474
0.999
M
68.1
82.1
59.5
93.9
0.716
0.880
0.587
0.979
Table 5: AUC of scoring functions with FSD, using members (Mem), non-members (Non), and mix of them (All) on LLaMA-7B. Base denotes the scoring function without FSD.
Table 6: AUC of scoring functions with FSD using the original WikiMIA, data removing timestamps (Deletion). and data replacing the year of timestamps with 2023 (Replacement). The results are shown as Base/+Ours.
Bold shows the superior result.
Method
Base
All
Mem
Non
Method
WikiMIA
Deletion
Replacement
Perplexity
0.64
0.68
0.78
0.92
Perplexity
0.64/ 0.92 0.62/ 0.76
0.54/ 0.71
Lowercase
0.58
0.54
0.67
0.69
Lowercase
0.58/ 0.69
0.58/ 0.62
0.52/ 0.63
Zlib
0.62
0.65
0.79
0,90
Zlib
0.62/ 0.90
0,58/ 0.72
0,55/ 0.68
MIN-K%
0.65
0.61
0.81
0.85
MIN-K%
0.65/0.85
0.61/ 0.69
0.54/ 0.67
presented in Appendix C.3. To investigate the impact of different fine-tuning datasets on pretraining data detection, we fine-tune the LLaMA-7B model with each fine-tuning dataset.
Our results in Table 5 show that method can improve the performance of baseline methods using members as the fine-tuning dataset. However, our method performs inferior when using members for fine-tuning compared with non-members. Moreover, it is not realistic to construct a member dataset without accessing pretraining data in real-world scenarios. In addition, this feasible for constructing non- members as fine-tuning dataset based on the model release date and data creation timestamp. Overall, our method achieves superior performance when using non-members for fine- tuning, while ensuring applicability in real- world settings. We also investigate the performance of our method when fine-tuning using data from an unrelated domain. The results in Appendix D.3 show that our method can improve the performance when fine-tuning data from a mix of domains.
Is method affected by distribution difference? Existing works generally construct bench- mark datasets based on the LLM release date and data creation timestamp (Ye et al., 2024; Shi et al., 2024). For example, the WikiMIA dataset considers events occurring post-2023 Recent works indicate evaluation results are suspect on benchmark datasets because they possibly sample members and non-members from different distributions (Duan al., 2024; Das al., 2024; Maini et al., 2024). We find the temporal shift between members and non-members in the WikiMIA dataset, which is shown in Appendix C.4. The issue shows that we may distinguish members and non-members with timestamps in the dataset. To eliminate the impact of temporal differences be- tween members and non-members on evaluation, we implement two strategies to mitigate the tem- poral shift in the dataset: (1) removing timestamps in the dataset (Deletion). and (2) replacing the year of timestamps with 2023 the dataset (Replacement). We conduct experiments with baselines and our method on the original WikiMIA dataset, Deletion and Replacement, respectively.
non-members.
Our results in Table show that our method also improves the performance baseline methods when mitigating the temporal shift between members and non-members. In this setting, the per- formance of baselines and our method is reduced, as deleting or replacing a word will change the probability of the subsequent word, thereby perturbing the likelihood based metric. Though base- line methods yield results comparable to random guessing on the Replacement dataset, our method improves the AUC scores of the perplexity-based method, increasing it from 0.54 to 0.71 Replace-

---

Published as a conference paper at ICLR 2025
Table 7: AUC score of FSD with different fine-tuning methods. Base denotes baseline methods without model fine-tuning. Bold shows the superior result.
Metric
AUC
TPR @5%PPR
Method
Base
Adal LoRA
IA3
LoRA
Base
AdaL
TA3
LoRA
Perplexity
0.64
0.82
0.91
0.92
0.09
0.39
0.52
0.41
Lowercase
0.58
0.62
0.72
0.69
0.10
0.13
0.17
0.18
7lib
0.62
0.76
0.84
0.90
0.09
0.24
0.32
0.47
MIN-K%
0.65
0.80
0.90
0.85
0.15
0.22
0.39
0.25
ment. Overall, our method is effective even if there is no distribution difference between members and non-members. The TPR@5%FPR score of the experiment is presented in Appendix D.2.
Is FSD effective with different fine-tuning methods? To expose LLMs unseen data, ploy LoRA to fine-tune the pre-trained model. The results demonstrate that our method achieves impressive performance for pretraining data detection when fine-tuning with LoRA. However, can a similar effect be achieved using different fine-tuning methods? To this end, we investigate the im- pact of fine-tuning methods on performance by applying AdaLoRA (Zhang al., 2023), IA3 (Liu et al., 2022), and LoRA to fine-tune LLaMA-7B with WikiMIA, respectively.
In Table 7, report the AUC and TPR@5%FPR scores for pretraining data detection for our method and baselines. The results show that our method improves the performance of baselines when using different tine-tuning methods. Although our FSD achieves inferior performance with A compared with IA3 and LoRA, it still improves the performance of baseline methods. Our method can be implemented with different fine-tuning methods and does not require specific fine- tuning technique. In addition, we also conduct experiments to explore the impact of different fine- tuning parameters on the performance of our method. The results show that our method is insensitive to LoRA rank and the number of fine-tuning epochs, which are presented in Appendix D.3.
CONCLUSION
In this paper, we introduce Fine-tuned Score Deviation (FSD), a novel detection method that can universally improve the performance of existing detection methods. To the best of knowledge, our method is the first to utilize some collected non-members the task of pretraining data detection. Our core idea behind FSD exposing the LLM to a few unseen examples. In effect, unseen data have larger score than seen examples when using FSD, which makes it more distinguishable between seen and unseen data. Extensive experiments demonstrate the effectiveness of our method for detecting pretraining data on common benchmark datasets across various models. summary, the FSD is an effective approach for accurately detecting pretraining data of LLMs.
enlarge the gap between seen examples and unseen examples by
Limitations Our method requires to collect a few examples that belong to the same domain but are not involved in the training. Generally, can utilize the data content published after the release of the LLM. Therefore, our method is applicable for detecting benchmarks or copyrighted resources in specific domain (e.g., math tests, magazines). The diversity of the test may make it challenging to construct an effective auxiliary dataset of unseen data. In addition, our method requires fine- tuning on few con-member data, so the effectiveness of the proposed score might be affected by the data quality of non-members.
Ethical Statement Our work focuses on pretraining data detection from large language models. The proposed methodology aims to address issues involving data contamination or copyright in- fringement. In addition. our method can be used to identify potential privacy leakage risks and ensure the safety of LLMs, aligning with established ethical standards for content moderation. Re- garding data access, the evaluated datasets we employed our work come from prior research and do not involve personal privacy information.
10

---

Published as a conference paper at ICLR 2025
ACKNOWLEDGEMENT
This research is supported by the Shenzhen Fundamental Research Program (Grant No. JCYI20230800001809900 and the SUSTech-NUS Joint Research Program. Bingyi Jing sup- ported in part by the National Natural Science Foundation of China under grant 1237 1290. We gratefully acknowledge the support of the Center for Computational Science and Engineering the Southern University of Science and Technology.
REFERENCES
Josh Achiam, Steven Adler, Sandhini Agarwal, Lama Ahmad, Ilge Akkaya, Florencia Leoni Ale- man, Diogo Almeida, Janko Altenschmidt, Sam Altman, Shyamal Anadkat, al. Gpt-4 technical report. arXiv preprint 2031v:2303.08774 2023.
Simone Balloccu, Patrícia Schmidtová, Mateusz Lango, and Ondřej Dušek. Leak, cheat, repeat: Data contamination and evaluation malpractices in closed-source llms. In Proceedings of the 18th Conference of the European Chapter of the Association for Computational Linguistics (Volume 1: Long Papers), pp. 67-93 2024.
Stella Biderman, Hailey Schoelkopf, Quentin Gregory Anthony, Herbie Bradley, Kyle O'Brien, Eric Hallahan, Mohammad Aflah Khan, Shivanshu Purohit, USVSN Sai Prashanth, Edward Raff, al. Pythia: a suite for analyzing large language models across training and scaling. In International Conference on Machine Learning, pp. 2397-2430. PMLR, 2023.
Sidney Black, Stella Biderman, Eric Hallahan, Quentin Anthony, Leo Gao, Laurence Golding, Ho- race He. Connor Leahy, Kyle McDonell, Jason Phang, et al. Gpt-neox-20hs An open-source autoregressive language model. In Proceedings of BigScience Episode# 5-Workshop on Chal- lengex & Perspectives in Creating Large Language Models, pp. 95-136, 2022.
Tom В Brown, Benjamin Mann, Nick Ryder, Melanie Subbiah, Jared Kaplan, Prafulla Dhariwal, Arvind Neelakantan, Pranav Shyam, Girish Sastry, Amanda Askell, et al. Language models are few-shot learners. In Proceedings of the 34th International Conference Neural Information Processing Systems, pp. 1877-1901, 2020.
Nicholas Carlini, Florian Tramer, Eric Wallace, Matthew Jagielski, Ariel Herbert-Voss, Katherine Lee, Adam Roberts, Tom Brown, Dawn Song, Ulfar Erlingsson, al. Extracting training data from large language models. In 30th USENIX Security Symposium (USENIX Security 21), pp- 2633-2650, 2021.
Aakanksha Chowdhery, Sharan Narang, Jacob Devlin, Maarten Bosma, Gaurav Mishra, Adam Roberts, Paul Barham, Hyung Won Chung, Charles Sutton, Sebastian Gehrmann, et al. Palm: scaling language modeling with pathways. Journal of Machine Learning Research, 24(240):1- 113, 2023.
Debeshee Das, Jie Zhang, and Florian Tramèr. Blind baselines beat membership inference attacks for foundation models. arXiv preprint arXiv:2406. 16201, 2024.
Edoardo Debenedetti, Giorgio Severi, Nicholas Carlini, Christopher Choquette-Choo Matthew Jagielski, Milad Nasr, Eric Wallace, and Florian Tramèr. Privacy side channels in machine learn- ing systems. In 33rd USENIX Security Symposium (USENIX Security 24), pp. 6861-6848, 2024,
Michael Duan, Anshuman Suri, Niloofar Mireshghallah. Sewon Min, Weijia Shi, Luke Zettlemoyer, Yulia Tsvetkov, Yejin Choi, David Evans, and Hannanch Hajishirzi. Do membership inference attacks work on large language models? arXiv preprint arXiv:2402.0784/ 2024.
André Vicente Duarte, Xuandong Zhao, Arlindo L Oliveira, and Lei Li. De-cop: Detecting copy- righted content in language models training data, Forty-first International Conference on Ma- chine Learning, 2024.
Leo Gao, Stella Biderman, Sid Black, Laurence Golding, Travis Hoppe. Charles Foster, Jason Phang, Horace He, Anish Thite, Noa Nabeshima, et al. The pile: an 800gb dataset of diverse text for language modeling. arXiv preprint 20X.v:2101.00027 2020.
11

---

Published as a conference paper at ICLR 2025
Shahriar Golchin and Mihai Surdeanu. Time travel in Ilms: Tracing data contamination in large language models. In The Twelfth International Conference on Learning Representations, 2024.
Michael M Grynbaum and Ryan Mac. The times openai and microsoft over ai use of copy- righted work. The New York Times, 2023.
Edward Hu, Phillip Wallis, Zeyuan Allen-Zhu, Yuanzhi Li, Shean Wang. Wang, Weizhu Chen, et al. Lora: Low-rank adaptation of large language models. In International Conference on Learning Representations, 2022.
Yucheng Li. Estimating contamination via perplexity: quantifying memorisation in language model evaluation. arXiv preprint arXiv:2309. 10677, 2023.
Gaoyang Liu, Chen Wang, Kai Peng, Haojun Huang, Yutong Li, and Wenqing Cheng. Socinf: mem- bership inference attacks on social media health data with machine learning. IEEE Transactions on Computational Social Systems, (5):907-921, 2019.
Gaoyang Liu, Tianlong Xu, Rui Zhang, Zixiong Wang, Chen Wang, and Ling Liu. Gradient-leaks enabling black-box box membership inference attacks against machine learning models. IEEE Trans- actions on Information Forensics and Security, 2023.
Haokun Liu, Derek Tam, Mohammed Muqeeth, Jay Mohta, Tenghao Huang, Mohit Bansal, and Colin A Raffel. Few-shot parameter-effiiennt fine-tuning is better and cheaper than in-context learning. Advances in Neural Information Processing Systems, 35:1950- 2022.
Inbal Magar and Roy Schwartz. Data contamination: From memorization to exploitation. In Pro-
ceedings of the 60th Annual Meeting of the Association for Computational Linguistics (Volume 2: Short Pupers), pp. 157-165, 2022.
Pratyush Maini, Hengrui Jia, Nicolas Papernot, and Adam Dziedzic. Llm dataset inference: did you train on my dataset? arXiv preprint arXiv:2406.0644 2024.
Justus Mattern, Fatemehsadat Mireshghallah, Zhijing Jin, Bernhard Schoelkopf, Mrinmaya Sachan, and Taylor Berg- Kirkpatrick. Membership inference attacks against language models via neigh- bourhood comparison. In Findings of the Association for Computational Linguistics: ACL 2023, pp. 11330-11343, 2023.
Maximilian Mozes, Xuanli He, Bennett Kleinberg, and Lewis D Griffin. Use of iims for illicit purposes: Threats, prevention measures, and vulnerabilities. arXiv preprint arXiv:2308. 12833, 2023.
Yonatan Oren, Nicole Meister, Niladri Chatterji. Faisal Ladhak, and Tatsunori Hashimoto. Proving test set contamination in black- box language models. In The Twelfth International Conference Learning Representations, 2024.
Adam Paszke, Sam Gross, Francisco Massa, Adam Lerer, James Bradbury, Gregory Chanan, Trevor Killeen, Zeming Lin, Natalia Gimelshein, Luca Antiga, et al. Pytorch: An imperative style, gegh-performance deep learning library. Advances in Neural Information Processing Systems, 32, 2019.
Alec Radford, Jeffrey Wu, Rewon Child, David Luan, Dario Amodei, Ilya Sutskever, et al. Language models unsupervised multitask learners. OpenAl blog, 1(8):9, 2019.
Mathieu Ravaut, Bosheng Ding, Fangkai Jiao, Hailin Chen, Xingxuan Li, Ruochen Zhao, Chengwei Qin, Caiming Xiong, and Shafiq Joty. How much are lims contaminated? comprehensive survey and the llmsanitize library. arXiv preprint arXiv: 2404.00999 2024.
Oscar Sainz, Jon Campos, Iker García-Fenrnn Ferrero, Julen Etxaniz, Oier Lopez de Lacalle, and Eneko Agirre. NIp evaluation in trouble: On the need to measure Ilm data contamination for each benchmark. In Findings of the Association for Computational Linguistics: EMNLP 2023, pp. 10776-10787, 2023.
12

---

Published as a conference paper at ICLR 2025
Ahmed Salem, Yang Zhang, Mathias Humbert, Mario Fritz, and Michael Backes. MI-leaks: Model and data independent membership inference attacks and defenses on machine learning models. In Network and Distributed Systems Security Symposium 2019. Internet Society, 2019.
Weijia Shi, Anirudh Ajith, Mengzhou Xia, Yangsibo Huang, Daogao Liu, Terra Blevins, Danqi Chen, and Luke Zentlemoyer. Detecting pretraining data from large language models. In The Twelfth International Conference on Learning Representations, 2024.
Reza Shokri, Marco Stronati, Congzheng Song, and Vitaly Shmatikov. Membership inference at- tacks against machine learning models. In 2017 IEEE Symposium Security and Privacy (SP), pp. 3-18. IEEE Computer Society, 2017.
Gemini Team, Rohan Anil, Sebastian Borgeaud, Yonghui Wu, Jean-Baptiste Alayrae, Jiahui Yu, Radu Sorieut, Johan Schalkwyk, Andrew M Dai, Anja Hauth, al. Gemini: family of highly capable multimodal models. arXiv preprint arXiv:2312. 1805, 2023.
Hugo Touvron, Thibaut Lavril, Gautier Izacard, Xavier Martinet, Marie-Anne Lachaux, Timothée Lacroix, Baptiste Rozière, Naman Goyal, Eric Hambro, Faisal Azhar, et al. Llama: open and efficient foundation language models. arXiv preprint arXiv:2302. 13971, 2023,
Hugo Touvron, Louis Martin, Kevin Stone, Peter Albert, Amjad Almahairi, Yasmine Babaci, Niko- lay Bashlykov, Soumya Batra, Prajjwal Bhargava, Shruti Bhosale, el. Llama 2: open foundation and fine-tuned chat models. arXiv preprint wxXh:22307.09288 2023b.
Stacey Truex, Ling Liu, Mehmet Emre Gursoy, Lei Yu, and Wenqi Wei. Demystifying memhership inference attacks in machine learning as a service. IEEE transactions on services computing, 14 (6):2073-2089 2019.
Ben Wang and Aran Komatsuzaki. GPT-J6BB: a billion parameter autoregressive language model. https: / com www.koo///iiioeeee http://innnoloooosmsseeeeoo tormess--rtrrssoor-rr-rr jax, May 2021.
Roy Xie, Junlin Wang, Ruomin Huang, Minxing Zhang, Rong Ge, Jian Pei, Neil Zhenqiang Gong, and Bhuwan Dhingra. Recall: Membership inference via relative conditional log- ligeliiilsoods. arXiv preprint 2rXiv:2406.15000 2024.
Cheng Xu, Shuhao Guan, Derek Greene, M Kechadi, et al. Benchmark data contamination of large language models: A survey, arXiv preprint arXiv:2406.042: 042444 2024a.
Ruijie Xu, Zengzhi Wang, Run-Ze Fan, and Pengfei Liu. Benchmarking benchmark leakage in large language models. arXiv preprint arXiv:2404.1882. 18824, 2024b.
Wentao Ye, Jiaqi Hu, Liyao Li, Haobo Wang. Gang Chen, and Junbo Zhao. Data contamination calibration for black-box llms. arXiv preprint arXiv:2405. /930,, 2024.
Samuel Yeom, Irene Giacomelli, Matt Fredrikson, and Somesh Jha. Privacy risk in machine learn- ing: Analyzing the connection to overfitting. In 2018 IEEE 31 Computer Security Foundations Symposium (CSF), pp. 268-282, IEEE Computer Society, 2018.
Jingyang Zhang, Jingwei Sun, Eric Yeats, Yang Ouyang, Martin Kuo, Jianyi Zhang, Hao Yang, and Hai Li. Min-k%++ Improved baseline for detecting pre-training data from large language models. arXiv preprint arXiv:2404.029333 2024.
Qingru Zhang, Minshuo Chen, Alexander Bukharin, Nikos Karampatziakis, Pengcheng He, Yu Cheng, Weizhu Chen, and Tuo Zhao. Adalora: adaptive budget allocation for parameter- efficient fine-tuning. arXiv preprint arXiv:2303. 105/2, 2023.
Susan Zhang, Stephen Roller, Naman Goyal, Mikel Artetxe, Moya Chen, Shuohui Chen, Christo- pher Dewan, Mona Diab, Xian Li, Xi Victoria Lin, et al. Opt: open pre-trained transformer language models. arXiv preprint 2rXiv:2205.00066 2022.
Kun Zhou, Yutao Zhu, Zhipeng Chen, Wentong Chen, Wayne Xin Zhao, Xu Chen, Yankai Lin, Ji-Rong Wen, and Jiawei Han. Don't make your Ilm an evaluation benchmark cheater. arXiv preprint ar arXiv:2311 019964, 2023.
13

---

Published as a conference paper at ICLR 2025
A RELATED WORK
Pretraining data detection, which is an increasingly important topic for large language models, lates to large amount of literature on membership inference attacks and data contamination. We discuss some of the relevant works ours in directions below.
Membership Inference Attacks Our work mainly studies how to detect a given example in the pretraining data, which is consistent with the objective of membership inference attacks (MIAs) (Shokri et al., 2017; Truex et al., 2019). This task aims to determine whether given data point is member of training data. Metric-based attack methods, such loss (Yeom et al., 2018), entropy (Salem et al., 2019), confidence (Liu et al., 2019) and gradient (Liu et al., 2023), infer mem- bership of data by comparing the calculated metrie value with preset threshold. Previous works have generalized metric-basee methods to large language models (Duan et al., 2024; Xie et al., 2024; Zhang et al., 2024; Mattern al., 2023), by calculating the ity) for membership inference. Recent works apply MIAs to pretraining data detection by designing al.,
metric (e.g.. perplex-
likelihood-base scoring functions to measure the membership of data (Shi et al., 2024; 2024). In this work, we analyze the limitations of existing scoring functions for pretraining data detection, and design an effective method to improve their performance. In particular, this work is the first to explore the importance of collecting unseen data in pretraining data detection.
Data Contamination Data contamination has been studied in the literature (Xu et al., 2024a; Magar & Schwartz, 2022; Balloccu et al., 2024), where training data may inadvertently include evaluation benchmark data, resulting in unauthentic evaluation results. Thus, it is important to assess the leakage of benchmark data into pretraining data (Zhou al., 2023). On the one hand, model developers can remove evaluation benchmark data from training data by cetrieval-based methods with access to pertaining data (Ravaut al., 2024; Chowdhery et al., 2023). Specifically, those methods employ n-gram tokenization and string-matching for detecting data contamination (Brown al., 2020; Touvron al., 2023b; Team al., 2023; Radford et al., 2019). On the other hand, researchers utilize prompting techniques (Golchin & Surdeanu, 2024), performance analysis (Ye al., 2024; Debenedetti al., 2024), model likelihood (Oren al., 2024; Shi et al., 2024; Xu al., 2024b) to detect potential contamination without access to the training data. Our work focuses on pretraining data detection, an area that similar to data contamination. Different from data contamination detection, our FSD can also be applied to the detection of copyrighted resources in real-world scenarios.
B DETAILS OF DATASETS
Previous works construct benchmark datasets to evaluate the performance of detection methods for pretraining data detection. Following the prior literature, we conduct experiments on 5 benchmark datasets: WikiMIA (Shi et al., 2024) selects old Wikipedia event data as member data by leveraging the Wikipedia data timestamp and the model release date, since Wikipedia is commonly pretraining data source. BookMIA (Shi et al., 2024), which contains excerpts from copyrighted books in the Books3 subset of the Pile dataset (Gao al., 2020), can be used for detecting potential copyright infringement in training data. ArXivTection (Duarte et al., 2024) is curated collection of research articles sourced from ArXiv. BookTection (Duarte et al., 2024), which comprises passages from 165 books, is constructed based on BookMIA. We also conducted experiments on the Pile dataset (Maini et al., 2024), which is large-scale text dataset for training language models, including text data from various sources such as books, GitHub, and website content.
C EXPERIMENTAL DETAIL
C.I DATASET SPLIT
We report the performance of detection methods WikiMIA and ArXivTection datasets across various large language models. In our experiments, we first randomly sample 30% the dataset, and then select all mon-members from this subset to construct the fine-tuning dataset. The remaining
14

---

Published as a conference paper at ICLR 2025
Table 8: The train set and test set used in the experiment
Dataset
Type
Member
Non-member
Total
Train Set
231
231
WikiMIA
Test Set
599
538
1,157
Train Set
238
238
ArXivTection
Test Set
536
549
1,085
Table 9: The train set, test set and validation set used in the experiment
Dataset
Type
Member
Non-member
Total
Train Set
1,413
1,413
BookMIA
Test Set
2,887
3,022
5,909
Validation set
500
500
1,000
Train Sct
1,796
1,796
BookTection
Test Set
6,833
3,657
10,490
Validation set
500
500
1,000
70% the dataset used for testing. The detailed information of the constructed dataset is shown in Table 8.
C.2 COPYRIGHTED BOOK DETECTION
To conduct experiments of copyrighted book detection on BookMIA and BookTection, we randomly sample 30% of the dataset and select all non-members from this subset the fine-tuning dataset. Subsequently, we randomly sample 500 members and non-members from the remaining 70% of the datasets, constructing balanced validation of 1,000 examples for evaluation. The detailed information dataset split is shown in Table 9.
C.3 FINE-TUNING WITH MEMBERS
To investigate the impact model fine-tuning with different fine-tuning datasets on pretraining data detection, we construct three types of fine-tuning datasets. In this ablation, we sample members (Mem) and non-members (Non) from WikiMIA as fine-tuning datasets, respectively. addition, we randomly sample data from WikiMIA to construct fine-tuning dataset (AlI). The details of fine-tuning datasets are shown Table 10
Table 10: The train and test used in the experiment
Datasets
Type
Member
Non-member
Total 262
Train Set
262
Mem
Test Set
599
558
1,157
Train Set
231
231
Non
Test Set
599
558
1,157
Train Set
262
231
493
All
Test Set
599
558
1,157
C.4
TEMPORAL SHIFT
In Table 11, we provide some illustrations of the temporal shift between members and non nom-mebeers in the WikiMIA dataset.
15

---

Published as a conference paper at ICLR 2025
Table 11: Illustrations of temporal shift between the member and non-member distributions.
Members
Non-Members
The 2014 On 30 June or July 2014, the Armed Forces of the Democratic Repub- lic of the Congo and United Nations forces launched an offensive against rebel groups in the Masisi and Walikale.
The 95th Academy Awards was a cere- mony held by the Academy of Motion Pic- ture Arts and Sciences (AMPAS) on March 12, 2023, at the Dolby Theatre in Los An- geles.
In 2014, series of groundbreaking diplo- matic meetings
The 36th Annual Nickelodeon Kids' Choice Awards ceremony was held on March 4, 2023, at the Microsoft The- ater in Los Angeles, California with Nate Burleson and Charli D' Amelio.
held between Wang Yu-chi, in his official capacity as the Minis- ter of the Mainland Affairs Council (MAC) of the Republic of China (ROC).
Concluding observations the second pe- riodic report of the Holy See was a 2014 report issued by the Office of the United Nations High Commissioner for Human Rights.
The 2023 Summer Metro Manila Film Fes- tival an ongoing iteration of the annual Summer Metro Manila Film Festival held in Metro Manila and throughout the Philip- pines.
The 2014 European Aquatics Champi- onships took place from 13 to 24 August 2014 Berlin, Germany.
On February 11. 2023, an octagonal unidentified flying object was detected over northern Montana.
The centenary of the outbreak of World War was commemorated in Europe in late July and early August 2014.
The 2023 Tokyo Marathon was the 16th edition of the annual marathon race in Tokyo, held on Sunday, 5 March 2023.
Table 12: TPR @ F%F for pretraining data detection with baselines and method from various models on WikiMIA and ArXiv'eection Base and +Ours respectively denote the baseline methods and our method. Bold shows the superior result.
GPT-J-6B
OPT-6.7B
Pythia-6.9B
LLaMA-7B
NeoX-20B
Dataset
Method
Base
+Ours
Base
+Ours
Base
+Ours
Base
+Ours
Base
+Ours
Perplexity
0.12
0.78
0.12
0.63
0.13
0.66
0.09
0.41
0.20
0.58
Lowercase
0.12
0.24
0.07
0.18
0.11
0.25
0.10
0.18
0.16
0.18
WikiMIA
Zlib
0.09
0.78
0.09
0.55
0.10
0.50
0.09
0.47
0.10
0.57
MIN-K%
0.17
0.40
0.14
0.50
0.17
0.35
0.15
0.25
0.25
0.36
Perplexity
0.26
0.79
0.12
0.63
0.25
0.66
0.10
0.81
0.27
0.77
Lowercase
0.13
0.23
0.15
0.22
0.15
0.25
0.09
0.16
0.13
0.20
ArXivTection
Zlib 0.15
0.80
0.07
0.60
0.14
0.50
0.08
0.66
0.16
0.77
MIN-K %
0.42
0.57
0.24
0.45
0.41
0.35
0.24
0.45
0.40
0.58
D DETAILED EXPERIMENTAL RESULTS
D.1 FINE-TUNED SCORE DEVIATION
We show in Figure 5 the score distribution between members and non-member using two scoring functions on ArXivTection. The results also demonstrate that our FSD score significantly increases the gap between non-members and members compared to the perplexity and Min-k% scores, thus enabling more effective pretraining data detection.
D.2 DETAILED RESULTS OF EXPERIMENT
We report the TPR @5%PPR score for pertaining data detection in Table 12, 13, 14.
16

---

Published as a conference paper at ICLR 2025
member member
non non-member member
0.6
Doraity
Dersity
0.4
0.08
0.2
0.00
15
20 Perplexity
30
0.0
10
Min-k%
(a) Perplexity
(b) Min-k%
0.5
1.0
non-member member
non-member
member
1.4
0.3-
2.0
0.2
0.6
0.2
-5.0
-2.5
0.0
2.5
5.0 Perplexity
7.5
10.0
12.5
-1.
-0.5
0.0
0.5
1.0
3.5
2.0
Min-kd%
(c) FSD with Perplexity
(d) FSD with Min-k%
Figure 5: Distribution of scores from pre-trained model vs. FSD. We contrast the score distribution from the pre-trained model using perplexity and our FSD with perplexity(a & c). Similarly, we contrast the Min-k% scores distribution from the pre-trained model and our FSD (b & d). Using FSD leads to enlarging the gap between members and non-membon non-members.
Can FSD improve the performance of detection methods based on scoring functions? We compare the TPR @ F5FFP score with our method and baselines on WikiMIA and ArXivTection datasets across various large language models in Table 12. The results show that our method signif- icantly improves the TPR @ 5%FPPR score of the baseline methods.
Is FSD effective with different-sized models? We verify the performance of baselines and our methods from different-sizze LLaMA models (7B, 13B, 30B) WikiMIA. In Table 13, we show the TPR FPFPPR score from different-sized ILaMA models. The results demonstrate that our method is effective with different-size models.
our method affected by distribution difference? We report the TPR@5%FPR score base- lines and our method on the original WikiMIA dataset, Deletion and Replacement. In Table 14, the results show that our method still improves the performance of baselines when mitigating temporal shift between members and non-members.
D.3 ADDITIONAL RESULTS
The performance of our method on the Pile dataset We also conduct experiments on the Pile dataset. Concretely, following prior work (Maini al., 2024), we evaluate our methods on the twenty subsets of the Pile dataset. Here, the validation set of the pile dataset was not trained on the Pythia models (Biderman et al., 2023). Thus, we perform experiments on the Pythia-6.9B model, utilizing the training and validation sets members and non-members, respectively. For each dataset, we randomly sample few non- non-members with a sample ratio of 0.3 from the validation set for fine-tuning. Then, evaluate method on a balanced dataset composed of members and non-menee non-members. Notably, in our experiments, there is no overlap between the fine-tume finettuning dataset and the evaluation data.
17

---

Published as a conference paper at ICLR 2025
Table 13: @ score for pretraining data detection with baselines and our method from the different-sized LLaMA model on WikiMIA. Base and +Ours respectively denote the baselines and our method. Bold shows the superior result.
LLaMA-7B
LLaMA-13B
LLaMA-30B
Method
Base
+Ours
Base
+Ours
Base
+Ours
Perplexity
0.09
0.41
0.11
0.61
0.15
0.40
Zlib
0.10
0.18
0.13
0.13
0.11
0.25
Lowercase
0.09
0.47
0.10
0.56
0.11
0.44
MIN-K%
0.15
0.25
0.18
0.26
0.19
0.20
Table 14: T score from the LLaMA-7B model with our method and baselines using the original WikiMIA, data removing timestamps (Deletion), and data replacing the year of timestamps with 2023 (Replacement). Base and +Ours denote the baseline methods and our method, respec- tively. Bold shows the superior result.
Origin
Deletion
Replacement
Method
Base
+Ours
Base
+Ours
Base
+Ours
Perplexity
0.09
0.41
0.13
0.23
0.04
0.12
Lowercase
0.10
0.18
0.06
0.13
0.03
0.15
Zlib
0.09
0.47
0.12
0.23
0.09
0.06
MIN-K% %
0.15
0.25
0.10
0.14
0.04
0.07
In Table 15, the results show that method improves the performance of baselines most subsets of the Pile dataset under the Pythia-6.95 model. For example, our FSD improves the AUC score of the perplexity-basee method from 0.528 0.885 on BookC2, significant direct improvement of 67%. At the same time, our FSD improves the average AUC score of the perplexity-basee method from 0.503 to 0.625 on the pile dataset, a notable direct improvement 24.3%. This demonstrates the effectiveness of our method in the Pile dataset.
Fine-tuning on BookC2 with varying data size We also conduct experiments on the BookC2 subset of the Pile dataset under the Pythia-6.9B model investigate the effect of varying the fine- tuning data size on the pretraining data detection. Specifically, we randomly sample varying amounts of non-members (0. 50, 100, 150, 200, 250, 300, 350, 400, 450, 500) from the validation of the BookC2 as fine-tuning datasets. In addition, sample 1400 members and non-membert from the train and validation sets of the BookC2 to construct balanced test set of 2800 examples.
Figure 6a shows that our method achieves better performance as the size of the fine-tuning dataset increases. Notably, our method highly data-efficient, achieving significant improvements with only few non-mene non-members for fine-tuning. For instance, our method improves the AUC score of the Zlib method from 0.48 to 0.78, by leveraging only 100 non-member data for fine-tuning. In addition, the results of the TPR @ @5% 5%FPR score show similar trend, which can be found in Figure 6b.
Fine-tuning using non-members from different domains Our method requires a few non- member data from a specific domain for fine-tuning. This raises question: how does our method perform when fine-tuned on non-member data from different domain? To investigate the perfor- mance of our method when fine-tuning using data from an unrelated domain. Firstly, we randomly sample 231 and 238 non-members from the WikiMIA and ArXiv Tection datasets to construct fine-tuning dataset comprising a mix of domains. Then, we fine-tune the LLaMA-7B model on the constructed dataset and evaluate method on WikiMIA and ArXivTeecion datasets.
Our results in Table 16 show that our method can also significantly improve the performance of baselines. indicating the effectiveness of methods when fine-tuning with non-members from a mix of domains. We also evaluate our methods on ArXivTection while fine-tuning using non- members from WikiMIA. The results indicate that method fails to improve the performance of
18

---

Published as a conference paper at ICLR 2025
Table 15: AUC score for pretraining data detection with baselines and our method from the Pythia- 6.9B on the Pile dataset. Base and +Ours respectively denote the baseline methods and our method. Bold shows the superior result.
Wiki
BookC2
Gutenberg
HackerNews
Enron
Method
Base
+Ours
Base
+Ours
Base
+Ours
Base
+Ours
Base
+Ours
Perplexity
0.471
0.614
0,528
0.885
0.528
0.661
0.471
0.565
0,510
0.678
Lowercase
0.466
0.626
0,518
0.725
0.546
0.551
0.450
0.512
0.484
0.659
Zlib
0.496
0.619
0,477
0.907
0.496
0.686
0.474
0.550
0.560
0.667
MIN-K%
0.512
0.611
0,510
0.841
0.536
0.612
0.498
0.535
0.570
0.646
CC
arXiv
Europarl
FreeLaw
GitHub
Method
Base
+Ours
Base
+Ours
Base
+Ours
Base
+Ours
Base
+0urs
Perplexity
0.541
0.546
0,514
0.505
0.514
0.601
0.478
0.515
0.509
0.548
Lowercase
0,502
0.547
0,523
0.530
0.521
0.556
0.476
0.507
0.491
0.513
Zlib
0.529
0.576
0.540
0.505
0.462
0.609
0.492
0.503
0.491
0.562
MIN-K%
0.557
0.542
0.515
0.502
0.512
0.583
0.492
0.500
0.513
0.551
Books3
Nih
Open Webtextt
PhilPapers
OpenSubtitles
Method
Base
+Ours
Base
+Ours
Base
+Ours
Base
+Ours
Base
+Ours
Perplexity
0,560
0.509
0.463
0,599
0.490
0.580
0.571
0.869
0.525
0.521
Lowercase
0.550
0.524
0.608
0.512
0.486
0.547
0.633
0.718
0.538
0.528
Zlib
0.550
0.581
0,416
0.599
0.475
0.586
0.678
0.871
0.550
0.530
M
0.552
0.554
0.463
0.560
0.510
0.567
0.606
0.826
0.525
0.535
Stacknxchange
Math
YoutubeSubtitles
USPTO
Ubuntu
Method
Base
+Ours
Base
+Ours
Base
+Ours
Base
+Ours
Base
+Ours
Perplexity
0.640
0.678
0.530
0.504
0.392
0.756
0.537
0.606
0.282
0.767
Lowercase
0.579
0.641
0.508
0,513
0.495
0.546
0.510
0.582
0.496
0.476
Zlib 0.595
0.686
0.513
0.502
0.445
0.736
0.484
0.604
0.423
0.592
MIN-K%
0.637
0.670
0.524
0.510
0.380
0.692
0.549
0.596
0.329
0.561
Table 16: AUC score for pretraining data detection with baselines and our method on WikiMIA and ArXivTection under the LLaMA-7B. Wiki (Mix) denote evaluating on WikiMIA and fine-tuning using data from mix of domains. ArXiv (Wiki) denote evaluating on ArXivTection and fine-ture fine-tuning on WikiMIA. Base and +Ours respectively denote the baseline methods and our method.
Wiki (Mix)
ArXiv (Mix)
ArXiv (Wiki)
Method
Base
+Ours
Base
+Ours
Base
+Ours
Perplexity
0.64
0.91
0.68
0.93
0.68
0.52
Lowercase
0.58
0.73
0.50
0.73
0.50
0.50
Zlib
0.62
0.91
0..77
0.92
0.57
0.64
MIN-K%
0.65
0.84
0.76
0.87
0.76
0.61
baselines, since the fine-tuning data comes from an entirely unrelated domain to the evaluation data.
How do the fine-tuning parameters affect the performance of method? To investigate the
impact of different fine-tuning parameters on the performance of our method, we conduct experi- ments the WikiMIA dataset with different fine-tuning parameters, including learning rate (e.g.
19

---

Published as a conference paper at ICLR 2025
Table 17: AUC score of baselines and our method WikiMIA under the LLaMA-7B with different fine-tuning parameters. Base and +Ours respectively denote the baseline methods and our method. Bold shows the superior result.
Learning Rate
LoRA Rank
Epoch
Method
Base
10
10
10
Base
16
32
Base
2
Perplexity
0.64
0.81
0.84
0.92
0.64
0.92
0.92
0.92
0.64
0.91
0.91
0.92
Lowercase
0.58
0.60
0.64
0.69
0.58
0.69
0.68
0.69
0.58
0.65
0.64
0.69
Zlib
0.62
0.73
0.78
0.90
0.62
0.91
0.90
0.90
0.62
0.87
0.87
0.90
MIN-K
0.65
0.76
0.81
0.85
0.65
0.87
0.85
0.86
0.65
0.86
0.87
0.86
1.0
Perplexity Lowercase Zlib
Perplexity Lowercase Zlib
0.9
@PR@5%%FP
0.4
0.8
0.3
0.7
0.2
0.6
100
200
300
400
500
100
200
300
400
500
Sample size
Sample size
(a) AUC
(b) TPR@5%IPR
Figure 6: AUC and TPR @ 5%FPR of scoring functions with FSD, using auxiliary datasets with varying sizes. Notably, * represents the baseline without FSD.
le-3, le44, le-5), epoch (e.g. 2, 3) and LoRA rank (e.g. 8, 32). In Table 17, the results show that our method is relatively insensitive to LoRA rank and the number of fine-tuning epochs. However, considering the learning rate parameter, learning rate 0.001 enables our method to perform better.
20


Reviews:
[{"review_id": "keI5Kucoph", "reviewer": "Reviewer_oZEC", "summary": "The paper addresses the problem of detecting prestrainign data from LLMs. They notice that when fine-tuning an LLM on a few non-members selected by a cut-off date, the perplexity on other non-members (also based on the same cut-off date) decrease. Based on this observation, the authors propose a new method (Fine-tuned Score Deviation; FSD) to improve prior membership inference attacks (MIA) by adjusting the scores based on this fine-tuning phenomenon. They conduct experiments on 4 cut-off based benchmarks: WikiMIA, BookMIA, ArXivTection and BookTection and show significant gains wrt the baselines. The specific questions they tackle are:\n\n1. Can FSD improve the performance of current MIA methods?\n2. How many samples does the method need for fine-tuning?\n3. Does FSD work across different model sizes?\n4. Does FSD work with members for fine-tuning, instead of non-members?\n5. Does FSD work for any fine-tuning method?", "questions": "* Please conduct your experiments on an IID setup (see the works of Duan et al., 2024, Das et al., 2024, and Maini et al., 2024 that you cite; eg: using Pythia use as members the training set and as non-members the dev and test sets) so that we can clearly see the effectiveness of your work.\n* It is possible that the results of Figure 4 are explained by the temporal bias of the evaluation. Training more on new data might make the method identify better the n-gram distribution of the documents after the cut-off. We can reject this hypothesis by evaluating on an IID setup.", "limitations": "", "rating": 5, "confidence": 5, "soundness": 2, "presentation": 4, "contribution": 2, "strengths": "1. The idea is simple and intriguing, enlarging the difference between members and non-members could boost any MIA method.\n2. The idea works for any MIA method.", "weaknesses": "The evaluation has major flaw. It only uses benchmarks that separates members and non-members based on cut-off dates. As Duan et al., 2024, Das et al., 2024, and Maini et al., 2024 show, this is fundamentally wrong because it introduces a temporal shift that bias the benchmark. These works have consistently proved the need to avoid evaluating MIA based on cut-off dates. This paper acknowledges these works on lines 458-459 and questions whether the results of this ICLR submission are influenced by this temporal shift. However, instead of following the recommendations from these papers for a correct evaluation, they simply remove timestamps. This is not enough as Duan et al., (2024) shows. Duan et al., shows that the cut-offs introduces changes in the n-grams. An IID split have a higher n-gram overlap between members and non-members than the temporal split. Therefore, even when removing the timestamps in the plain texts, the n-gram distribution is different, making the evaluated method classify temporal data rather than membership. Moreover, the authors acknowledge this possibility in lines 535-537 but they do not follow the recommendations of the papers mentioned above, which are to evaluate on for example Pythia with data from the Pile.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "hOCk0VtkuL", "reviewer": "Reviewer_J4zF", "summary": "The paper proposes a new method for improving the accuracy of pretraining data detection in LLMs, called Fine-tuned Score Deviation. they shows that fine-tuning models on a small set of non-member data increases the deviation between score distributions of seen and unseen data. They validate this claim via experiments on WikiMIA, BookMIA, ArXivTection and BookTection datasets. FSD consistently improves the performance of existing detection methods based on scoring functions like perplexity and Min-k%.", "questions": "1. Do you have intuitions or theoretical understanding about why finetuning increases the gap between distributions?\n\n2. What is the role of the finetuning technique you use? Is there a dependency on LORA?\n\n3. How sensitive is FSD to using unseen data from a loosely related domain impact detection performance?", "limitations": "", "rating": 6, "confidence": 3, "soundness": 3, "presentation": 2, "contribution": 2, "strengths": "Very interesting observation, and notable improvement in pretraining data detection accuracy. I think this paper has a clearly defined objective, and interesting empirical results.\n\nThe results are strong, showing substantial improvements in AUC and tpr at low fpr across the selected datasets and models. FSD helps improving existing score-based pretraining data detection methods.", "weaknesses": "1. Have you experimented on the MIMIR dataset? This dataset seems to be more challenging, and authors claim this is because seen and unseen examples are not from different temporal distributions. I am very interested to see how this method helps MIA on MIMIR dataset.\n\n2. They have not explained or given intuition about why finetuning increases the gap between distributions.\n\n3.  The method requires fine-tuning the LLM, which makes it more expensive compared to existing methods. If authors find some intuition about why finetuning has this impact, they may find less expensive methods.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "nzUB71SEv2", "reviewer": "Reviewer_Zrvw", "summary": "The paper tackles the issue of detecting pretraining data. It introduces a method where a pretrained model is fine-tuned with non-member data from the same domain. The goal is to increase the gap in scores like perplexity between member and non-member data, making it easier to spot training data.", "questions": "* Could you explain more about the datasets, like how they’re labeled and how well they fit the models used? Is there a class imbalance in any of the datasets?\n* Should fine-tuning happen for each example $x$ (or each group of examples $x_i$ within the same domain)? If so, that might be a big drawback.\n* Fine-tuning can be delicate, and it’s easy to go past the optimal point. An experiment on how sensitive this method is to fine-tuning parameters (e.g., number of epochs, LoRA rank) would be useful.\n* For a more diverse test set, what happens if we fine-tune once on a mix of domains?\n* If non-member data in the specific domain is not available, is there any alternative in the general frame of this method?\n\n\n\n**Minor suggestions:**\n* If terms like \"Data Contamination\" or \"Membership Inference Attack\" are central, it might help to introduce them earlier (e.g., in the Introduction or Background). If not, why are they in the Related Work section?\n* Adding an ethics statement could underscore the method's role in detecting sensitive information used in LLM training, addressing the ethical significance of such methodologies.\n* Consider renaming either the “Method” section or subsection for clarity.\n* Start a distinct “Results” section.", "limitations": "", "rating": 8, "confidence": 3, "soundness": 4, "presentation": 3, "contribution": 3, "strengths": "* Clear introduction to the problem and motivation.\n* The method is well explained and supported with intuitive examples, such as Figure 2.\n* Comparisons with other relevant baselines, and showing significant improvement over them.\n* Thorough experiments, including ablation studies that address additional research questions.\n* Creative use of non-member data.", "weaknesses": "* The paper assumes access to unseen data in the same domain but doesn’t define 'domain' clearly. Could the authors explain how they handle differences between domains, especially if vocabularies differ a lot, and how they decide what data counts as 'same domain'?\n* *\"To the best of our knowledge, our method is the first to utilize some collected non-members in the task of pretraining data detection\"* - There’s limited information on how to collect non-member data, which seems a key aspect of your method.\n* The approach assumes access to both unseen data and model probabilities for each token, compared to other black-box assumption methods.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "dNbmBXaj1f", "reviewer": "Reviewer_h5aY", "summary": "This paper found that fine-tuning significantly reduces the perplexity of non-member data, which inspired the author to propose a new data detection method, FSD. The method determines whether data was used in training by comparing its perplexity in the original model versus the fine-tuned model. Experimental results showed a significant improvement over baseline methods.", "questions": "See weaknesses.", "limitations": "", "rating": 6, "confidence": 5, "soundness": 3, "presentation": 4, "contribution": 3, "strengths": "1. The improvement is very significant.\n2. The experiments and analysis are solid.\n3. The writing is good and easy to follow.", "weaknesses": "1. This method requires fine-tuning the models. However, many models do not support fine-tuning especially white-box models, making this approach impractical in reality.\n2. Fine-tuning every model you want to detect would be very costly.\n3. Meta-evaluation typically fine-tunes a model on the data that needs to be detected, in order to assess whether the detection method is effective. However, this method is similar to the meta-evaluation.\n4. The in-distribution non-member data is not easy to be acquired for fine-tuning.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}]

Author rebuttal:

Decision: Accept (Poster)