TDDBench: A Benchmark for Training data detection
Authors: Zhihao Zhu
Yi Yang
Defu Lian
Venue: ICLR 2025 Poster (2025)
Keywords: Training data detection; Benchmark; Copyright certification
Primary area: datasets and benchmarks

Abstract: Training Data Detection (TDD) is a task aimed at determining whether a specific data instance is used to train a  machine learning model. In the computer security literature, TDD is also referred to as Membership Inference Attack (MIA). Given its potential to assess the risks of training data breaches, ensure copyright authentication, and verify model unlearning, TDD has garnered significant attention in recent years, leading to the development of numerous methods. Despite these advancements, there is no comprehensive benchmark to thoroughly evaluate the effectiveness of TDD methods.
In this work, we introduce TDDBench, which consists of 13 datasets spanning three data modalities: image, tabular, and text. We benchmark 21 different TDD methods across four detection paradigms and evaluate their performance from five perspectives: average detection performance, best detection performance, memory consumption, and computational efficiency in both time and memory. With TDDBench, researchers can identify bottlenecks and areas for improvement in TDD algorithms, while practitioners can make informed trade-offs between effectiveness and efficiency when selecting TDD algorithms for specific use cases. Our extensive experiments also reveal the generally unsatisfactory performance of TDD algorithms across different datasets. To enhance accessibility and reproducibility, we open-source TDDBench for the research community at https://github.com/zzh9568/TDDBench.

Published as a conference paper at ICLR 2025
TDDBENCH: A BENCHMARK FOR TRAINING DATA DETECTION
Zhihao Zhu Yi Yang* Defu Lian
1 University of Science and Technology of China
The Hong Kong University of Science and Technology 2 2.h 980ma il csst.. edu. cn, imylyangeust. .hk 1ladeffuusst.eeduco
ABSTRACT
Training Data Detection (TDD) is a task aimed at determining whether specific data instance is used train machine learning model. In the computer rity literature, TDD is also referred to as Membership Inference Attack (MIA). Given its potential assess the risks of training data breaches, ensure copyright authentication, and verify model unlearning. TDD has garnered significant atten- tion in recent years, leading to the development of numerous methods. Despite these advancements, there is no comprehensive benchmark to thoroughly evalu- ate the effectiveness of TDD methods. In this work, we introduce TDDBench, which consists of 13 datasets spanning three data modalities: image, tabular, and text. We benchmark 21 different TDD methods across four detection paradigms and evaluate their performance from five perspectives: average detection perfor- mance, best detection performance, memory consumption, and computational ef- ficiency in both time and memory. With TDDBench, researchers identify bottlenecks and areas for improvement in TDD algorithms, while practitioners make informed trade-offss between effectiveness and efficiency when selecting TDD algorithms for specific use cases. Our extensive experiments also reveal the generally unsatisfactory performance of TDD algorithms across different datasets. To enhance accessibility and reproducibility, we open-source TDDBench for the research community at httpss//eiabbbboomm//25566610100
INTRODUCTION
Training Data Detection (TDD) Shi et al. 2024). also known as Membership Inference Attack (MIA) in computer security literature (Shokri et al. 2017). aims to determine whether a specific data instance was used to train a target machine learning model. TDD has a wide range of applications. For example, can be used to assess model's 's memorization of its training data and to audit the risks data leakage (Carlini et al. 2022b). TDD has gained even more importance in the era of deep learning and large language models (LLMss,, where models, often with billions of parameters, act as opaque black boxes. This raises the need to examine whether model owners have illegally utilized copyrighted material, such books (Abd-Alrazaq et al., 2023), or personal emails (Mozes et al., 2023). Moreover, TDD contributes to discussions on machine learning accountability in the era of AI, as concerns grow over how these models handle sensitive data. As machine unlearning becomes increasingly employed to remove users' personal data from models, TDD serves as a critical tool to validate these unlearning processes (Chen et al. 2021 Kurmanji al. 2024).
Given the growing importance of TDD, several benchmarks have been developed to evaluate TDD algorithms (Niu et al., 2023) He et al., 2022 Duan al. [024)). However, these benchmarks have several limitations: 1). Most evaluations primarily focus on TDD algorithms for image data, leaving other modalities like text and tabular data underexplored. 2). Many TDD methods developed in the past two years, particularly those focused on deep learning and LLMs, are not included in these benchmarks. 3). The effect the target model (i.e., the model that was trained using the data) on TDD algorithms has not been thoroughly examined. 4). Current evaluations focus primarily on the
cCorresponding authors

---

Published as a conference paper at ICLR 2025
detection performance of TDD algorithms, while practical considerations like efficiency, memory consumption, and other factors relevant to real-world deployment are often overlooked.
verfiiatot
verification
U Uniearning confimation
Investigation of training data memorization
Privacy risk assessment
benc hmark datasets 41 target models
(Section 2.2)
TDDBench
(Section 2.3)
21 detection algorithms data modalities
Soation
Section
Section
Soction
Q1: What is the overall performance TDD algorithi across different datasets and model architactures?
OZ: How impact performance of algorithm, including model size and training-data memortzatoo?
How does TDD algorithm knowledge about the target model architecture is limited?
04: the TDO algorithm perform performance, practicality efficiency, and other factors?
Figure 1: TDDBench in downstream applications and the benchmarking of TDD algorithms.
To address these limitations, we introduce TDDBench, a comprehensive framework for benchmark- ing TDD algorithms. Figure [ provides overview of TDDBench. The benchmark includes 13 datasets across three data modalities (tabular, text, and image) and evaluates 21 state-of-the-art TDD algorithms on 41 different target models, including the large language model Pythia- 12B. We also categorize the 21 TDD algorithms into four types based on their algorithmic characteristics, includ- ing metric-based learning- beasing-based, model-based, and query-based based. Using this new benchmark, we conduct extensive experiments to thoroughly assess TDD algorithms. Specifically, we aim to inves- tigate: 1). The performance of TDD algorithms across various datasets and data modalities. 2). The impact of the target model on TDD algorithms.
3). The limitations and areas for improvement in TDD algorithms. 4). The performance of TDD algorithms from multiple perspectives, including detection performance, practicality, and efficiency in terms of time and memory usage.
The experimental results reveal several key findings. First, there is significant performance gap be- tween different types of TDD algorithms, with the others. However, this outperformance comes at a cost, as model-based methods require building multiple reference models, leading to high computational expenses. Second, memorization of train- ing data plays crucial role in the performance of TDD algorithms, with larger target prone to ddemorization-exhiipe gorithms is highly dependent on knowledge of the underlying target model architecture. Overall, our experiments show that there is single best method across all scenarios, and notably, many TDD algorithms perform poorly on data modalities beyond images, indicating the need for further improvement non- non-image domains.
TDD methods generally outperforming
often higher TDD success rates. Third, the performance of TDD al-
The main contributions of this paper are threefold:
novel and comprehensive TDD benchmark: We introduce TDDBench, comprehensive, open- source benchmark for training data detection (TDD), encompassing three data modalities and 21 state-of-the-are TDD algorithms. Our experiments underscore the limitations of current TDD algo- rithms, which face challenges in handling diverse data modalities and various model architectures.
New insights in TDD performance: By benchmarking 21 state-of-the-arr TDD algorithms across various detection scenarios, we identify promising directions for future research in TDD, many of which have not been thoroughly explored in prior benchmarks.
Multi-aspect metrics:
Our comprehensive evaluation of TDD performance goes beyond simple de- tection accuracy to include practical considerations such as computational complexity, highlighting the trade-off's necessary for deploying TDD algorithms in real-world applications.
2 TDDBENCH: TRAINING DATA DETECTION BENCHMARK ACROSS MULTIPLE MODALITIES
2.1 PRELIMINARIES AND PROBLEM DEFINITION
Training Data Detection (TDD), also known as membership inference, is formally defined as fol- lows: Given target machine learning model fo and data point x, the objective of TDD is to
2

---

Published as a conference paper at ICLR 2025
determine whether the target model used the data point during its training phase (Shokri al. 2017 Carlini et al. 2022a). Here, 9 denotes the parameters of the target machine learning model, and fo often referred to the target model.
In this work, we consider black-box training data detection, meaning that we have access only to the outcomes of the target model for specific data points. There First, many real-world target models hold significant commercial value and typically do not pub- licly disclose model parameters, making access to internal model parameters infeasible. Secondly, existing literature has shown that white-box detection methods offer limited advantages compared black-box methods (Sablayrolles et 2019 Nasr et al. 2019).
reasons for this assumption.
2.2 DATA MODALITIES, DATASETS AND TARGET MODELS
TDDBench consists of 13 datasets across three data modalities: image, tabular, and text. It also implements 11 distinct model architectures for these data modalities, resulting in total of 41 target models. Additionally, TDDBench incorporates 21 state-of-the-ar TDD algorithms. We illustrate the main differences between the proposed TDDBench and existing benchmarks in Table
Table 1: Comparison between TDDBench and existing benchmarks. TDDBench comprehensively includes the most algorithms and datasets across image, tabular, and text modalities, as well as model architectures that encompass large language models.
Coverage
Data type tabular
Benchmark
# algo
# datasets
# architectures
LLM
image
text
He et al. 2022
6
4
x
Niu et al.
2023
15
7
X
Duan et al.
2024
8
X
X
/
TDDBench (ours)
21
13
11
presents presents summary of the datasets in TDDBench. It includes three data modalities: image, tabular, and text. TDDBench incorporates datasets commonly used to evaluate TDD algo- rithms in previous literatures (Truex et al., [019]] Hui et al., 2021), such as CIFAR-10 and Purchase. We also compile new datasets that potentially contain private or copyright-sensin sensitive information, including CelebA (human faces), BloodMNIST (medical), Adult (personal income), and Tweet (so- cial networks). which are more likely to necessitate TDD for tasks like copyright verification and unlearning confirmation. Additionally, WIKIMIA is dataset specifically designed evaluate TDD algorithms on large language models.
Dataset. Table
Table 2: Benchmarking datasets used in TDDBench.
Modality
Datasct
#Samples #Classes
Brief description
CIFAR-10
Krizhevsky al.
2009
60.000
10
General dataset
Image
110AR-100 Knzhevsky
2009
60.000
100
General dataset
BloudMNiST Yang el al. 2023
17,092
Medical image
CelehA [Liu et al. 2015
202,599
Human face
Purchase Shokri el al. 2017
197,324
100
Purchase record
Texas Shokn el al 2017
67,330
100
Hospital discharge data
Tahular
Adult Asuncion et 2007
48,842
Personal income
Student Cortez & Silva
2008
4,424
Education information
Rotten Tomatoes (Pang & Lec 2005
10,662
Movie reviews
Tweet Eval (Barbioni et al. 2020
12,970
User tweets
Text
GLUE-Col. (Wang ot al. 2018
9,594
Books and journal artieles
RCtHR Articles (hhalidds et al. 2023
5,063
Legal General dataset
WIKIMIA
Sin al.
1,650
Target Models. We select various model architectures for each data modality. Specifically, for image datasets, we train WRN28-2 (Zagoruyko) 2016), ResNet18 (He et al. 2016), VGG11 (Si monyan & Zisserman, 2014), and MobileNet-v2 (Sandler et al. 2018). For the tabular datasets, we employ Multilayer Perceptron (Kumelhart et al. 1986), CatBoost (Dorogush et al. 2018), and Logistic Regression (Hosmer Jr et al.
3

---

Published as a conference paper at ICLR 2025
For textual datasets, except WIKIMIA, in contrast to the target models for the image and tabular modalities, which are trained from scratch, we fine-tune the open-source pre-trained language mod- els DistilBERT (Sanh et al., 2019), RoBERTa (Liu et al., 2019), and Flan-T5 (Chung et all, 2024) on the text datasets, enabling us to detect fine-tuned data using the TDD algorithm. Finally, for the WIKIMIA dataset, use it to perform TDD on large language models, specifically focusing on the open-sourced Pythia(Biderman et al. 2023). Training details of the target models are presented in Appendix A.7
In summary, we implement different target models for each data modality. Since we have four image datasets, each with four target models, the total combination is 16 target image models. Similarly, for tabular data, four datasets and three target models give us total of 12 target tabular models. For text data, four datasets and three target models provide a total of 12 target text models, Finally, Pythia is used as the target model to examine TDD on the WIKIMIA text dataset. In total, we have 41 target models, which to our knowledge, is one of the most comprehensive benchmarks for TDD.
2.3 TDD ALGORITHMS
We implement 21 state-of-the-art TDD algorithms in IDDBench. To facilitate comparison and discussion, we categorize these TDD algorithms into four groups based on the algorithm's 's design paradigm: metric-based, learning-based,, model-based, and query-based algorithms. Table B pro- vides an overview of the implemented TDD algorithms in TDDBench, outlining their categories and detection criteria. These TDD algorithms are discussed in detail in Appendix
A.6 A.6
Table 3: Summary of training data detection methods in TDDBench.
Algorithm type
Algorithm
Detection criterion
Netric- c-ricclaaa Yeom et al. 2018 Met:loooon
Loss
Confidence
Metrio-bused
Metric-gesz Lenoo & Heansson 2020
Correctness
Matric-ent Shokki a di 2017 Siny & Minal
2021
Entropy
Metric ment [Sng
Modified prediction entropy
Tearn-orl g 1
Shokri al. 2017
Prediction vector
Learn-top3
Sa 201y
Top3 confidence
Learningbbsed
Leara-soxted
Salens de
Sorted prediction vector
Tearn-1 abel
(Nkk or 2013
Prediction vector, tae label
Leazn-mergs Amil
Merging of various detection criteria
Model-loss Sablaytolkes cl
2019
Loss
Xodel-ca-ibe Ton Con Waatoo ot
2012
Loss
Model based
Model-1ira Carlihi
Scaled logit
Model-Ep2 el Pizzz
Scaled logit Confidence
Xodel-robust
Zantzithh di 2014
Choqueto-Chooo
Correctaess
Snary--rrancert (L3 & Zhhing 2023
Loss from surrogate model
Query-based
Query-ade [i & Zhang 2021 Uliouuetc dn al.
2021
Distance from the decision houndary
Query-neightn ggoor Dayaraman
Query y- (erriin ara 1024
Scaled logit
Query-re.
Scaled logit
Metric-based based methods rely on the analysis of certain statistical properties of a target model's out- put, such as confidence scores, prediction probabilities, or loss values, to distinguish between train- data. Specifically, detric-loss (Yeom et af.] 2018) is the first metric-
ing data and non-trainin based detection method, predicting that data points with loss below a certain threshold are part of the training data for the target model. Similarly, other works have proposed using the maximum con- fidence of the target model output (denoted as Metric conf (Song et al., 2019)). the correctness of the target model output (denoted as prediction probability distributions (denoted as Metricc-ee (Shokri et al. 2017: Song & Mittal
(Leino & Fredrikson, 2020)), the entropy of
202 )), and modified entropy of the prediction (denoted as Metric- (Song & Mittal 2021]
Learning-based methods involve training an auxiliary classifier (meta-classifier) to distinguish be- tween training data and non-training data. In the literature, neural networks (NNs) are often em- ployed as the auxiliary classifier. The primary differences between learning-based TDD methods lie in the choice of input features for the auxiliary classifier. Earlier work (Shokri et al. 2017) has pro- posed using the original prediction vector of the target model (denoted as Learn oriiiiiignn)) Other works have suggested using the top-3 prediction confidences (denoted Learn-top3 (Salem et al. 2019)). , the sorted prediction vector (denoted as Learn- (Salem et al. 2019) the true label of the example combined with the prediction vector (denoted Learn- Learn-label
4

---

Published as a conference paper at ICLR 2025
Nast et al. 2018) and mix of different detection metries (denoted as Learn-merge (Amit et al. 2024). In black-box TDD scenarios, a shadow model is constructed to mimic the behavior of the target model, providing the necessary data to train the auxiliary classifier.
Model-based methods involve building multiple reference models, some of which are trained with the focal data point æ; while others are trained without it. The detection method then analyzes the characteristics (such as loss distribution) of data points when they are included in training versus when they are not. The target model's output on the focal data point is then compared the refer- ence models' characteristics to determine whether it was used in training. Compared to metric-based and learning- based methods, model- based methods do not solely rely on the target model's output, but can compare it with reference models. These methods have gained significant attention in recent years due their superior performance. In the literature, different model-based methods utilize reference models in various ways, including learning the loss distribution of data points (denoted as Model-loss (Sablayrolles et al., 2019) and Model-cal brat ion (Watson et al. (02T)) ) transforming TDD into likelihood ratio problem based on the scaled logits of prediction results (denoted as Model livellliie -iira (Carlini et al. 2022a)), designing TDD that satisfies different false pos- itive ratios (denoted Model-for ( Ye et al. 2022) ) and creating more robust TDD methods (denoted as -rodel-robust (Zarifzadeh et al. 2024)).
Query-based methods involve using additional data instances, particularly those close to the focal data point J, to query the target model. Compared to the other three types of detection meth- ods, query-based methods leverage likelihood that the focal data point was used in model training. Specifically, we consider a data sugmentation-basel query method (denoted a neighbor-based method (denoted as Query neighbor (Jayaraman et 2021] Mattern et al. 2023), surrogate model- method (denoted as Query-transfer (Li & Zhang 2021)), an adversarial bearning-base method (denoted as Query-adv (Li & Zhang 2021 Choquette Chooo al. 2021) ), quantile regression model- based method (denoted as Query-qrm (Bertran et al.,
output information from the target model estimate the
(Choquette-
et al. 2021 ).
2024 and a reference-model--eee based query method (denoted as Query-ref (Wen et al. 2023)).
also worth noting that different types of TDD methods may have varying requirements and assumptions for executing the detection. For example, metric-based methods have the fewest as- sumptions, relying solely on the target model's output for prediction. In contrast, some model-based and query-based methods require additional auxiliary data to build reference models for prediction. In TDDBench, to ensure a fair comparison, we provide auxiliary data for methods that need it, ensuring that each method achieves its best possible detection performance.
EXPERIMENT RESULTS AND ANALYSES
Having compiled TDDBench, we now benchmark the performance of TDD algorithms. Since TDD algorithms can largely be categorized into four types based on their design paradigms, our exper- imental analysis conducted at the category level. This allows us systematically compare the strengths and weaknesses of each type of TDD algorithm.
We conduct experiments in three modalities including image, tabular, and text, answer the follow- ing questions: Q1: What is the overall performance of the TDD algorithm across different datasets and model architectures? Q2: How does the target model impact the performance of the TDD al- gorithm, including model size and training-dats memorization? Q3: How does the TDD algorithm perform when knowledge about the target model architecture is limited? Q4: How does the TDD algorithm perform in terms of overall performance, practicality, efficiency, and other factors?
3.1 EXPERIMENT SETTING
Evaluation Protocol. We follow prior literatures in TDD evaluation (Carlini et al.) 2022a Ye et al.) 2022). Specifically, given a dataset TDDBench, we divide the dataset into a target dataset and an auxiliary dataset in a 50:50 ratio. For the target dataset, we further split it into two halves, where the first half serves as the training dataset to train the target model (e.g., an image classifier), and the remaining half is not used in training the target model. Therefore, the training dataset serves as the positive examples for training data detection, while the remaining data serves as the negative examples.
5

---

Published as a conference paper at ICLR 2025
For TDD algorithms, such as model-based and learning-based methods that require training refer- ence models or shadow models, we follow the approach in (Carlimi et al. 2022a Wen et al. (0033) by randomly partitioning the target dataset multiple times to train various reference and shadow models. The auxiliary dataset, also referred to as the population dataset in (Ye et al. 2022) and the shadow dataset in (Shokri al.] 2017), is available at the user's discretion for use in the TDD algorithms.
The auxiliary and target datasets do not overlap, ensuring that the auxiliary data is not accidentally used in training the target model. This characteristic allows for the training of quantile regression model and reference model that exclude the focal data point which are utilized in certain TDD algorithms.
Target Model Implementation. We implement target models as described in Section 2.2. Tech- niques such as early stopping, data augmentation, and dropout are utilized to maximize the target model's predictive accuracy (e.g., for tasks like image classification or sentiment analysis). The training and test accuracy of the target models, along with detailed training information, can be found in Appendix A.7
TDD Method Implementation. For the metric-based TDD methods, as they rely solely on the target model's prediction outcome, the implementation is straightforward For the learning-based TDD methods, we construct a two-layer neural network with 64 and 32 hidden units as the auxiliary classifier. The learning rate is set to 0.001, using the Adam optimizer, and training continues until the validation accuracy does not improve for 30 epochs or until a maximum of 500 epochs is reached. For the model-based TDD methods, TDD methods, including Query-ne- neig-nerrroo, ghbor, Query augment, and Query-res we limit the detection algorithms to a maximum of 10 additional queries per data point.
train 16 reference models. Finally, for the query-based
Evaluation Metrics, Mean, and Standard Deviation. TDD is framed as binary classification problem that determines whether a data point was used in training the target model. Accordingly, we primarily use AUROC to evaluate the performance of TDD algorithms. Additionally, we include nine supplementary metrics, such as Precision, Accuracy, and TPR@ 1% FPR. with detailed exper- imental results provided in Appendix A.. To ensure the robustness of the experimental results, we perform multiple random partitions for each dataset and independently repeat the experiments five times. We then report the average performance of all TDD algorithms. Standard deviations across the five repeated experiments are also measured, and due to page limitations, the complete standard deviation results reported in Appendix A.8
3.2 OVERALL DETECTION PERFORMANCE ACROSS DIFFERENT DATASETS AND MODELS
The main results from benchmarking TDD algorithms are presented in Tables and 5 Specifically, Table 4 reports the average performance of TDD methods across different datasets, controlling for the same target model architecture within each modality. Table on the other hand, presents the average performance of TDD methods across different target model architectures, benchmarked on CIFAR 10 for image data, Purchase for tabular data, and Rotten Tomatoes for text data. Additionally,
results involving large language models are illustrated in Figure 4(C) in Section 3.3.2 The results
lead to several key findings:
Overall performance is not satisfactory. In most experimental settings, the AUC scores range between 0.5 and 0.6. From an AUC perspective, this is clearly unsatisfactory, as it indicates a high rate of false negatives and false positives. In other words, data points used by the target model are frequently misclassified as not being used, and vice versa. This is concerning and highlights the urgent need for advancing the performance of TDD methods.
Model-based TDD methods achieve generally better detection performance. Across datasets and target models, model-based detection algorithms consistently outperform other methods. For instance, as shown in Table 4. all five model-based algorithms achieve an average performance near or above 0.65 across all 12 datasets, whereas the performance of metric-based and learning-baat based methods is substantially lower. Overall, these results highlight the performance advantage of model- based TDD methods.
Data's task label information is useful. Some TDD methods leverage the focal data point's 's ground truth label (c.g.. image class label or sentiment class), while others do not. Experimental results demonstrate that incorporating label information significantly improves detection performance. For

---

Published as a conference paper at ICLR 2025
instance, Metric intriccment consistently outperforms Metric-ent by utilizing data labels. Simi- lar improvements are observed with Learn-labes compared to Leazn-original where the former benefits from leveraging the label information while the latter does not.
Hybrid method has potential. Notably, Query-ref, which generates crafted query data for the image modality, achieves the best performance among all 21 TDD algorithms. While categorized as query-based, this method also trains reference models, similar to model-based methods, making it a hybrid of query-based and model-based approaches. This highlights the potential of combining the merits of different methods to enhance detection accuracy.
Table 4: AUROC of TDD algorithms across different datasets. WRN28-2, Multilayer Perceptron, and DistilBERT are trained on image, tabular, and text datasets, respectively. The last column of each table displays the average performance of the corresponding TDD algorithm across different datasets. Complete results with standard deviations are provided in Table 16 in the Appendix A.8
Modality Dataset
Image
Tabslor
Text
CIFAR-10
CIPAR-100
BoodMNNSS
CalebA
Parchase
Texas
Adult
Student
Rones
Theet
CoL.
BCJR
Het: ric loxx
0.635
ILASS
0.527
0.509
0.619
0.629
0.500
0.566
0.582
0.566
0,571
0.521
0.590
Metric cun.
0.635
ILSS3
0.527
0.509
0.619
0.629
0.500
0,566
0,582
0.566
0,571
0.521
0.590
Matric-cozz
0.552
11708
0.517
1.507
0.331
0.510
0
0.560
0.557
0.550
0.550
0.519
0.557
Hotric-ent
0.625
01843
0.525
0.508
0.616
0.563
0.498
0.520
0.561
0.525
0.519
0.507
0.568
Motric-mant
0.35
0.55
0.527
0.509
0.620
0.630
0,500
0,566
0.582
0.566
0.571
0.523
0.591
Tearn-origina]
0631
0.70
0.508
0.500
0.652
0.597
0,502
0.531
0.548
0.579
0.588
0.506
0.590
Tearn-top]
BA26
0851
0.525
0.501
0.677
0.573
0.500
0.520
0.561
0.578
0.53
0,500
0.575
Learn worted
BA28
0850
0.529
0.508
0.996
0.575
0.501
0.520
0.561
0.528
0,510
0.501
0.575
Learn Jabe]
B633
0882
0.515
0.507
0.616
0.569
0,503
0.590
0.584
0.570
0.622
0.117
0.604
Leaen Twege
0.666
0L893
0.525
0.507
0.684
0.686
0.502
0.515
0.584
0.569
0.620
0L530
01612
Hodel-loss
0.664
0.852
0.560
0.725
0.767
0,509
0.670
0.773
0.756
0.752
0665
0,844
Hodel-osllbrrtien
0.39
0.663
0.555
0.530
0.630
0.718
0.508
0.548
0.685
0.714
0.69
063
0.648
Hodel-11za
0.90
0.937
0.536
0.12
0.755
0.753
0.503
0.534
0.753
0.728
0753
0604
0.679
Hodel-fpr
0.547
0883
0.552
0.516
0.897
0.725
0.907
0.611
0.679
0.722
0.708
0635
0
Nodel-robest
0.755
0
0.552
0.762
0.509
0.569
0.766
0.745
0.746
0621
0,677
Query-Auement
0.573
0.761
0.517
0.612
6,500
0.560
0.570
0.551
0.561
0.588
0.570
Query-2rarsser
0.522
0.622
0.581
0.499
0.522
0,530
0.526
0.10
0.531
Query-ady
0.615
0.838
0.508
0.579
0.563
0.511
0:568
0.19
0.579
Query-ncichhor
0.511
0.553
0.612
0.535
0.550
0.523
Query-qen
0.532
0.574
0.530
0.526
0.501
0511
0.512
0.522
Query-ret
0.735
N/A
N/A
N/A
Table 5: AUROC of TDD algorithms across different target model architectures. MLP stands for Multilayer Perceptron, and LR stands for Logistic Regression. The last column of each table dis- plays the average performance of the corresponding TDD algorithm across different model archi- tectures. Complete results with standard deviations are provided in Table 17 in the Appendix A.8
Datasel
model model
WRN28-2
ResNet18
VUG11
MobilaNet-v2
MLP
PunchnseTabbllar CalBoost
LR
DistBBERT'
Redien-Komattee(Tea)) RoBERIo
Flan:
Avg.
Mac-ioo Met merrona o-0-f
0,635
0.659
0.654
0.592
0.619
0.948
0.640
0.582
0.571
0.517
0.645
BU635
0.659
0.684
0.592
0.619
0:948
0.640)
0.5M2
0.571
0.517
0.645
Metric-ccrr
0.552
0.557
0.574
0.515
0.551
0.836
0.622
0.557
0.542
0,513
0.565
Mel oeteeoe
BA228
0.654
(1600
0.582
9.516
0.345
0.594
0.561
0.555
0.509
0.632
0,635
0.659
0,685
0.592
0.620
0.950
0.642
0.582
0,571
0.517
0.645
Ceasn- originn
0.631
0.625
0.094
0.535
0.682
0.535
0.644
0.558
0.546
0.515
Tearn-top3
0.678
0.653
0.600
0.582
0.967
0.660
0.561
0.509
56asn-5o2t.d
0.628
0.654
0.630
0.578
0.666
0.963
0.661
0.561
0.509
Seacn-label
0.633
0.612
0.707
0.557
0.656
0.951
0.701
0.584
0.565
0.520
0.619
Seasa-nerqe
0.656
0.628
0.727
0.328
0.684
0.968
0.716
0.584
0.566
0.518
0.657
Modee2 losu
0.564
0.709
0.729
0.647
0.725
0.975
0.776
0.773
0.656
0.602
0.721
Modaa cabbaatton
0.639
0.671
0.690
0.395
0.684
0.865
0.719
0.695
0.622
0.592
0.677
Model T.
1.690
0.749
(L780
0.601
IL755
11995
0.761
0.753
0.630
0.569
0.728
0.617
0.684
0.712
0.619
0.697
0.976
0.726
0.679
0.623
0.589
0.695
Model-robuse
0.635
0.677
0.704
0.602
0.983
0.7%6
0.766
0.639
0.574
3very-augment
0.573
0.575
0.633
0.542
0.612
0.570
0.512
0.592
Quezy-Lrans
0.522
0.522
0.533
0.507
0.529
0.587
0.574
0.530
0.515
0.506
Onery-adv
0.615
0.621
0.666
0.583
RU620
0.727
0.662
0.571
0.552
0,516
query-neighhoo
0.511
0.512
0.509
0.509
0.533
0.820
0,530
0.533
0.527
0.504
0.548
Query-grm
0.532
0.537
0.541
0.530
0.523
0.9466
0.632
0.524
0.528
0.506
0. 0.500
Quezy-ro1
0.735
0.800
0.543
0.656
NAA
N/A
N/A
N/A
N/A
N/A
0.759
3.3
THE IMPACT OF TARGET MODEL
In this section, we examine the impact of the target model TDD detection performance.
3.3.1
DATA MEMORIZATION AND ÖVERHITTING
A common view is that the effectiveness of TDD is closely tied to the level of training- -ata memo- rization overfitting exhibited by the target model during training (Yeom al. 2018 Long al.)
7

---

Published as a conference paper at ICLR 2025
2018). The disparity between the target model's accuracy the training set and the test set, known as the train-test test accuracy gap, serves as an indicator of data memorization in prior literature (Car-) document the train-test gaps of 12 distinct target models in Table 4 along with the corresponding performance of all detection methods. Specifically, the training of target models is repeated five times with different random training samples. Figure il- lustrates the performance of various detection algorithms across different train-test gaps, with error bars representing 95% confidence intervals obtained from five independent trials. It is evident that the performance of all TDD methods is positively correlated with the train-test accuracy gap of the target model.
limi et al. 2022a). In experiments,
Takeaway. It is crucial for future advanced TDD algorithms to evaluate the generalizability of target models and report TDD performance when the train-test gap small.
0.55
0.85
0.90
Mode
0.50
Modil caend
0.80
Model basrd
0.85
based
Netri: Sated
Mardi
Querycseedd
Qanry su
Quity boord
0.80
AUROC
0.77
AUROC
0.70
0.65
0.65
0.60
0.5
0.55
0.55
0.59
0.50
0.50
0.1
0.4
0.00
0.05
0.25
5.00
0.12
accuracy gap
accuracy gAP
accuracy
(a) Image
(b) Tabular
(c) Text
Figure 2: TDD algorithm performance (AUROC) versus the target model's train-test accuracy gap. The reported performance is averaged across all datasets.
3.3.2 TARGET MODEL SIZE
We examine the impact of target model size on TDD performance. Specifically, in our experiment, vary the number of layers in the ResNet architecture for image data, the number of hidden units in the MLP for tabular data, and the parameter sizes the large language model Pythia for text data.
Due to limitations in computing resources, TDD on large models often does not involve the creation of shadow models or reference models. Drawing from prior studies (Shi et al. 2024: Duan et al. 2024), utilize multiple detection methods suitable for pretrained large language models. Detailed descriptions of these detection methods can be found in Appendix A.2 Additionally, due to the lack of specific information regarding the training data used for large language models, utilize the WIKIMIA (Shi et al. 2024), which collects training and non-training data for the large language model based on the model's release timeline evaluate the TDD method in large language models.
The results of the experiments are illustrated in Figure 4 It is observed that, in most cases, the performance of the detection method improves as the size of the model increases. This aligns with the expectation that an increase in model size typically enhances model memorization (Carlini et al., 2023 Arpit al., 2017). However, an exception oceurs when the number of layers in the ResNet model is expanded from 34 to 50, resulting in decline in the detection method's performance. One potential explanation for this anomaly is that the integration of residual connections in ResNet helps alleviate issues related to excessive memorization stemming from the increased depth of the model.
Takeaway. TDD algorithms generally demonstrate improved performance as the model size in- creases, highlighting their potential in the era of large models.
3.4 PERFORMANCE WHEN KNOWLEDGE ABOUT THE TARGET MODEL IS LIMITED
In the above experiments, we assumed that despite the black-box setting, TDD algorithms had some knowledge about the target model's training algorithm. However, in real-world scenarios, it is pos- sible that the data owner may lack detailed knowledge about the target model's architecture, leading to significant differences between the reference and shadow models constructed by the TDD method and the actual target model. To explore this issue, we assess the performance of TDD when the reference and shadow models differ from the target model.
8

---

Published as a conference paper at ICLR 2025
0.60
0.80
0.85
Hoddl banid
0.75
Learring 2aeed
Pettic Baied
9.75
Leart Eg Saaid
Merickased
0.80
MIDOC
0.70
0.70
AUROS
Miigtaa
0.65
0.65
0.60
0.50
0.60
9.55
0.55
10
18
34
12
255
160N
4100
6.08
Number of lyees
Number of hiddon units
Number of parameters
(a) Image:ResNet
(b) Tabular:MLP
(c) Text:yyhii
Figure 3: TDD algorithm performance (AUROC) versus model size, measured by the number of layers in ResNet, the number of hidden units in MLP, and the number of parameters in large language models Pythia.
The results, presented in Table show a noticeable decline in detection performance when the data owner has limited knowledge about the target model's architecture. For example, Learn-origina exhibits a 5.3% performance decline when using ResNet18 as the shadow model. This performance degradation can be attributed discrepancies between the shadow and target models, which result in biased input features for training the auxiliary classifier.
Takeaway. The overall performance of TDD algorithms, without knowledge of the target model's training algorithm, remains unsatisfactory. This underscores the ineffectiveness of TDD algorithms most datasets when information about the target model is limited.
Table 6: TDD algorithm performance (AUROC) when the reference or shadow models are different from the target model (i.e., when knowledge about the target model limited). Complete results with standard deviations provided in Table 18 in Appendix A.
Target model
Shadow/lefereneee model
WRN28-2
ResNett8
WRRNS-2(CIPAR-II
VGGI1
MohulaNee-v2
MLP
MI.PPPuchase) CatHlosst
LR
DeniBBEKT
DisilBERT SanilBERKTRottc--oonnaaec
RoBEXIa
Flas- 15
Avg.
Searn-originat
0.631
0.578
0,632
0.539
0.652
0.651
0.564
0.558
0.560
0.546
0.591
0.628
0.628
0.628
0.528
0.646
0.515
0.561
0.561
0.678
0.629
0.678
0.629
0.666
0.532
0.561
Sewsn label
0.33
0.644
0.563
0.666
0.551
0.580
Leann-merge
0,656
0.581
0.651
0.509
0.684
0.517
0.595
0.584
0.584
0.580
Woddslloon
0.561
0.657
0.611
0.532
0.725
0.503
0.611
0.773
0.607
0.589
0.651
99ode--se i-aiaatttioi..i
0.639
0.634
0.617
0.614
0.684
0.579
0.588
0.695
0.595
0.587
0.623
Modai-1iza
0,690
0.659
0,665
0,510
0,755
0.696
0.588
0.753
0.602
0.553
0.656
Model:
0.647
0.668
01638
IL664
0.697
D1645
0.643
0.679
0.557
0.567
0164
96ca_-robust
0.635
0.639
0.633
0.521
0.711
0.632
0.625
0.766
0,624
0,591
0.648
Quecy augnent
0.565
Query-tranczer
0.529
0.514
Quezy-qrn
0.532
0.532
0.524
0.528
Qnery-ref
0.740
0.708
N/A
3.5 PERFORMANCE TRADEOFF OF TDD ALGORITHMS
Table 7: Quantitative evaluation of different types of TDD algorithms including the average and best AUROC, maximum runtime and memory usage.
Algorithm type
Average performance
Best performance
Running time(s)
Memory usage(MB)
Metric-hased
0.626
0.645
232
0
Leaming--used
0.646
0.657
2,107
855
Model-based
0.706
0.728
4,089
13,680
Query-based
0.604
0.759
40,963
13,680
Most evaluations of TDD algorithms primarily focus detection accuracy. However, other factors, such as computational efficiency, are equally important in real-world applications. For instance, model-based methods, which require building numerous reference models, may be too costly in terms of time and memory when applied to large Al models. Therefore, TDDBench, we empha- size the computational complexity of running different TDD algorithms. Specifically, we document

---

Published as a conference paper at ICLR 2025
the maximum runtime and memory usage for each type of TDD algorithm. This provides holistic evaluation beyond detection accuracy. We present an overall assessment of the four types of TDD algorithms in T Evidently, each type of TDD algorithm has its own advantages and disadvan- tages. While model-based methods offer the best average performance, they come with significantly higher running time and memory usage. Therefore, data owners performing TDD must strike bal- ance between practicality, resource utilization, and detection accuracy, depending on their specific scenario. For instance, in resource-constrained environments, metric-based methods are suitable choice for TDD, as they require minimal computational resources and fewer assumptions compared to other methods.
Takeaway. None of the TDD algorithms are satisfactory, as performance improvements often ne- cessitate increased consumption of computing resources.
RELATED WORK
Training data detection (TDD) is commonly employed to assess privacy risks in machine learning models (Murakonda & Shokri 2020). has been applied across various domains, including image classification (Hui al., 2021 text generation (Shejwalkar et al. (021). graph neural networks (Wu et al.) 2021). and recommendation systems (Zhang et al., 2021). TDD has a wide range of applications such dataset copyright protection (Maini al. 202 and for verifying machine
unlearning (Chen et al.) 202T). Shokri et al.) 2017 introduce the first TDD algorithm, utilizing shadow models to help identifying differences the model's predictions for training data versus other data. Yeom et al.) 2018 demonstrate that satisfactory results could be achieved by utilizing only the loss of the target model on the sample. [Callini et al.] 2022a criticize methods based solely on the target model's output, arguing that they overlook the inherent characteristies of the data, which can lead to biased estimates regarding whether sample belongs to the training dataset. They propose training multiple reference models to better understand how the sample's characteristics influence metrics like loss. There is a rapidly growing body of literature on TDD methods, and we provide brief summarization in Section 2.3
Existing benchmarking works. He et al. 2022 evaluate 9 TDD algorithms on image data, while Niu et al. 2023 expand the evaluation to 15 algorithms, focusing how sample differences within datasets affect TDD performance. Duan et al. 2024 investigate five TDD algorithms on large lan- guage models (LLMMs) and find that current TDD algorithms perform poorly in this context. In summary, existing TDD benchmarks have limited coverage of data modalities and algorithms, un- derscoring the need for a more comprehensive analysis of TDD algorithms. This paper, along with the developed TDDBench, aims to address this gap by providing in-depth insights into the develop- ment and performance tradeoff in state-of oo-thh--rtttttt-- TDD algorithms.
CONCLUSIONS
In this article, we introduce TDDBench, novel and comprehensive training data detection bench- mark. Unlike existing benchmarks, TDDBench extends evaluations across multiple data modalities, including image, tabular, and text. It also includes large language models and benchmarks 21 state- of-the-art TDD algorithms. Our comprehensive evaluation sheds critical light on the development of TDD algorithms and helps both researchers and practitioners reconsider the trade-offs involved in using TDD algorithms. Based on our findings with TDDBench, we believe future work on TDD algorithms should focus on, but not be limited to: (I) designing TDD algorithms robust to target models less prone to overfitting, (2) creating TDD algorithms that require minimal knowledge of the target model architecture, (3) achieving a better balance between performance and practical con- siderations such as computational complexity, and (4) developing algorithms tailored to specific application contexts or data modalities, such as training data detection for large language models and recommendation systems.
6 ACKNOWLEDGEMENTS
The work was supported by grants from the National Natural Science Foundation of China (No. 224A20253).
10

---

Published as a conference paper at ICLR 2025
REFERENCES
Alaa Abd-Alrazeu Rawan AlSaad, Dari Alhuwail, Arfan Ahmed, Padraig Mark Healy, Syed Latifi, Sarah Aziz, Rafat Damseh, Sadam Alabed Alrazak, Javaid Sheikh, et al. Large language models in medical education: opportunities, challenges, and future directions. JMIR Medical Education, (1):e482,,, 2023.
Guy Amit, Abigail Goldsteen, and Ariel Farkash. Sok: Reducing the vulnerability of fine-tuned language models membership inference attacks. arXiv preprint arXiv:2403. 088481 2024.
Devansh Arpit, Stanislaw Jastrzębski, Nicolas Ballas, David Krueger, Emmanuel Bengio, Maxin- der S Kanwal, Tegan Maharaj, Asja Fischer, Aaron Courville, Yoshua Bengio, et al. A closer look at memorization in deep networks. International conference on machine learning, pp- 233-242. PMLR, 2017.
Arthur Asuncion, David Newman, al. Uci machine learning repository, 2007.
Kyungjune Back and Hyunjung Shim. Commonality in natural images rescues gans: Pretraining gans with generic and privacy-free synthetic data. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 7854-7864, 2022.
Francesco Barbieri, Jose Camacho- CollacooC Luis Espinosa Anke, and Leonardo Neves. Tweete- val: Unified benchmark and comparative evaluation for tweet classification. In Findings of the Association for Computational Linguistic:: EMNLP 2020, pp. 1644-1650, 2020.
Martin Bertran, Shuai Tang, Aaron Roth, Michael Kearns, Jamie H Morgenstern, and Steven Z Wu. Scalable membership inference attacks via quantile regression. Advances in Neurul Information Processing Systems, 36, 2024.
Stella Biderman, Hailey Schoelkopf, Quentin Gregory Anthony, Herbie Bradley, Kyle O' O'Brien, Eric Hallahan, Mohammad Aflah Khan, Shivanshu Purohit, USVSN Sai Prashanth, Edward Raff, et al. Pythia: A suite for analyzing large language models across training and scaling. In International Conference on Machine Learning, pp. 2397-2430. PMLR, 2023.
Nicholas Carlini, Chang Liu, Úlfar Erlingsson, Jemnej Kos, and Dawn Song. The secret sharer: Evaluating and testing unintended memorization in neural networks. In 28th USENIX security symposium (USENIX security 19), pp. 267-284, 2019.
Nicholas Carlini, Florian Tramer, Eric Wallace, Matthew Jagielski, Ariel Herbert- Voss, Katherine Lee, Adam Roberts, Tom Brown, Dawn Song, Ulfar Erlingsson, al. Extracting training data from large language models. In 30th USENIX Security Symposium (USENIX Security 2/), pp- 2633-2650, 2021.
Nicholas Carlini, Steve Chien, Milad Nasr, Shuang Song, Andreas Terzis, and Florian Tramer. Mem- bership inference attacks from first principles. In 2022 IEFE Symposium on Security and Privacy (SP), pp. 1897-1914. IEEE, 2022a.
Nicholas Carlini, Matthew Jagielski, Chiyuan Zhang, Nicolas Papernot, Andreas Terzis, and Florian Tramer. The privacy onion effect: Memorization is relative. Advances in Neural Information Processing Systems, 35:13263-13276 2022b.
Nicholas Carlini, Daphne Ippolito. Matthew Jagielski, Katherine Lee, Florian Tramer, and Chiyuan Zhang. Quantifying memorization across neural language models. The Eleventh International Conference Learning Representations, 2023.
Ilias Chalkidis, Nicolas Garneau, Cătälina Goantä, Daniel Katz, and Anders Søgaard. Lexfiles and legallama: Facilitating english multinational legal language model development. In Proceedings of the 6lst Annual Meeting of the Association for Computational Linguisties (Volume 1: Long Pupers), pp. 15513-15533 2023.
Min Chen, Zhikun Zhang, Tianhao Wang, Michael Backes, Mathias Humbert, and Yang Zhang. When machine unlearning jeopardizes privacy. In Proceedings of the 2021 ACM SIGSAC confer- on computer and communications security, pp. 896-911, 2021.

---

Published as a conference paper at ICLR 2025
Christopher A Choquette- Choo, Florian Tramer, Nicholas Carlini, and Nicolas Paperno. Label-only membership inference attacks. In International conference on machine learning, pp. 1964-1974. PMLR. 2021.
Hyung Won Chung, Hou, Shayne Longpre, Barret Zoph, Yi Tay, William Fedus, Yunxuan Li, Xuezhi Wang, Mostafa Dehghani, Siddhartha Brahma, et al. Scaling instruction-finetune lan- guage models. Journal of Machine Learning Research, 25(70):1-53, 2024.
Paulo Cortez and Alice Maria Gonçalves Silva. Using data mining to predict secondary school student performance. Proceedings of Sth Anmual Future Business Technology Conference, 2008.
Anna Veronika Dorogush, Vasily Ershov, and Andrey Gulin. Catboost: gradient boosting with categorical features support. arXiv preprint M.Kiv:8810.11366 2018.
Alexey Dosovitskiy. An image is worth 16x16 words: Transformers for image recognition at scale.
arXiv preprint wwXXiv22000.19222 2020.
Michael Duan, Anshuman Suri, Niloofar Mireshghallah, Sewon Min, Weijia Shi, Luke Zettlemoyer, Yulia Tsvetkov, Yejin Choi, David Evans, and Hannaneh Hajishirzi. Do membership inference attacks work on large language models? arXiv preprint arXiv:2402.07841, 2024.
Jamie Hayes, Luca Melis, George Danezis, and Emiliano De Cristofaro. Logan: Membership infer- ence attacks against generative models. arXiv preprint 1rXiv: 2017.
2017.
Kaiming He, Xiangyu Zhang, Shaoqing Ren, and Jian Sun. Deep residual learning for image recog- nition. In Proceedings of the IEEE conference computer vision and pattern recognition, pp. 770-778, 2016.
Xinlei He, Zheng Li, Weilin Xu, Cory Cornelius, and Yang Zhang. Membership-doctor: Compre- hensive assessment of membership inference against machine learning models. arXiv preprint arXiv:2208. 10445, 2022.
David W Hosmer Jr. Stanley Lemeshow, and Rodney X Sturdivant. Applied logistic regression. John Wiley & Sons, 2013.
Bo Hui, Yuchen Yang. Haolin Yuan, Philippe Burlina, Neil Zhenqiang Gong, and Yinzhi Cao. Practical blind membership inference attack via differential comparisons. arXiv preprint arXiv:2101.01341, 2021.
Bargav Jayaraman, Lingxiao Wang, Katherine Knipmeyer, Quanquan Gu, and David Evans. Re- visiting membership inference under realistic assumptions. Proceedings Privacy Enhancing Technologies, 2021.
Jinyuan Jia, Ahmed Salem, Michael Backes, Yang Zhang, and Neil Zhenqiang Gong, Memguard: Defending against black-box membership inference attacks via adversarial examples. In Pro- ceedings of the 2019 ACM SIGSAC conference on computer and communications security, pp- 259-274, 2019.
Yigitcan Kaya and Tudor Dumitras. When does data augmentation help with membership inference attacks? In International conference on machine learning, pp. 5345-5355. PMLR, 2021.
Alex Krizhevsky, Geoffrey Hinton, et al. Learning multiple layers of features from tiny images. 2009.
Meghdad Kurmanji, Peter Triantafillou, Jamie Hayes, and Eleni Triantafillou. Towards unbounded machine unlearning. Advances in neural information processing systems, 36, 2024.
Klas Leino and Matt Fredrikson. Stolen memories: Leveraging model memorization for calibrated [White-Boxs Box} membership inference. In 29th USENIX security symposium ( (USENIX Security 20), pp. 1602-1622, 2020.
Zheng Li and Yang Zhang. Membership leakage in label-only exposures. In Proceedings of the 2021 ACM SIGSAC Conference on Computer and Communications Security, pp. 880-895. 2021.
12

---

Published as a conference paper at ICLR 2025
Yinhan Liu, Myle Ott, Naman Goyal, Jingfei Du, Mandar Joshi, Danqi Chen, Omer Levy, Mike Lewis, Luke Zettlemoyer. and Veselin Stoyanov, Roberta: A robustly optimized BERT pretraining approach. CoRR, abs/1907.11692 2019.
abs / 1907 .1692
Ze Liu, Yutong Lin, Yue Cao, Han Hu, Yixuan Wei, Zheng Zhang, Stephen Lin, and Baining Guo. Swin transformer. Hierarchical vision transformer using shifted windows. In Proceedings the IEEE/CVF international conference on computer vision, pp- 10012-10022, 2021.
Ziwei Liu, Ping Luo. Xiaogang Wang. and Xiaoou Tang. Deep learning face attributes in the wild. In Proceedings of International Conference on Computer Vision (ICCV), December 2015.
Yunhui Long, Vincent Bindschaedler, Lei Wang, Diyue Bu, Xiaofeng Wang, Haixu Tang, Carl A Gunter, and Kai Chen. Understanding membership inferences on well-generalized learning mod- cls. arXiv preprint arXiv: 04889, 2018.
Pratyush Maini, Mohammad Yaghini, and Nicolas Papernot. Dataset inference: Ownership resolu- tion in machine learning. arXiv preprint SEt::2204.107066 2021.
Justus Mattern, Fatemehsadat Mireshghallah, Zhijing Jin, Bernhard Schoelkopf, Mrinmaya Sachan, and Taylor Berg-Kinng Kirkpatrick. Membership inference attacks against language models via neigh- bourhood comparison. In Findings of the Association for Computational Linguistics: ACL 2023, pp. 11330-1133,, 2023.
Maximilian Mozes, Xuanli He, Bennett Kleinberg, and Lewis Griffin. Use of Ilms for illicit purposes: Threats, prevention measures, and vulnerabilities. arXiv preprint arXiv:2308. 12833, 2023.
Sasi Kumar Murakonda and Reza Shokri. Ml privacy meter: Aiding regulatory compliance by quantifying the privacy risks machine learning. arXiv preprint arXiv:2007.09339 2020.
Milad Nasr, Reza Shokri, and Amir Houmansadr. Machine learning with membership privacy using adversarial regularization. Proceedings of the 2018 ACM SIGSAC conference on computer and communications security, pp- 634-646, 2018.
Milad Nasr, Reza Shokri, and Amir Houmansadr. Comprehensive privacy analysis of deep learning: Passive and active white-box inference attacks against centralized and federated learning. In 2019 IEEE symposium on security and privacy (SP), pp. 739-753. IEEE, 2019.
Jun Niu, Xiaoyan Zhu, Moxuan Zeng. Ge Zhang, Qingyang Zhao, Chunhui Huang, Yangming Zhang, Suyu An, Yangzhong Wang, Xinghui Yue, et al. Sok: Comparing different membership inference attacks with comprehensive benchmark. arXiv preprint 4rX1v:2337.061/23, 20233
Bo Pang and Lillian Lee, Seeing stars: Exploiting class relationships for sentiment categorization with respect to rating scales. In Proceedings the 43rd Annual Meeting of the Association for Computational Linguistics (ACL' '05), pp. 115-124 2005.
David E Rumelhart, Geoffrey E Hinton, and Ronald J Williams. Learning internal representations by error propagation, parallel distributed processing, explorations in the microstructure of cognition, ed. de rumelhart and meclelland. vol. 1. 1986. Biometrika, 71(599-607):6 1986.
Alexandre Sablayrolles, Matthijs Douze, Cordelia Schmid, Yann Ollivier, and Hervé Jégou. White- box vs black-box: Bayes optimal strategies for membership inference. In International Confer- ence on Machine Learning, pp. 5558-5588558 5567. PMLR, 2019.
Ahmed Salem, Yang Zhang, Mathias Humbert, Pascal Berrang, Mario Fritz, and Michael Backes. MI-leaks: Model and data independent membership inference attacks and defenses on machine learning models. In Proceedings 2019 Network and Distributed System Security Symposium. Internet Society, 2019.
Mark Sandler, Andrew Howard, Menglong Zhu, Andrey Zhmoginov, and Liang Chich Chen. Mo- bilenetv2: Inverted residuals and linear bottlenecks. Proceedings of the IEEE conference on computer vision and patlern recognition, pp. 4510 4510-4520, 2018
13

---

Published as a conference paper at ICLR 2025
Victor Sanh, Lysandre Debut, Julien Chaumond, and Thomas Wolf. Distilbert, distilled version of bert: smaller, faster, cheaper and lighter. ArXiv, abs/1910.01100, 2019.
Virat Shejwalkar, Huseyin A Inan, Amir Houmansadr, and Robert Sim. Membership inference at- tacks against nlp classification models. In NeurIPS 2021 Workshop Privacy in Machine Learning, 2021.
Weijia Shi, Anirudh Ajith, Mengzhou Xia, Yangsibo Huang, Daogao Liu, Terra Blevins, Danqi Chen, and Luke Zettlemoyer. Detecting pretraining data from large language models. In The Twelfth International Conference on Learning Representations, 2024.
Reza Shokri, Marco Stronati, Congzheng Song, and Vitaly Shmatikov. Membership inference at- tacks against machine learning models. 2017 IEEE symposium on security and privacy (SP), pp. 3-18. IEEE, 2017.
Karen Simonyan and Andrew Zisserman. Very deep convolutional networks for large-scale image recognition. arXiv preprint arXivv1409.153: 2014.
Kihyuk Sohn, David Berthelot, Nicholas Carlini, Zizhao Zhang, Han Zhang, Colin A Raffel, Ekin Dogus Cubuk, Alexey Kurakin, and Chun-Liang Li. Fixmatch: Simplifying sepi-super learning with consistency and confidence. Advances in neural information processing systems, 33:596-608, 2020.
Liwei Song and Prateek Mittal. Systematic evaluation privacy risks of machine learning models. In 30th USENIX Security Symposium (USENIX Security 21), pp. 2615-2632, 2021.
Liwei Song, Reza Shokri, and Prateck Mittal. Privacy risks of securing machine learning models against adversarial examples. In Proceedings of the 2019 ACM SIGSAC conference computer and communications security, pp. 241-257, 2019.
Hugo Touvron, Thibaut Lavril, Gautier Izacard, Xavier Martinet, Marie-Anne Lachaux, Timothée Lacroix, Baptiste Rozière, Naman Goyal, Eric Hambro, Faisal Azhar, et al. Llama: Open and efficient foundation language models. arXiv preprint arXiv:2302. 13971, 2023.
Stacey Truex, Ling Liu, Mchmet Emre Gursoy, Lei Yu, and Wenqi Wei. Demystifying membership inference attacks in machine learning service. IEEE transactions services computing, 14 (6):2073-2089, 2019.
Alex Wang, Amanpreet Singh, Julian Michael, Felix Hill, Omer Levy, and Samuel R Bowman. Glue: A multi-task benchmark and analysis platform for natural language understanding. EMNLP 2018, pp. 353, 2018.
Lauren Watson, Chuan Guo, Graham Cormode, and Alexandre Sablayrolles. On the importance of difficulty calibration in membership inference attacks. In International Conference on Learning Representations, 2021.
Yuxin Wen, Arpit Bansal, Hamid Kazemi, Eitan Borgnia, Micah Goldblum, Jonas Geiping, and Tom Goldstein. Canary in a coalmine: Better membership inference with ensembled adversarial queries. In The Eleventh International Conference on Learning Representations, 2023.
Bang Wu, Xiangwen Yang, Shirui Pan, and Xingliang Yuan. Adapting membership inference at- tacks gnn for graph classification: Approaches and implications. In 2021 IEEE International
Conference on Data Mining (ICDM), pp. 1421-1426. IEEE, 2021.
Jiancheng Yang, Rui Shi, Donglai Wei, Zequan Liu, Lin Zhao, Bilian Ke, Hanspeter Pfister, and Bingbing Ni. Medmnist v2-a large-scale lightweight benchmark for 2d and 3d biomedical image classification. Scientific Data, 10(1):41, 2023.
Jiayuan Ye, Aadyaa Maddi, Sasi Kumar Murakonda, Vincent Bindschaedler, and Reza Shokri. En- hanced membership inference attacks against machine learning models. In Proceedings of the 2022 ACM SIGSAC Conference on Computer and Communications Security, pp. 3093-3106, 2022.
14

---

Published as a conference paper at ICLR 2025
Wentao Ye, Jiaqi Hu, Liyao Li, Haobo Wang, Gang Chen, and Junbo Zhao. Data contamination calibration for black-box 1lms. arXiv preprint arXiv:240S.119300 2024.
Samuel Yeom, Irene Giacomelli, Matt Fredrikson, and Somesh Jha. Privacy risk in machine learn- ing: Analyzing the connection overfitting. In 2018 IEEE 31st computer security foundations symposium (CSF), pp. 268-282. IEEE, 2018.
Zuobin Ying, Yun Zhang, and Ximeng Liu. Privacy-preserving in defending against membership inference attacks. In Proceedings the 2020 Workshop on Prevecy- Machine Learning in Practice, pp. 61-63, 2020.
Sergey Zagoruyko. Wide residual networks. arXiv preprint arXiv: 2016.
Sajjad Zarifzadeh, Philippe Liu, and Reza Shokri. Low-cost high-power membership inference attacks. In Forty-first International Conference on Machine Learning, 2024.
Minxing Zhang, Zhaochun Ren, Zihan Wang, Pengjie Ren, Zhunmin Chen, Pengfei Hu, and Yang Zhang. Membership inference attacks against recommender systems. In Proceedings of the 2021 ACM SIGSAC Conference on Computer and Communications Security, pp. 864-879, 2021.
15

---

Published as a conference paper at ICLR 2025
A APPENDIX
A.1 TRAINING DATA DETECTION ON VISION TRANSFORMER MODELS
In this section, showcase the performance of the TDD algorithm on vision transformer models. Specifically, we trained ViT (Dosovitskiy) 2020) and Swin (Liu et al. 2021) models on the CIFAR- 10 dataset and evaluated the TDD algorithm's effectiveness on these models. As shown Table and Table 9 the TDD algorithm remains effective on vision transformer models, and model-based algorithms continue to have clear advantages over other types of methods.
Table 8: TDD performance across different metrics on ViT (Dosovitskiy 2020) trained on CIFAR- 10 dataset. MA (mmmberssh advantage) (Jayaraman et al. 2021) equals the difference between the true positive rate and the false positive rate. For all metrics except for FPR and FNR, higher values indicate better performance of the corresponding TDD algorithm.
Algorithm
Precision
Recall
Fl Fl-score
Acc
FNR 1
FPR
MA
ISPP
TPR@ 10%FPR
AUROC
Metric-loss
0.555
0.811
0.659
0.583
0.190
0.644
0.167
0.010
0.119
0,599
Metric-conf
0.555
0.811
0.659
0.583
0.190
0.644
0.167
0.010
0,119
0,599
Metric-com
0.555
0.204
0.000
Metric-ent
0.536
0.499
0.517
0.516
0,501
0.428
0.071
0,011
0.114
0.543
Metrie-ment
0,555
0.659
0.5K3
0.190
0.011
0.121
0.599
Lean--original
0.522
0.675
0.589
0.532
0.325
0.612
0.063
0.013
0.122
0.537
Leam-top3
0.539
0.477
0.506
0.536
0.523
0,405
0.072
0.010
0,116
0.541
Leaa--sorted
0.539
0.474
0.504
0.536
0.526
0.402
0.072
0.010
0.116
0.541
Learn-label
0.552
0,787
0.649
0.577
0,213
0.634
0,153
0.015
0,141
0.597
Leam--merge
0.556
0.799
0.655
0.583
0.201
0.634
0.165
0.016
0.138
0.604
Model-loss
0.596
0.718
0.651
0.617
0.283
0.483
0.235
0.056
0.244
0.672
Muubel-culibration
0.578
0.764
0.658
0.606
0.236
0.553
0.212
0.046
0.222
0.653
Model-lira
0.562
0,735
0.637
0.584
0,265
0.567
0.168
0.052
0,219
0.631
Model-fpe (pr
0.611
0.610
0,401
Modcl-rebust
0.603
0.666
0.633
0.615
0.334
0.436
0.231
0.056
0.255
0.671
Dugry-augment
0.557
0.581
0.229
0.003
0.067
Query trunsfer
0.525
0,738
0.614
0.518
0,262
0.662
0.077
0.001
0,102
0.538
Query-ady
0.564
0.207
0.008
0.119
Query-ncighbor
0.504
0.393
0.442
0.505
0.607
0.383
0.010
0.001
0.061
0.506
Query-qu -mm
0.585
0.801
0.656
0.583
0.199
0.636
0.165
0.000
0.000
0.597
Query ref'
0.676
0.558
0.611
0.649
0.442
0.260
0.298
0.089
0.308
0.718
Table 9: TDD performance across different metrics on Swin (Liu et al.) 2021) trained on CIFAR-10 10
dataset. MAmmembership advantage) (Jayaraman et al. 2021 equals the difference between the true positive rate and the false positive rate. For all metrics except for FPR and FNR, higher values indicate better performance of the corresponding TDD algorithm.
Algorithm
Precision
Recall
Fi-score
Acc
FNR
FPR
MA
@PR@1SFPR
TPR 10%EPR
AUROC
Metric-loss
0,571
0.855
0.685
0.609
0.146
0.636
0.218
0.009
0.116
0.621
Metric-conf
0.571
0,855
0.685
0.609
0.146
0.636
0.218
0.009
0.116
0.621
Metric-corr
0.085
0.000
Metric-ent
0.547
0.690
0.611
0.562
0,310
0.566
0.124
0.009
0.115
0.573
Mettie-meut
0,571
0.685
0.609
0.144
0.009
0.117
Lear- Lragioriginal
0.548
0.685
0.609
0.563
0.315
0.559
0.126
0.010
0.132
0.577
Leam-tto33
0.553
0.562
0.375
Leam-sortedd
0.553
0.624
0.586
0.562
0,376
0.500
0.124
0.009
0.116
0.573
Leam-label
0.571
0.857
0.686
0.610
0.143
0.638
0.219
0.024
0,169
0.638
Leam-merge
0.574
0,838
0.681
0.610
0.162
0.618
0.220
0.020
0.172
0.643
Model-loss
0.623
0.693
0.656
0.639
0.307
0.416
0.278
0.076
0.275
0.704
Model-calibrution
0.581
0,810
0.676
0.615
0.190
0.580
0.230
0.061
0.224
0.668
Model-lira
0.590
0,772
0.669
0.621
0.228
0.531
0,241
0,083
0.278
0.686
Model-fpe
0.615
0.636
0.625
0.621
0.364
0.394
0.242
0.058
0.275
0.670
Model-rohust
0.606
0,760
0.674
0.635
0,240
0.489
0.271
(L083
0.300
0.705
Query augment
0.567
0.679
0.603
0.153
0.009
Query trunsfer
0.542
0,850
0.662
0.570
0.150
0.711
0.139
0.009
0.105
0.560
Query-ady
0.577
0.713
0.068
0.008
0.155
Query-neighbor
0.505
0.422
0.460
0.506
0.578
0,410
0.012
0.002
0.079
0.507
Query -pm
0.567
0.866
0.686
0.606
0.134
0,655
0.211
0.000
0.000
0.623
Query ref"
0.639
0.842
0.726
0.689
0.158
0.464
0.378
0.128
0.383
0.759
16

---

Published as a conference paper at ICLR 2025
A.2 TRAINING DATA DETECTION ALGORITHMS IN LARGE LANGUAGE MODELS
In this section, we offer a brief introduction the TDD algorithms for large language models; for more detailed information, please refer to the related works.
Loss (Yeom al. 2018) refers to the Metric- -oss mentioned in the article. Instead of using cross-entropy as in classification models, the log likelihood of each text under the target model serves the basis for detection in pretraining language models.
Zlib (Carlini et al. 2021
calibrates the sample's 's loss under the target model using the sample's zlib
compression size.
MIN-K % (Shi et al. 2024) utilizes the k% of tokens with the lowest likelihoods as the detection
basis, rather than average loss.
Reference (Carlini et al. 2021) borrows from model-based approaches, utilizing the reference model to help correct the detection basis derived from the prediction results of the target model. Reference models for TDD in large language models are typically open-source and have architec- tures similar to the target model, thus avoiding the significant computational cost of training the reference model from scratch.
Neighbor, or Quezy-neighnn -eiinbbiiooo... (Mattern et al. 2023). supplements the detection information provided by the sample point with the loss of the target model on the neighboring samples of æ.
PAC, short for Polarized Augment Calibration (Ye et al. 2024). introduces new detection metric called polarized distance through data augmentation. This metric helps determine whether data has been trained by large language models.
Additionally, we present the performance of the TDD algorithm various sizes of Llama models TTovrron et al. 2023). The experimental results Table 10 indicate that the performance of the TDD algorithm improves as the model size increases, which aligns with the results observed for the detection results on Pythia (corresponds to Figure 4 Section 3.3.2)
Table 10: The performance of the TDD algorithm across different sizes of Llama models.
Target Model
Algorithm
Llama-7b
Llama- 13b
Llama-65b Llama-30b
Neighbor
0.555
0.552
0.566
0.586
LOSS
0,666
0.678
0.704
0.707
PAC
0,679
0.689
0.704
0.714
Zlib
0.683
0.697
0.718
0.721
MIN-K%
0,697
0.715
0.737
0.737
Reference
0.802
0.809
0.833
0.831
A.3 TRAINING DATA DETECTION IN SEMI-SUPERVISED LEARNING SCENARIOS
Research on TDD algorithms has primarily concentrated on two types of training methods. The first supervised learning, which forms the basis for most TDD algorithms, covering various fields such as image (Carlini et al. 2022a), table (Shokri et al. 2017). and text (Amit et al. 2024). This is also the setting for our main experiments. The second type self-supervised learning which typically focuses on detecting whether the pre-trained corpus of large language models can be identified. This type of algorithm also known as pretraining data detection (Shi et al 2024), and our experiments on Llama and Pythia evaluated the TDD algorithm's 's performance in this setting.
In this part, we conduct experiments on the CIFAR-I0 and CIFAR-100 datasets to assess the ef- fectiveness of TDD algorithms in semi-supervised learning scenarios, opposed to the more com- monly studied supervised learning setting. To this end, we evaluate the performance of TDD on WRN28-2 trained using FixMatch (Sohn al.) 2020), state-of-the-arr semi- remi-supervised learning method. This investigation aims to provide insights into the versatility and robustness of TDD algo- rithms in diverse learning settings.
17

---

Published as a conference paper at ICLR 2025
Table 11: TDD performance WRN28-2 trainied with semi-supervised method FixMatch (Sohn ct al. 2020).
Dataset
CIFAR-10
CIFAR-100
Algorithm
Accuracy
Precision
Recall
F1-score
Accuracy
Precision
Recall
Fl-score
Metric-loss
0.624
0,560
0.805
0.661
0.714
0,598
0.721
0.653
Metric-cont
0.624
0,560
0.805
0.661
0.714
0.598
0.721
0.653
Metric-corr
0.533
0.487
0,910
0.635
0.683
0.531
0.786
0.634
Metric-ent
0.643
0,567
0.870
0.687
0.736
0,587
0,820
0.684
Metric-ment
0.624
0,561
0.804
0.661
0.714
0.602
0.712
0.652
Learn-original
0.641
0,564
0.889
0.690
0.737
0,585
0.832
0.687
Learn-to33
0.643
0.569
0.862
0,685
0.733
0,591
0.801
0.680
Learn-sorted
0.643
0.568
0.865
0.686
0.734
0.591
0.804
0.681
Learn-label
0.617
0,550
0.835
0.664
0.722
0,584
0.778
0.668
Learn-merge
0.621
0.560
0.789
0,655
0.717
0.607
0.712
0.655
Model-loss
0.620
0,561
0.775
0.651
0.721
0.606
0.726
0.661
Model-calibratin
0.604
0,546
0.774
0.641
0.690
0.555
0.737
0.633
Model-lira
0.614
0.555
0.778
0.648
0.720
0.628
0.685
0.655
Model-fpr
0,597
0.555
0.664
0.605
0.694
0.613
0.625
0.619
Model-robust
0.589
0.526
0.655
0.605
0.745
0.668
Query-augment augment
0.575
0.515
0.652
0.697
0.631
Query-transfer
0.531
0.496
0.589
0.539
0.595
0.453
0.699
0.550
Query-ady
0.607
0.561
0.753
0.643
0.749
0.649
0.769
0,704
Query- neightr
0,505
0.473
0.582
0.522
0,515
0.393
0,470
0.428
Query-qrm
0.611
0.562
0.710
0,628
0.697
0,595
0.671
0.631
Query-ref 0.642
0.540
0.735
0.623
0.738
0.548
0.711
0.619
The experimental results lead to the following conclusions: The TDD detection method remains effective semi-supervised training, but its performance declines compared to supervised learning- Specifically, the model-basee method, which shows clear advantages in supervised learning, per- forms moderately in the semi-supervised setting. This may be because the model-based approach relies training reference model, and its performance is significantly affected when the reference model is unaware of the cup--super learning method used by the target model.
Future direction. Based on the experimental findings, believe that investigating TDD algorithms tailored to specific training methods represents a promising and impactful research direction.
A.4 DEFENSE STRATEGIES AGAINST TRAINING DATA DETECTION
In the field of computer security, training data detection is known as Membership Inference Attack, which aims to extract private information about the training data from target models. To counteract this detection, various measures (Baek & Shim 2022 Ying et al.) 2020) have been proposed. Since the effectiveness of training data detection is often linked to the degree of overfitting in the target model, many defense methods focus on reducing overfitting. These methods include dropout strate- gies Saaem et al.) 2019), label smoothing (Kaya & Dumitras 2021), early stopping (Song & Mittal and data augmentation (Kaya & Dumitras 2021).
2021
Beyond reducing model overfitting, another key defense strategy involves modifying the output vector of the target model lower the risk of training data leakage. For instance, Jia et al.) 2019 suggests adding carefully designed noise to the model's output vector, which does not affect the target model's 's performance but can mislead detection algorithms. [hhkki al. 2017 recommends constraining the target model to output only prediction labels without confidence scores, rendering many TDD algorithms ineffective.
In addition to these common methods applicable across different data types, Hayes et al. 2017 em- ploys differential privacy to prevent external parties from determining whether specific data was used in generative model. Their results indicate that differential privacy can balance model us- ability with defense effectiveness. [Shejwalkar et al., 2021 proposed a defense strategy based on knowledge distillation, demonstrating that the distilled model can better resist training data detec- tion.
To better assess the robustness of TDD algorithms, examine their performance when the target model is combined with various defense strategies. Specifically, we selected four general defense strategies: using dropout and label smoothing the target model mitigate overfitting, and altering
18

---

Published as a conference paper at ICLR 2025
the target model's output vector to noise vectors and hard label. Our experimental results across three datasets indicate that these defense strategies, particularly the addition of noise, can effectively diminish the performance of TDD algorithms. TDD algorithms with strong performance, such as those that are learning-baie based and output vectors. Introducing small amounts of noise to the model output can significantly compromise the effectiveness of these TDD algorithms.
heavily depend on the authenticity of the model's
Future direction. Based on these findings, we suggest that counter potential defense mechanism of the target model, a promising direction is to develop adaptive TDD approaches, which involve designing more effective TDD algorithms tailored to specific defense strategies.
Table 12: TDD performance under different defense strategies.
Daiaset
CIPARl0tinage
Purchase
tottaness
Nun:
Desportt
Lackonly
Nore
Drspect
Naio:
Label-unly
Labs--anly
Memicslea
0.635
0.587
IL589
0,588
NIA
0.620
0.615
0.657
0.623
N/A
0.582
0,577
0,590
0.587
N/A
Menic cour
0.635
0.557
1.539
0.558
NIA
0.620
0.615
0.657
0.623
N/A
0.592
0.577
0.5M0
0.567
NAA
Matri--arr
0.552
0.547
0.557
0.549
0,552
0.351
0.558
0.556
0.551
0.51
0.557
0.546
0.555
0.534
ILS57
Metric oor
8.628
01543
1150%
0.583
N/A
0616
0.006
0.655
0.620
N/A
0:561
0.54
0.573
U.70
N/A
Meniz-meal
0.635
0.589
0.58
N7A
0.620
0.615
0.656
0.623
N/A
0.377
0,500
0.587
NIA
oiiitdl
0,493
0.539
0.52
0.67
0.659
0.558
0.574
NA
Leamsopp
0.26
0.545
0.995
0.54]
NYA
0.77
0.66
0,622
0.600
NIA
0.561
0.560
0,573
0.570
NIA
Leam sorted
0.655
0.543
1157%
0343
NAA
11657
0.655
0.651
0.678
NA
0.561
0.56
0,513
0.570
NIA
Lesm-label
0634
0.537
0.565
0.554
VNA
0.686
0.629
0.635
0.622
NA
0.564
0.578
0.591
0.588
NIA
Leom merge
0.666
01.89
10009
0.589
VM
11084
UAMI
0A64
0.098
N/A
0.5%1
0357
0.5K3
0.992
NIA
Midel
0.726
0.%
Miskl-caiiraiin
0.39
0.52
0.64
Midel-lia
0.600
11561
0364
0,755
0A98
0
0.7%
0483
0674
0
NIA
Misds-fje
0.647
0.571
0.515
0.563
VWA
0.697
0.511
0.547
0.703
NIA
0,67
0653
0.537
0.655
NIA
Midel-inbrett
0,635
16611
8413
NYA
0711
0.92
0.761
0.12
N/A
0.766
0492
0,708
0730
NA
Qutry aaryy-
0.511
0.586
0.33
0,62
0,612
0.599
Qsery mander
0.575
8,13
0.991
0.596
0.339
0.607
0.623
0.599
0.53
Qsery neighbor
0665
11:841
0151
0.115
0.332
0.544
N/A
0.51
0523
NAA
Duare-gea
0.532
0.575
0.63
0.5%
SIA
0.23
0.617
0,658
0.650
N/A
0.534
0,574
0,5M0
0,589
NA
Query-nel
0.735
0.694
1.750
0.407
N/A
N/A
N/A
N/A
N/A
N/A
N/A
N/A
N/A
N/A
NA
A.5 TRAINING DATA DETECTION WITH DIFFERENT SIZES OF DATA POINTS
To examine the effect of the size of evaluated data points on the TDD algorithm, we varied the size of the target dataset and assessed the algorithm's 's performance on target models trained with these different dataset sizes. As shown in Figure 1, the model-based -bsed method consistently delivers the best detection performance across various sizes, which aligns with earlier findings. Furthermore, there is no strong correlation between data size and the performance of the TDD algorithm.
Macelzaoeed Merictared Ourrs
Model basrd Matti-baeed
0.80
0.75
Past coaeed Queryhaod
0.75
0.75
Caity bunel
AUROC
0.70
AUROC
AUROC
0.70
0.65
0.60
0.60
0.55
0.55
6.0k
12.0k
18.0k
24.0k
30.0k
20.0k
40.0k
80.0k
100.0k
1.kk
4.0k
Site
data points
Size of data points
50ze data points
(a) IIFAe:CIFAR10
(b) Tabular:Purchase
(c)
TextRootie-tommato
Figure 4: TDD algorithm performance (AUROC) versus data size, measured by the number of data points in the target dataset.
A.6 DETAILS OF THE ALGORITHMS INCLUDED IN TDDBENCH
A. METRIC-BASED DECTECTION
Several studies (Carlini et al. 2019 2021 indicate that models retain a certain degree of memory regarding the training data during the learning process. This memorization result in significant differences between the model's predictions on training and test data, which can be leveraged as decision basis for TDD algorithms. Specifically, Metric-loss (Yeom et al. 2018) utilizes
19

---

Published as a conference paper at ICLR 2025
the loss of the target model's prediction data points the detection criterion. Since the target model is instructed to minimize training loss during optimization, a training data point typically exhibits lower loss than test data point. Similarly, Metric-conf (Metric-confidencc) (Song et al. 2019) identifies that the maximum confidence of the target model's predictions can also serve the detection criterion. Metric-cort (Metriss tettric-oor- c--cor (Leino & Fredrikson 2020) further demonstrates that even without access to the model's prediction confidence or logits for a specific data point, comparing the predicted label with the true label can provide an effective detection basis. Metric-com achieves training data detection (TDD) with fewer assumptions than both Metric-loss and Metric-conf
Beyond individual prediction values, the distribution of prediction results can also serve as a valu- able detection criterion. Metric-ent (Metric-entropp) (Shokri et al. 2017 Song Mittal 2021) posits that the target model exhibits greater confidence in its predictions for training data, as evidenced by more concentrated distribution of prediction confidences across different classes. Building on this, entropy is utilized as the detection criterion for Metric serr--eett.. Metric-ment (Metric-modified entropy) (Song & Mittal 202T) further incorporates the true label of the data point into Me i c-ent to prevent the detection algorithm from predicting data points where the target model has misclassified as its training data.
A.6.2 LEARNING- DECTECTION
The metric-based based algorithms design various metrics to extract detection basis from the prediction results of the target model. However, manually designed metrics may not accurately capture the differences between the predicted results of training and test data. A more robust approach is to use neural networks to automatically extract training data detection (TDDD)-friendly information from the target model's 'sssi predictions, known as learning- basdd detection algorithms. Learn-origina (Shokr et al. 2017 7) is the first algorithm to propose building an auxiliary classifier for TDD. It inputs the prediction vector of the target model into the auxiliary classifier, aiming directly produce detection result. To train the auxiliary classifier, Learn-original employs a shadow model similar to the target model, utilizing shadow training techniques. Since the shadow model's 's training process is conducted by the detectors, they can obtain both the training and test data of the shadow model. The predictions made by the shadow model on its training and test data are utilized to facilitate the training of the auxiliary classifier.
Different learning-based detection algorithms primarily differ in the input features of their auxiliary classifiers. For instance, Learn-top3 and Learn-sorted (Salem et al. 2019) utilize the top-3 prediction confidences and ranked prediction vectors as input features, respectively. Building on Learn-label (Nast et al.] (0188) supplements the input features with the true label of the data point. Learn-merge (Amit et al., 2024) further incorporates the entropy, loss, and predicted label into the input features. It is noteworthy that while Learn-or's g nal builds multiple shadow models, most learning-basse methods utilize only one. Moreover, previous work demonstrates that training data detection (TDD) with a single shadow model achieves perfor- mance comparable to that of multiple shadow models. Therefore, to ensure a fair comparison among learning- based methods, standardize the number of shadow models to one for all staming-based approaches.
Learn-original
A.6.3 MODEL-BASED DECTECTION
The two lines of TDD methods discussed above rely solely on the prediction results of the target model. overlooking the inherent characteristics of the data points, which may introduce bias into the detection criteria. For example, abnormal training data may exhibit higher losses than normal test data due to inherent data characteristics (Carlini et al. 2022b making it challenging for Metric-los to detect these data. Therefore, the design of the detection criterion must consider data characteristics to climinate bias.
Model-based algorithms address this issue by utilizing a set of reference models that share a similar architecture to the target model. These reference models are used to obtain predictions for data point across different models, which helps to de-bias the detection criteria of the target model. To elab- orate, Model-loss (Sablayrolles et al. 2019) calculates the mean loss of data point x across all reference models and then subtracts the loss from the target model to eliminate bias induced by data
20

---

Published as a conference paper at ICLR 2025
characteristics. In contrast, els that exclude data point x from its training data, allowing it to implement model-basee based TDD for any new data point. Moreover, Model-lira (Carlini et al. 2022a) treats the detection process likelihood ratio test, determining whether the rescaled logit value of data point x in the target model originates from models trained
WWassnn et al. 202 uses only reference mod-
Building Model 1 1 ra, Model-fpr (Ye al. 2022) designs detection method that meets the specified arbitrary false positive ratio. Model-robut Zarilzadeh al. 2024) introduces robust TDD method that utilizes only one reference model.
A.6.4 QUERY-BASED DECTECTION
The motivation for based algorithms stems from two main reasons. Firstly, some of them aim implement label-only training data detection (TDD), where the target model provides only pre- dicted labels. In such cases, the detector can depend solely on prediction correctness as the detection criterion, which limits the ability acquire more intricate and effective detection information. To address this limitation, Query-augment 202T] proposes obtaining multiple neighbors of data point z through data augmentation. The cor- rectness of the target model on these augmented data points is then combined to form input features for the auxiliary classifier in the learning-basee algorithm. Query-trans on the other hand, suggests training surrogate model based the prediction labels of the target model. The surrogate model is expected to closely resemble the target model and subsequently re- place it to provide more detailed prediction results for arbitrary data points, enabling the generation of a more intricate detection criterion. Moreover, cueer-advvQueeri S (Li & Zhang 2021 Choquette- Choo al., 202I) considers the distance of a data point from the target model's decision boundary as a detection criterion with the aid of adversarial tools. This is based on the assumption that training data will generally be farther from the decision boundary than test data.
mttyyaaatttttmmmnttoooo (Choquette-
et al.)
(Li Zhang, 202T),
Another type of query-based algorithm does not assume that the target model only returns prediction labels. In these algorithms, additional queries are introduced to provide more information to aid in detection. For example, Query-neighbor (Jayaraman et al., 202T Mattern et al. 2023) adds random noise to the data point x and uses the difference between the loss of the target model on and its average loss on the neighboring points of z as the detection criterion. Query-qrm (Bertran jet al. 2024) collects a large amount of data that is explicitly not from the target model's training data and obtains the scaled logits of the target model on these samples to train quantile regression model. This quantile regression model can determine the likelihood that ær not part of the target model's training data. Additionally, extra queries for adversarial samples generated based on reference models. These samples help to better reflect the differences in the predicted results of x when is training data versus when is not.
(eaeyrrre(uuryyy seery-rrfereee rerencce) (Wen et al. 2023) makes
Remark. Query-ref is categorized query-based method rather than reference- based method because of its innovative query sample generation strategy. It is specifically designed to generate suitable query data for image datasets, rather than for tabular text data.
A.6.5 HOW TO OPERATE A MODEL-BASED OR QUERY-BASED TDD ALGORITHM
In this section, we outline the implementation of Model-based and Query- algorithms. Specifi- cally, we demonstrate how to train reference model for focal data in the Model- algorithms, as well as how to obtain additional query results in the Query-based algorithms. By following these steps in Alg T and Alg 2 you can effectively implement both Model-based and Query-based TDD algorithms.
A.7 PERFORMANCES AND TRAINING DETAILS OF TARGET MODELS
In Table 13 we present the accuracy of the target model across various datasets. Table 14 extends this evaluation by showcasing the accuracy of the target model under diverse model architectures. Additionally, Table [5] complements these results by detailing the training configurations and hy- perparameters used for each model, offering transparency and reproducibility for the experimental setup.
21

---

Published as a conference paper at ICLR 2025
Algorithm 1: How to train reference models in Model-based TDD algorithms
Input: Reference dataset D, focal data æ, target model f, number of reference models N;
Output: Whether was used to train
for N times do
Sample subset from DD:
Train a reference model using the combined dataset D d
Obtain z'ss detection metric (e.g. loss) from this reference model, which is trained with Train a reference model using the dataset \ d;
Obtain x's detection metric (e.g. loss) from this reference model, which is trained without
end
Obtain x's detection metric from the target model f Implement Model-based TDD using the detection criterion from reference models and target
model:
Algorithm 2: How to obtain extra queries in Query-based TDD algorithms
Input: Focal data æ, target model *f*, number queries per sample N; Output: Whether
used to train f
for N times do
Modify the data point x based the chosen data augmentation strategy (e.g., add noise, flip):
Input the modified data d' into the target model *f* obtain the query results
end
Implement training data detection using the query results obtained from the different modified data points:
Table 13: Training accuracy and test accuracy of target models trained on different datasets (corre- sponds Table in Section WRN28-2, Multilayer Perceptron, and DistilBERT are trained on image, table, and text datasets, respectively. Typically, target models trained on datasets with more categories exhibit smaller test accuracy and greater train-test accuracy gaps.
Modality
Dataset
# Classes
Train accuracy
Test accuracy
Train- test accuracy gap
CIFAR-10
10
0.981
0.877
0.104
CIFAR-100
100
1.000
0.417
Image
BloodMNIST
8
0.955
0.034
CelcbA
2
0.988
0.976
0.013
Purchase
100
1.000
0.897
0.103
Texas
100
0.766
0.546
0.220
Tabular
Adult
2
0.831
0.830
0.001
Student
3
0.855
0.735
0.121
Rotten tomatoes
2
0.947
0.833
0.113
Tweet Eval
2
0.840
0.739
0.101
Text
GLUE-CoLA
2
0.864
0.763
0.100
ECIHR Articles
13
0.476
0.438
0.038
22

---

Published as a conference paper at ICLR 2025
Table 14: Training accuracy and test accuracy of target models trained with different architec- tures(corresponds to Table S in Section (.2). CIFAR-10 and Purchase datasets were used to train image models and tabular models from scratch, respectively. The Rotten Tomatoes dataset was used to fine-tune the pre- pre-trained text models.
Datasel
Target model
Train accuracy
Test accuracy
Train-test accuracy gap
WRN28-2
0.981
0.877
0.104
ResNet18
0.992
0.880
0.112
CIFAR-10
VGGH
1.000
0.853
0.147
MobileNet-v2
0.934
0.845
0.089
Multilayer Perceptron
1.000
0.897
0,103
Purchase
CatBoost
1.000
0.725
0,276
Logistic Regression
0.999
0.755
0.244
DistilBERT 0.947
0,833
0.113
Rotten
RoBERTa
0.964
0.881
0.083
Flan-T5
0.911
0.886
0.025
Table 15: Training details for various model architectures, including learning rate, weight decay, maximum training epochs, and more. MLP stands for Multilayer Perceptron, and LR stands for Lo- gistic Regression. 'N/' indicates that the model does not require consideration the corresponding hyperparameter.
Modality
Target model
Learning rate
Weight decay
Maximum epochs
Optimizer
Learning rate schedale
Batch size
WRN28-2
0.1
0.0005
200
SGD
Cosine Annealing
256
Image
ResNeti8
VGG11
0.0005
200
SGD
Cosine Annealing
Anneeling
256
MobiloNe--2
0.0005
200
SGD
Cosine Annealing
MLP
0.001
0,0001
200
Adam
N/A
256
Tibular
CaiBoost
0,85
N/A
10,00
N/A
N/A
N/A
N/A
N/A
Text
RoBERTh
0.00002
0.01
Adam'W
N/A
Flan-T5
0,00002
0.01
10
Adam'
NAA
32
A.8
COMPLETE VERSION OF THE EXPERIMENTAL. RESULTS
The comprehensive evaluation of TDD performance across different datasets, various target model architectures, and diverse shadow and reference models is presented in Table T6 Table 17 and Table respectively.
18
A.9
PERFORMANCE UNDER DIFFERENT METRICS
We present the TDD performance evaluated across multiple metrics for three distinct data modalities,
as detailed in Table 19 Table 20 and Table
23

---

Published as a conference paper at ICLR 2025
Table 16: Complete version of TDD performance across different datasets (corresponds to Table 4 WRN28-2, Multilayer Perceptron, and DistilBERT are trained on image, table, and text datasets, respectively.
Modsite
Irage
Tabsla
CHARR-10
BosdNNNTT
CalebA
Pacdous
Tinot
Addt
Sandert
ColAA
DOHER
Katric 1000
0,858
8.527
6,58
0.415
0,009
0.903
0.546
0.883
6,566
0,571
0.501
0.999
Vetric-pont
<+ (3K3)
(=14864)
0,58
(4 (+-80)))
8557
0,59
15 (S0005)
0.35
C5 (1.001))
0.62)
In 1-0033:
0L995
0.566
0.562
[: (-88666
0.565
(4 (5-1003)
0,571
(4 (4-007)
0.521
[s (555)
(L995
(8:00R9))
(4.0030)
(20000)
(2:0.80)
Ketric-eat
0/425
0L548
0522
0
0.66
0.350
0561
0.324
0.339
0.477
Kairic mant
(5 (183))
RA35
6358
(.884)
(1 (J02)
8.577
0,99
C2 (L86))
4820
CE BJIL)
Is M003:
0L905
0.566
0.92
0.56
C2 (LRS))
0,571
CE (JRR))
0.522
L: 4.894
0
Sesrn-original
(1 (/.0
0
(803)
C1 6mm
0.96
L. ROOT:
0,933
0,55
C1 CHIDD
0.597
B035:
21 (100M)
0.58
0,92
C1 L000
0.565
C1 (.0
0,506
0L595
0.03%
0.993
0-53
0.901
BEST
0.555
0,90
0.53
0.31
Saarn-aorted
0.755
0,950
(1004)
(1 8/14)
0256
0.57
SHIT
C.0000
6.30
0.61
0
C1 0000
GMIL
Cezrn-label
<t: (510))))
0.005)
C2 1-.0010
(5-0000)
8/066
BJ883
0-511
6.977
ILER4
4.00
8:964
0.954
4,59
BA12
Kodel-lass
(+
0.833
8,500
6,522
8.707
8,21
0.79
6:754
6.83
Kodel-cslibratton
<5 (152))
0420
0261
C1: (114)
0355
t: M404
0
C5 COILD
I: B006
EL3SS
CE DUIEE
CMMS
IS BCOS
C2 (3.0.14
CAS
Is BOB3
(J855)
(.82))
10002
(UITA)
Kodel-far
0,677
(=0000)
0,532
0552
0
0.225
2015)
(=0
067
0721
CAS
Kode1 ZOCMgE
(4 (UER))
9,625
(4044)
0.553
[= WOOS
0,553
C2 (U
0,311
CL CHITT)
E UUBS
(2 0
0.66
0.355
C2 (30014)
0.466
IS ALTE2)
Cosry-augracs
<1 (1PS))
0,515
0517
ROON
RS12
0.00
CI BIITI
BA12
8,595
0,50
BC36:
CI L.005)
0.581
21 (000
0,16
BC30
Coirg--einnter
0.582
0822
50000
0
0.3G
0
0-501
0.77
Quezy-ade
0,655
GITSS
0,338
(101))
(1 6m0
0.65
0,514
0.26
(LUB0)
0,579
6,595
0.563
21 BARD
0,55:
0.565
0,10
0,575
Cwerg-neighhoo
<8 (U00)
0.0000
0/04)
(200000)
C5 (.07))
Is (.995)
(200
(2-0.00)
(5: (-14)
IS (632
Comzy-crm
0.112
0.321
0.905
0.33
8,999
0.75
C-SM
6.991
0.12
Quezy-zad
<E 0.100)
8.735
(= (.966)
0.341
CE 8/22)
8,566
[- (.877
0:336
NIA
NVA
NIA
NIA
XIA
SiA
NIX
NIA
NIA
NIA
XIA
NIA
NIA
N/A
N0A
NIA
[: (L776)
0,92
Table 17: Complete version of TDD performance across different target model architectures (corre- sponds to Table MLP stands for Multilayer Perceptron, and LR stands for Logistic Regression.
CIPARIQ(msg)) RoNNel5
Purdhows
Tange
WRN28-2
VOGIT
MololNNo-22
MLP
CaeBuoi
I.R
DiailBRT
RoBERT.
Flas-T5
Metric-lose
(5 (.853)
06635
(1 (.42)42)
0,659
0,684
(.004)
( 8.857)
0.593
(5 0.003)
0,699
(+:0,00)
[U98
[5 0.033)
0.640
(= (.8007)
0.582
(: (::0,119)
0,571
(5 (.8.44)
0,517
(5 (.11.))5)
0.645
Motele-conE
(5 (.88
0A33
(4 0.82))
8,659
(4 (B00))
0,644
<S 8.87))
0.593
0.619
(5 (200))
0198
16 (200)
0.640
<s 6.977)
0.582
(5 (5:0,19)
0,571
(5 (1:0.10))
0.517
(5 (.115
8.645
Metrie-core
(2000009)
0.552
(1 (.000)
0.557
(1 (.00))
0.574
(1 0.019)
0.54
(1 0,00)
0.351
(1 0.000
0.636
[4 0.02)
0.622
(1 (E00000)
0.557
(2 0.014)
0.542
(2: 0.000
0.513
(1 0,006)
0.565
Hetric-eat
0.688
0.055))
(6 0.03)
0.654
0.60
(.044)
0.58)
0.000
0616
0.00))
(.01))
0.943
0.964
0.00)
(=0707)
0.361
0.599
0.003
6.118)
Metric-ment
0.635
0.85)
(/82))
0,685
0.592
(.56))
0,200
0,003
0.009)
0.643
0,003
0.582
(.07)
0.019)
6.908)
0.116)
Jearn-origtral
(5 0.064)
B631
(+ 0.63))
0,625
0,084
(L004)
(.07)
0,522
0.002
(1925
0,09%)
[+ 0.005)
0.644
(= 0.009)
0.558
(5 0.015)
(5 0.007)
0315
(5 6.12
B635
Learn-top3
(s 0.087)
0.628
(4: (LB47)
0.653
(4 (UBOS)
0
0LER3
(.099)
(5 00033)
0.677
E5 0,099)
0.677
LS 0000)
0.660
(5 0.007)
0.561
E6 (005)
0.555
(5 0.00))
0,509
E6 0,120
D647
Learn-zortad
(1 0.857)
0.628
(1 (Aa)))
0.654
(1 (L004)
C.580
CE 0.577
0.578
(2 0.055
0.666
C1 0.019)
0963
[1 0.045
0.66]
(2 (.007)
0.561
LE 0.055)
0.555
(1 0.003
0,509
CE C
B6646
Learn-label
0.635
0.055)
(+ (t-01)))
0.612
0:077
(.00))
0.550
0.56
0.005)
0.944
0.044)
0:001
00055
0.384
8.099)
0.000)
965
0.330
6.007)
0.699
0.121)
Learn-marge
0.656
6.M5)
(4 0017)
0.828
0,777
0.22
0,844
9,005
0.044)
6.716
0.0%)
0.584
0.099)
0/021)
0(05)
0.128)
Model-loss
(5 0.050
B6664
(+ 0.028)
0,709
(4 (L005)
0,729
0045)
(5 0.002)
0,725
(1915
0.015)
[+ 0.003
0.776
0,773
8.020)
6.634)
(5 8.817)
0A42
(+ 6.105)
0.221
Hodel-calibsrrion
D.639
0.671
0,895
0.565
0.644
0965
0.11
0.605
0,622
0.592
0677
(5 0.040)
(4 (.155)
(4 (000))
Ca 0106))
(5 0000
E5 03009)
L6 0000)
(.022)
E6 0005)
C5 0
E5 0,075)
Model-lirs
(2 (.88)))
(1 (L66))
6:449
(1 (UISS)
0.789
CE 0.53)
BASI
(2 0.033)
0,258
CE 0,003)
1L995
[1 0.005
03961
(: 3.024)
0753
LE 0.025)
0,60
(1 0.00)
0
E1 (.120
B728
Hodel-fpe
0.647
0.865))
(+ 0.03)
0.684
0712
(L00)))
0,619
0.99
0.04))
0.976
0.013)
0.724
000))
0,679
E 0Vl09)
0.623
0.599
0.000)
0.10%
Hodel-robuat
0.635
0,050
(4 0011)
0.677
0.704
(000)
0041
0711
0000
ILMES
0.013
0.7%
0,00)
0.766
QU0Z)
0005)
0,115)
Query-sugrent
B3133
0.025)
(+ 0.00)
0,515
0,633
0.542
0.34))
0612
0000
0U096
0,002)
0.664
0.044)
0.570
0.007)
0,000
E5 0.009)
0512
E+ 0.657)
0,992
Queey-tranaEer
(s 0.008
0.522
(4 (UB0))
0,532
(4 (Uma)
0,533
0.567
0003)
C5 0000))
0.529
ES 0,000)
0.557
LS (000)
0.574
0.550
0.011)
E6 (001)
0.155
E5 0.005)
0.506
EA 0,627)
0,513
Qsery-adv
(A 0.08)
0.155
(4 (1026)
0.71
(# (031
0.66
CE 0.042)
0.5K1
( 8,003)
CE 0.002)
0/737
0062
0.005
(= 0.007)
0.571
LE 8.020)
(1 (.007)
1.516
EL 0.003
0613
Qoery-reeighoor
(+0000))
0.311
(+ 0,044)
0.12
0,509
(.004)
0,505
(.00))
0.333
0.00)
0.200
0.015)
0,530
0/00)
(- 0.004)
0.533
E 01004)
0:527
0.504
(:0005
0,49
0.092)
Query-em
(5
0.532
(6
0.541
0.530
0.523
0.057)
0.946
0.007)
0U6T2
0.004)
0.524
0.08)
00005)
0.5
(5: 0.007
0. 506
0L10)
Qoery-ree
0.735
6.108)
8,800
0.085)
0,543
(U013)
0.163)
N/A
N/A
NIA
24

---

Published as a conference paper at ICLR 2025
Table 18: Complete version of TDD performance across various shadow and reference models (cor- responds to T). MLP stands for Multilayer Perceptron, and LR stands for Logistic Regression.
Stadinhhfeeeees maddd
Target model
WRNN8-2
ResNart
WENNE-JCCBRR-I)
VOG11
VohileNase?
MFP
MIP1iPrchor) CarRioat
1.
DisiiRPRT
RoRFRTTs
Flae-T5
Ang.
Learn Searn-origina:
( +.0.64)
01.31
(+ 0.000
0,57
0.522
(181))
0,33
0.052
0.0001))
(+-0005)
(1001
[+ (000))
0.564
(=0.099)
0.558
(+-0197)
0.500
0.566
0.991
(.001)
Zearn-top3
0.629
0.628
0.057)
0.628
0.633
0.677
0.003)
0.646
0.00))
te
0.515
(2:0007)
0.361
(2 (2-0.07)
0.561
(2 (200707)
0,633
Learn-aorted
D.638
0.057)
(U8S))
0,628
0.050)
0.025)
0,019)
0,532
0.045
(2:0.00))
(300007)
(.458)
Tearn-latel
(5: 0.086)
01655
(6: 0.024)
0.591
0.644
(L060)
C 0.014)
0.363
(5 0.005)
0,566
(5 0.005)
11051
[1 0.20%)
0.551
(= 0.009)
0.584
(1 (3:00099)
0.554
(5: 6.0077
0.550
(1 (.045)
0,000
Learn-nerge
< 6.865)
IU656
(+ 0.022)
0,581
(+ (UK6))
0.651
<a 0.077)
0,509
( 6,00%
0,684
(+ 0.019)
0.517
( 0.025)
0.995
( (.09)
0.564
( 0.009)
0.584
( 0.008)
0.580
(+ 0.000)
0.594
Hodel-1oso
0.050)
(+ (.05))
0.567
(.09))
0.000
00008
(000))
0.60
0.733
8.020)
0
0108)
0.005)
Medei-caaibattion
0.639
0.040)
(1 0,033
0.634
0.817
0,614
0.019)
0.684
0.002)
0,579
0.000
0.595
0.595
0.007)
0.557
0.007)
0.043
Model-lira
0.085)
(4-0.64)
0.666
(UIST)
0.055)
0.003)
0.003)
0,58
0.0C))
0.005)
Hodel-far
U.667
0.668
(J61))
0,638
DDSS)
DAGI
0.056)
0.99
0.000
C6 0.001)
0.645
(5-00477)
0.677
0.008
0641
0,085)
Hodel-robust
(6 0.030)
D.635
(1 0.036)
0.539
(4 (L034))
0,633
C 0.022)
0,621
CE 0,002)
0711
C5 0,003)
0.622
EE 0.202)
0.625
0.766
9.022)
C1 0.010)
0.624
(2 0,006)
0.591
(2 C.03)
0.688
Quory-augrent
0.573
0.055)
(4 (.19))
0.555
0.575
0,553
0,122
6
0,612
0,612
0000
0,570
(LINT)
0.569
0.008)
0.565
6.011)
0,550
0.006)
Query
0.522
(100.18)
0,529
0.518
(E 0.17)
0,518
0.529
0.004)
0.535
0.003
0.529
0,001)
0.550
0.529
0U010)
0.006)
0,012)
D.532
0,533
Qvery-ggm
(#:0,072)
(4:0.072)
0.074)
0.057)
0.003)
0.063)
0.033)
0.061)
Quary-zet
(110))
01755
6,740
0.100)
0.722
(LIR0)
0,708
(.074)
N/A
N(A
NIA
NIA
NAA
SIA
0.726
0.888)
Table 19: TDD performance across different metrics on WRN28-2 trained on CIFAR-IO dataset. MA(membership ad advantage) (Jayaraman et al. 202T) equals the difference between the true positive rate and the false positive rate. For all metrics except for FPR and FNR, higher values indicate better performance of the corresponding TDD algorithm.
Algoralam
Procisios
Recail
Fl-sone
FPR
MA
TPRO
TPR JOSSPR
AUROC
Metzie-losa
0,579
(1030)
0,905
0.000
(+ (.133)
0.706
0,623
0.045)
(+:0080)
0,095
0.659
(=0001)
0.246
0,009
0.005)
0,132
0.635
0,053)
0,579
0.05
0.706
0.005
0.659
0.246
0,009
IL131
0.635
(0030)
[1 0.039)
(2333)
(.045)
(.09)
Q.15))
0.091)
0.005)
0,053)
Nelsie-core
0,508
0.941)
(+00011)
0.687
0.009)
(+004)))
0,018
0.25))
(+0.018)
0.105
0.000
0.000)
0.553
0.009)
0,574
0.876
0.122
0,229
0.012
0U628
(10077)
(1000116
0.050)
0073)
(100001)
0,001)
0,058)
Moteto-mont
(+ 0.44)
(+00.01)
0,846)
0,000
0.04))
0.500
0.091)
0,010
0.005)
0,053)
original
0,722
(+00665)
0.685
0.051)
(+0))12))
0,230
0,015
0.005)
0,631
0.064)
Tearn-top3
0.576
0.033)
(1007))
0,777
0.016)
0,061)
100000
0.125
6033)
(L000
0,232
0.012
0,0000
0013)
0,628
0,057)
Leazc-sortod
0,575
0.002)
(0700)
0.691
0,613
0,000
(000000
0.125
6.32)
0,230
0,010
(00000
0,32
0014)
0,628
0,057)
Leasn-label
0,573
(.027)
(1 (.06))
0.325
0.708
0.038)
0.619
0,045)
0,055
0.010
Q.13))
(.000)
0,277
0,015
0,005)
0.556
0031)
0.633
0,0%6)
L0azc-M0r00
(U830)
[4 0.009)
0.908
6.708
0.134)
(.846)
0.092
0.039)
(4 Q.154)
0658
0.250
0,093)
0,020
0,008)
[E (J04T)
6,6%
0,065)
Modsl-losa
0
0.065)
(1 (100
0811
0,022)
0,002)
0,189
Q.86)
(L.0)
0,050
0,015)
0.27))
0,661
0,050)
Medsl-calibeation
0.566
(1029)
11 (1000)
11835
0.674
0.019)
0,597
0.027)
0.365
0.0000
Q.89))
(.00))))
0.193
(,0111
UL73
0013))
0.639
0,040)
Model-1ica
0
(L009))
[+ 0.06))
0.10
0,682
0.026)
0.623
(.84))
0,190
0.300
(+ (L10))
0367
0,343
0.097)
0,120
(,669)
(+ 0.107)
0,690
0,085)
Model fpc
0.624
(1042)
14 0.044)
11520
(# 0.8K0)
0.566
0.605
0.00%)
0074)
(1 6UE7)
0.100
0210
00074
0,034)
t4 0.064)
1.261
0.647
0.056)
Rodel-zobuat
0.553
(.015)
(+ 0.135)
I.882
(+ 0.09%)
0,676
(= 0.008)
0.583
0.135)
(+ 0L133)
0716
(= 0.066)
0.167
0.070
0:023)
++ 0.03)
0,635
0.030)
duesy-augnen:
0.539
(.03))
(+ (.00))))
1917
0.679
0.019)
0,002)
[+ 0.09)
0.083
(+ (.EE7)
0.783
0.135
0.45))
0:004
0.003)
++ 0.045
0.573
0.025)
Query-t
0.519
(1005)
[5 0.033)
1U943
0.670
0.112)
(.00%)
1J857
0J033)
( (L119)
0373
0.07))
6:006
0.003)
t 0.00))
0.522
0.008)
Quesy adv
0.583
(.095)
(+ (00077))
0915
0.712
0.031)
0.871
0/47)
0.084
0.27)
0.644
(.074)
0.261
0.044)
0.002
0.004)
++ 0.000)
0.044
0,615
0.018)
Query-neighhbo
0.514
0.005)
[1 (115)
D.477
(1 0.079)
0.452
0,510
0,001)
[L (1155
0.58%
(L 6.116)
0.396
CL 0.000)
0.021
0:000
0.000:
EL (0000))
0.074
0.511
0,003)
Query-qem
(= 8.037)
0.518
[t 0.30))
0.583
0.523
0.150)
0,529
0.068)
[t (300))
0.17
(t (:0255))
0.525
0.057
0.116)
0,000
0.000)
Ct (::0000)
0,000
0,532
0.072)
Query-zot
(_ (0075)
0.649
[1 0.140)
0.376
(LL 0.050)
0.696
(_ 0.062)
0.666
[1 0.140)
0.224
(1 0.150)
0.445
(LL 0.123)
0.351
(1 0,064)
0.152
(L 0.152)
0.355
(L 0,108)
0.235
25

---

Published as a conference paper at ICLR 2025
Table 20: TDD performance across different metrics on MLP trained on Purchase dataset.
MA(memberhhip advantage) (Jayaraman et al. 2021 equals the difference between the true pos- itive rate and the false positive rate. For all metrics except for FPR and FNR, higher values indicate better performance of the corresponding TDD algorithm.
Algoriluu
Prozisios
Recall
Fl-some
FPR
TPROISEPS
AUROC
Matzio-losa
0.006)
0.727
0.003)
0.03))
(+:006))
(.085))
(=00007)
(.800)
(.001)
0,619
0.003)
Netnle
0.002)
(t 0.000)
(+00.00
0.727
0/033)
(0:0006)
(t 0.085)
(+:0.00)
0253
0.800)
( 0.001)
0.619
0.003)
Met:lo-oore
(.002)
1,000
0.000)
(+00001)
0.690
0.01))
0,000
0.000)
0.007)
(+00000)
0.101
0,000
0.000)
0.000)
0.55)
0.001)
Net-lo-ent
0,582
0.002)
(1 0.008)
0548
(1.0.00)
0.721
0.834
0.000)
0.022
0.08)
0.680
0.006)
(100007))
0,263
0,010
0.800)
E1 0,001)
0,808
0.16
0.003)
Metsio-ment
0,587
(U002)
(1 0.005)
0.962
(1 0.003)
0.729
[= 0.003)
L1 0.005)
0.088
(1 0.008)
0.676
(4 0,007)
0.256
0.010
0,800)
L1 0.001)
0LEOR
( 0,003)
0,620
Lesrn-originel
0,582
0.003)
(+ (10000))
0.965
0.724
0.002)
0.002)
0.044
0010)
0,270
0.00%)
0.016
0.007)
0.005)
0,653
0.002)
Tearn-top3
0,585
0.002)
[1 0.006)
0.96
(L 0.003)
0.726
0.839
0.003)
0.041
0006)
(1 (.005)
0,678
CL 0.0007
0.279
0,018
0,0013
CL 0010)
0.125
0
0,003)
0.586
0.89
0.225
0,640
0.001
0,620
0,250
0:018
0.167
0,666
Leaze-sozted
(= (L005)
14: 0.007)
( 0J003)
t= (0004)
( (U00))
C 0,000)
0004:
E 0054)
CE 0,028)
Leaze-label
0,385
(.002)
L1 0.003)
0.965
0.728
0,092)
L_ 0,033)
0.640
0,035
0.033)
(1 0.006)
0,685
(L 0.000)
0,260
C2 0,000
0:016
11 0004)
B155
0,656
0,005)
0.539
0.728
0.644
B.07
BA6S
0,288
ILI87
0.684
(= (U002)
[# 0.005)
(# 0.002)
[= 0.003)
tE 0.005)
(: (L006)
( 0.006)
C= 0,0011
+: (U0NE)
C 0.003)
Hodsl-loss
(= (L006)
0.511
[+: 0.028)
0.843
(+ 0.006)
0.708
t= 11001)
0,853
1e 0025)
0.377
( Q.130)
0337
(2 0.003)
0,206
0,086
1000:
+ 0.003)
0.275
0.225
0,002)
Model salibration
(= (L011)
0,350
11 0.063)
11842
0,693
0.016)
t= 0,00))
0.028
11 0003)
0.35%
( 0.03))
0.586
(1 0.000)
0.256
(E 0,002
11 0.002)
B.206
0L684
0,002)
Modal-1ira
(= (.005)
0.514
[t (J022)
0.913
(+ 0.006)
0.734
(= (,002)
0,670
1J887
(+ QUE1)
0.573
(= (=00004)
0,340
(= (.00P)
0.134
++ (UNIS)
IL378
( 0J003)
0.755
Model fpc
(- (U009)
0.636
{+ (005)
0.688
(+00.01))
0.646
(- 0.02))
0.640
[+ 0038
0.322
( (.02)
0.377
(+ (.000
0,251
0.073
00003
++ 00000
11296
CU6AT
0,004)
Model-robust
(= 0.002)
0.500
(+ 0.844)
0,89
(+ (.00))
0.697
(= 0.002)
0.678
[+ 0.04)
0.161
(+ (.074)
0 164
(+ 0.005)
0,275
0:004
0.003)
++ 0.033)
D.289
CE 0,000)
0.711
0.563
8.883
0.711
0,699
0.087
0.745
0,218
0,000
B,000
CUB12
(= (.002)
(+ (.00))
(+ 0.092)
(,001)
0.00)
OLD03)
(= 0,003)
0000)
++ 0.0000
C 0,001)
suery-transfe
(= 0.033)
0.523
[t 0.008)
IL981
C1 (100.004
0.682
(= 0.005)
0.544
(1 0.008)
IJ019
(1 (.00))
03933
(1 0.009)
OLISS
0,010
0,000)
CL 0.00))
BL100
0U529
0.004)
Query-adv
(- (UMIS)
0.569
{+ (-0855)
0.79
(+00.08)
0,600
(- 0.003)
0,607
(+ (00838)
0.121
(+ 0.058)
0.65
(+ 0.006)
0.214
0.000
0.000)
++ 0.0000
D,000
( 8,003)
0,620
Que-y-neighbor
0,524
0.005)
[1 0.944)
0.527
(.006)))
0.523
(_ 0.001)
0,524
(1 (.04)
0.475
(1L. (.09C)
0.180
CL 0,002)
0L04S
0,000
0.000:
EL 0.003)
0,115
CL 0,001)
0,533
Query-qem
t= (L037)
0,516
( 0.421)
0.113
0.262
0,313)
(= 0.064)
0,529
0.687
0.421)
(1 0.313)
0.255
(2 0.128)
0.058
0.000
0,000)
( 0.000)
0,000
(4 0,057)
0,523
26

---

Published as a conference paper at ICLR 2025
Table 21: TDD performance across different metrics on DistilBERT trained on tomates dataset. MAmmembership advantage) (Jayarmann et al. (021) equals the difference between the true positive rate and the false positive rate. For all metrics except for FPR and FNR, higher values indicate better performance of the corresponding TDD algorithm.
Algoriluu
Prozisios
Recall
Fl-some
PNR
HPR.
TPROISEER
AUROC
Ratzio-losa
(+0))))
0.680
0.013)
(= 0.009)
(+:031))
0.172
(+0017)
(=0001)
0.156
0.011
0.813)
+=00077)
Netile
0.010)
(t 0.30)))
(+.0.113)
0.650
(= 0.009)
(+:003))
0.372
(+:0017)
(+00019)
0.156
0,011
0.803)
(t (::0010)
0.007)
Met:lo-oore
(.00%)
0.008)
0.678
0.00)
0.000)
0.833
0.008)
0.00)
(+0001)
0.113
0.000
0.000)
(+ 0.000
0,557
0.006)
Net-lo-ent
0,536
0.009)
[1 0.00))
0.765
0.0000
0.629
[= 0.007)
0,585
(000))
0:234
(1: 0.050)
(1.0))3))
0.110
0,011
0,803)
C1 0.100)
0.119
0,561
0,007)
Metsio-ment
0,549
(.010)
(1 0.033)
0.828
(4 0.013)
0.680
[= 0,009)
L1 0.051)
0.172
(1 0.017)
(1 0.019)
6:156
0.011
0,803)
L1 0.010)
( 0,007)
0,582
Lesre-originel
0,513
0.000)
(+ (0005)))
0.780
0.017)
0,633
0.009)
0:220
0.09)
(.00))
0.105
0.017)
0,012
0.03)
++ 0.07)
0,558
0.009)
Tearn-top3
0,536
0.009)
[1 (.06))
0.766
0.629
0.000)
0,585
0,007)
0.234
0061)
6,55)
CL 0.003)
0.110
(1 0,003
0,011
CL 0.09)
0,119
0,561
0,007)
0.536
0.765
0.629
0,555
0.334
0.110
0:011
0111
0,561
Leaze-sozted
(= (1009)
14: 0.066)
(+ 0.100)
t= (L007)
1E 0000011
( (U50)
CE 0.033)
( 00003
E 0010000
0,007)
Leaze-laber
0,546
(.009)
L1 0.023)
0.866
0.679
0.012)
0,578
0,009)
0.124
00025)
(1 0.19))
0711
CL 0.008)
0.155
(1 00003
0:011
E: (000))
1122
0,564
0,009)
0.547
01622
0.66P
0.578
0.138
07115
0.177
0.022
HL222
0.584
(= (U010)
[# 0.17)
(# 0.DE1)
t= (.009)
te 0.17)
(# (.18)
(= 0.019)
(= 0,002
t (.009)
( 0.009)
Hodal-loss
(= (0011)
0,883
1+: (.06)))))
0.207
( (OEE))))
0,691
t= (.015)
0,091
1t 00000
0.29
( QUEE)
0324
(= (.830)
0,383
(= U0201
0,148
tE 0.34)
0.385
( 0.020)
0.773
Model salibnation
(= (L008)
0,306
11 0.065)
0377
(1 0.013)
0,680
t= 0,000)
0.039
11 0005)
0.23
( 0.411
0,500
(1 0,003)
0,277
( 00011
0.006
EE 0019)
112344
CL 0,012)
CUORS
Modal-1ira
(= (.007)
0.631
[t (J043)
0813
(+00)))))
0,110
(= (.07))
0.67
[t (0045)
I.187
+ (.13))
0471
(# 0.05))
0.342
(= 10026
0113
+ 0045)
1:774
C 0.034)
0.753
Model fpc
(= 0.06))
0.671
{+ (.00))))
0.506
(+00065)
0.573
(= 0.00)
0,630
(004)))
0.4%
( 0.048)
0.245
(+ 0.049)
0,260
( 0004)
0.141
++ 00000
IL340
CU679
0.041)
Model-robust
(= 0.0a))
0.851
(+ 0.833)
0.797
(+ 0,008)
0.715
(= 0.021)
0.684
[+ 0.63))
0.303
(+ 0L101)
0.29
(# 0.042)
3.368
(= 0.028)
0.162
++ 0.027)
0.975
CE 0.022)
06766
0,537
0876
0.666
0.565
0.122
0.747
0.131
0.000
8,000
O,570
(= (U008)
(+ 0.09919
(+ 0.003)
0.006)
[+ 0-0199)
++ 0.036)
(.00202)
0.000
++ 0,00000
C 0,007)
suery-transfe
(= 0.01 I)
0.525
[1 8.009
RL933
C1 0.001)
0,672
(= 0.007)
0.549
(1 0.009
IJ867
(1 0.007)
0335
(1 0.014)
0.098
0.008
0.005)
CL 0.000)
B,000
0,530
0.011)
Query-ady
(- 0.008)
0.537
(+ (+03)))
0.811
(+ (+00000)
0.667
0,566
0.005)
[+ (+0033)
0.119
(+ 0.040)
0.70
( 0,011)
0.131
0.000
0.000)
++ 0.000
0,000
+ 0.007)
6,871
Query-neighbor
0,525
0.008)
[1 0.021)
0,909
(1 0.010)
0.653
0,548
0.00%)
(1 (000))
0
(L 0.018)
0.313
CL 0.007)
0L096
CL 0,603)
0.011
EL 0.005)
BJ991
0,333
0,004)
Query-qem
[= (1023)
0,509
( 0.079)
0.722
(4 0.042)
0.597
(= 0.035)
0,525
LE 0.079)
0.278
(1 0.135)
0,672
(2 0.070)
0.049
0.000
0,000)
(1. 0.000)
0,000
0.524
0,038)
27


Reviews:
[{"review_id": "KJpTC21Cls", "reviewer": "Reviewer_KoC1", "summary": "This work provides a comprehensive evaluation of training data detection (TDD). The evaluation spans multiple data modalities, including image, tabular, and text, and implements 21 state-of-the-art TDD algorithms, including large language models such as Pythia-12B. This is the largest benchmark for TDD to date, offering a valuable reference for future research on this problem.\n\nAlthough this evaluation extensively covers multiple data modalities and four main types of TDD methods, the insights from the evaluation are limited. It lacks in-depth analysis of the results. For example, in Table 4, the authors do not explain why performance on CIFAR-100 is noticeably higher than on other datasets, and similarly in Table 5, there is no explanation for why the target model CatBoost outperforms others on tabular data. Some explanations of the results are also unclear. For instance, in Figure 2, the results indicate that the train-test gap is higher, and the target model performs better at detecting the data used in training. The authors attribute this to overfitting of the training data, which supposedly results in high performance when the train-test gap is large. However, wouldn’t overfitting typically degrade performance since the test set is independent? This explanation should be clarified. Additionally, in Fig. 3(a), the authors state that residual connections in ResNet help alleviate excessive memorization issues, thus decreasing the detection method’s performance. This explanation is also somewhat confusing. In Table 6, the authors claim that detection performance declines when the reference model has limited knowledge of the target model’s architecture. However, this is not always true; for the learn-top3 method, all types of reference models show similar performance without any decline. Thus, the explanation appears inconsistent.\n\nThe paper also lacks necessary explanations of the TDD protocols, such as the training-test protocol when using a reference model, or how the query model operates, which would make it easier for readers to follow.\n\nThe evaluation also omits Transformer-based models, which are widely used today. For image-based applications in particular, Transformer-based methods are among the most commonly used models, making it worthwhile to evaluate these models for TDD. For large language models, instead of focusing on the Pythia models, evaluating LLaMa, as the most widely used public LLM, would be highly relevant for TDD and aligns better with real-world scenarios where TDD is a concern.\n\nIn summary, this work presents an extensive evaluation across different datasets and multiple models, providing a complementary benchmark for future TDD research. However, the lack of in-depth analysis and necessary explanations of training-test protocols across TDD methods weakens the work’s overall quality. Additionally, the omission of evaluations on Transformer-based models and widely used public LLMs, such as the LLaMa models, limits the value of this work.", "questions": "- in Table 4, the authors do not explain why performance on CIFAR-100 is noticeably higher than on other datasets\n- in Table 5, there is no explanation for why the target model CatBoost outperforms others on tabular data\n- in Figure 2, the results indicate that the train-test gap is higher, and the target model performs better at detecting the data used in training. The authors attribute this to overfitting of the training data, which supposedly results in high performance when the train-test gap is large. However, wouldn’t overfitting typically degrade performance since the test set is independent? \n- in Fig. 3(a), the authors state that residual connections in ResNet help alleviate excessive memorization issues, thus decreasing the detection method’s performance. This explanation is confusing.\n- In Table 6, the authors claim that detection performance declines when the reference model has limited knowledge of the target model’s architecture. However, for the learn-top3 method, all types of reference models show similar performance without any decline. The explanation appears inconsistent.\n- The paper also lacks necessary explanations of the TDD protocols, such as the training-test protocol when using a reference model, or how the query model operates.\n- Why the evaluation omits Transformer-based models, especially for image datasets\n- Why chose Pythia instead of LLaMa as the most widely used public LLM", "limitations": "", "rating": 6, "confidence": 4, "soundness": 2, "presentation": 3, "contribution": 2, "strengths": "This work provides a comprehensive evaluation of training data detection (TDD). The evaluation spans multiple data modalities, including image, tabular, and text, and implements 21 state-of-the-art TDD algorithms, including large language models such as Pythia-12B. This is the largest benchmark for TDD to date, offering a valuable reference for future research on this problem.", "weaknesses": "Although this evaluation extensively covers multiple data modalities and four main types of TDD methods, the insights from the evaluation are limited. It lacks in-depth analysis of the results. For example, in Table 4, the authors do not explain why performance on CIFAR-100 is noticeably higher than on other datasets, and similarly in Table 5, there is no explanation for why the target model CatBoost outperforms others on tabular data. Some explanations of the results are also unclear. For instance, in Figure 2, the results indicate that the train-test gap is higher, and the target model performs better at detecting the data used in training. The authors attribute this to overfitting of the training data, which supposedly results in high performance when the train-test gap is large. However, wouldn’t overfitting typically degrade performance since the test set is independent? This explanation should be clarified. Additionally, in Fig. 3(a), the authors state that residual connections in ResNet help alleviate excessive memorization issues, thus decreasing the detection method’s performance. This explanation is also somewhat confusing. In Table 6, the authors claim that detection performance declines when the reference model has limited knowledge of the target model’s architecture. However, this is not always true; for the learn-top3 method, all types of reference models show similar performance without any decline. Thus, the explanation appears inconsistent.\n\nThe paper also lacks necessary explanations of the TDD protocols, such as the training-test protocol when using a reference model, or how the query model operates, which would make it easier for readers to follow.\n\nThe evaluation also omits Transformer-based models, which are widely used today. For image-based applications in particular, Transformer-based methods are among the most commonly used models, making it worthwhile to evaluate these models for TDD. For large language models, instead of focusing on the Pythia models, evaluating LLaMa, as the most widely used public LLM, would be highly relevant for TDD and aligns better with real-world scenarios where TDD is a concern.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "RBt0N3KymJ", "reviewer": "Reviewer_64P6", "summary": "Motivated by the limitations of the previous benchmarks on training data detection (TDD), the paper proposes a new benchmark that covers three different modalities (tabular, text, and image) and includes evaluation of latest methods. The paper gives several insights on the recent TDD methods with respect to TDD.", "questions": "Please refer to the above weaknesses", "limitations": "", "rating": 5, "confidence": 3, "soundness": 3, "presentation": 3, "contribution": 2, "strengths": "1. The paper analyzes the latest methods' limitations and strengths carefully. Particularly, the correlation between train-test performance gap and TDD performance is interesting.", "weaknesses": "1. The proposed benchmark is not truly large-scale as it considers regular-sized datasets.\n\n2. The proposed benchmark is not significantly different from the existing ones except that it covers different modalities. It would be more interesting if the paper does not follow data split rules from the existing benchmarks but explore something completely new. Another missing aspect is the size of data points that are evaluated under TDD.\n\n3. The claim that TDD outperformance comes at a cost of computational expense is a bit controversial particularly due to Fig. 2(a); learning-based can achieve the best performance although it requires significantly less computation than the model-based ones.\n\nOverall, it is valuable that the pepper gives an extensive amount of experimental results, but the paper is a bit lacking in novel aspects.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "yWpBHItiGK", "reviewer": "Reviewer_3bnU", "summary": "The paper introduces TDDBench, a comprehensive benchmark designed for Training Data Detection (TDD), which is for determining if specific data was used to train a model. This can be used to address issues like data privacy, copyright verification, and model unlearning. TDDBench includes 13 datasets covering image, tabular, and text data and benchmarks 21 TDD methods across four detection paradigms. The findings in this paper highlight that model-based methods outperform others but require significantly more resources. The benchmark reveals that existing TDD methods struggle, particularly with text and tabular data, and emphasizes the need for further research to enhance TDD effectiveness.\n\n-----\n\nHave gone through the responses, and I appreciate the authors' efforts. I keep my ratings.", "questions": "If a model is initialized with ImageNet pretrain  weights and then optimized with a different dataset  A. When using an  image example from ImageNet dataset as the input of the TDD  method to  test a model, should the  outcome to  be Yes  or  No?", "limitations": "", "rating": 6, "confidence": 3, "soundness": 3, "presentation": 3, "contribution": 3, "strengths": "The benchmark is comprehensive. The TDDBench covers multiple data modalities and includes a wide range of datasets, algorithms, and model architectures, making it one of the most inclusive TDD benchmarks available.\n\nThe writing and formatting are decent.", "weaknesses": "It will be great if the authors can propose a novel method to improve the performance based  on the analysis in their benchmark experiments.\n\nIt  lacks  an important discussion about the TDD and different training methods. There are supervised learning, self-supervised learning, unsupervised learning. Would  TDD method generalizable to different training methods?", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "WgGzWjnnZ1", "reviewer": "Reviewer_j3dK", "summary": "The paper benchmarks 21 different TDD methods across four detection paradigms and evaluate their performance from five perspectives: average detection performance, best detection performance, memory consumption, and computational efficiency in both time and memory.", "questions": "see above", "limitations": "", "rating": 6, "confidence": 3, "soundness": 3, "presentation": 3, "contribution": 3, "strengths": "1. The paper is written well and is easy to understand.\n\n2. The studied problem is very important.\n\n3. The benchmark is comprehensive", "weaknesses": "1. It might be useful to include some discussion and experimental results on the defense of the training data detection algorithms\n\n2. The authors can consider elaborating more on the takeaway messages from their experimentations and include more discussions on the current limations of the detection algorithms and the possible future directions at the end of the paper\n\n3. It might be better to discuss the differences between the proposed benchmark with an existing benchmark. That will be useful.\n\n[1] Ye et al., Data Contamination Calibration for Black-box LLMs", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}]

Author rebuttal:

Decision: Accept (Poster)