TDDBench: A Benchmark for Training data detection

## Abstract

Abstract Training Data Detection (TDD) is a task aimed at determining whether a specific data instance is used to train a machine learning model. In the computer security literature, TDD is also referred to as Membership Inference Attack (MIA). Given its potential to assess the risks of training data breaches, ensure copyright authentication, and verify model unlearning, TDD has garnered significant attention in recent years, leading to the development of numerous methods. Despite these advancements, there is no comprehensive benchmark to thoroughly evaluate the effectiveness of TDD methods.
In this work, we introduce TDDBench, which consists of 13 datasets spanning three data modalities: image, tabular, and text. We benchmark 21 different TDD methods across four detection paradigms and evaluate their performance from five perspectives: average detection performance, best detection performance, memory consumption, and computational efficiency in both time and memory. With TDDBench, researchers can identify bottlenecks and areas for improvement in TDD algorithms, while practitioners can make informed trade-offs between effectiveness and efficiency when selecting TDD algorithms for specific use cases. Our large-scale benchmarking also reveals the generally unsatisfactory performance of TDD algorithms across different datasets. To enhance accessibility and reproducibility, we open-source TDDBench for the research community.

## 1 Introduction

Training Data Detection (TDD) (Shi et al., ), also known as Membership Inference Attack (MIA) in computer security literature (Shokri et al., ), aims to determine whether a specific data instance was used to train a target machine learning model. TDD has a wide range of applications. For example, it can be used to assess a model’s memorization of its training data and to audit the risks of data leakage (Carlini et al., ). TDD has gained even more importance in the era of deep learning and large language models (LLMs), where models, often with billions of parameters, act as opaque black boxes. This raises the need to examine whether model owners have illegally utilized copyrighted material, such as books (Abd-Alrazaq et al., ), or personal emails (Mozes et al., ). Moreover, TDD contributes to discussions on machine learning accountability in the era of AI, as concerns grow over how these models handle sensitive data. As machine unlearning becomes increasingly employed to remove users’ personal data from models, TDD serves as a critical tool to validate these unlearning processes (Chen et al., ; Kurmanji et al., ).

Given the growing importance of TDD, several benchmarks have been developed to evaluate TDD algorithms (Niu et al., ; He et al., ; Duan et al., ). However, these benchmarks have several limitations: 1). Most evaluations primarily focus on TDD algorithms for image data, leaving other modalities like text and tabular data underexplored. 2). Many TDD methods developed in the past two years, particularly those focused on deep learning and LLMs, are not included in these benchmarks. 3). The effect of the target model (i.e., the model that was trained using the data) on TDD algorithms has not been thoroughly examined. 4). Current evaluations focus primarily on the detection performance of TDD algorithms, while practical considerations like efficiency, memory consumption, and other factors relevant to real-world deployment are often overlooked.

Figure: Figure 1: TDDBench in downstream applications and the benchmarking of TDD algorithms.
Refer to caption: x1.png

To address these limitations, we introduce TDDBench, a comprehensive framework for benchmarking TDD algorithms. Figure 1 provides an overview of TDDBench. The benchmark includes 13 datasets across three data modalities (tabular, text, and image) and evaluates 21 state-of-the-art TDD algorithms on 41 different target models, including the large language model Pythia-12B. We also categorize the 21 TDD algorithms into four types based on their algorithmic characteristics, including metric-based, learning-based, model-based, and query-based. Using this new benchmark, we conduct large-scale experiments to thoroughly assess TDD algorithms. Specifically, we aim to investigate: 1). The performance of TDD algorithms across various datasets and data modalities. 2). The impact of the target model on TDD algorithms. 3). The limitations and areas for improvement in TDD algorithms. 4). The performance of TDD algorithms from multiple perspectives, including detection performance, practicality, and efficiency in terms of time and memory usage.

The large-scale experimental results reveal several key findings. First, there is a significant performance gap between different types of TDD algorithms, with model-based TDD methods generally outperforming the others. However, this outperformance comes at a cost, as model-based methods require building multiple reference models, leading to high computational expenses. Second, memorization of training data plays a crucial role in the performance of TDD algorithms, with larger target models—often prone to memorization—exhibiting higher TDD success rates. Third, the performance of TDD algorithms is highly dependent on knowledge of the underlying target model architecture. Overall, our experiments show that there is no single best method across all scenarios, and notably, many TDD algorithms perform poorly on data modalities beyond images, indicating the need for further improvement in non-image domains.

The main contributions of this paper are threefold:

A novel and comprehensive TDD benchmark: We introduce TDDBench, a benchmark consisting of datasets across three modalities—image, table, and text. We have open-sourced TDDBench for the research community at [https://anonymous.4open.science/r/TDDBench-8078](https://anonymous.4open.science/r/TDDBench-8078).

New insights in TDD performance: By benchmarking 21 state-of-the-art TDD algorithms, we provide insights into recent advancements in TDD, including strategies for reducing reliance on model-specific knowledge and maximizing the benefits of greater computational resources.

Multi-aspect metrics: Our comprehensive evaluation of TDD performance goes beyond simple detection accuracy to include practical considerations such as computational complexity, highlighting the trade-offs necessary for deploying TDD algorithms in real-world applications.

## 2 TDDBench: Training data detection benchmark across multiple modalities

### 2.1 Preliminaries and Problem Definition

Training Data Detection (TDD), also known as membership inference, is formally defined as follows: Given a target machine learning model $f_{\theta}$ and a data point $x$, the objective of TDD is to determine whether the target model used the data point during its training phase (Shokri et al., ; Carlini et al., ). Here, $\theta$ denotes the parameters of the target machine learning model, and $f_{\theta}$ is often referred to as the target model.

In this work, we consider black-box training data detection, meaning that we have access only to the outcomes of the target model for specific data points. There are two reasons for this assumption. First, many real-world target models hold significant commercial value and typically do not publicly disclose model parameters, making access to internal model parameters infeasible. Secondly, existing literature has shown that white-box detection methods offer limited advantages compared to black-box methods (Sablayrolles et al., ; Nasr et al., ).

### 2.2 Data Modalities, Datasets and Target Models

TDDBench consists of 13 datasets across three data modalities: image, tabular, and text. It also implements 11 distinct model architectures for these data modalities, resulting in a total of 41 target models. Additionally, TDDBench incorporates 21 state-of-the-art TDD algorithms. We illustrate the main differences between the proposed TDDBench and existing benchmarks in Table 1.

**Table 1: Comparison between TDDBench and existing benchmarks. TDDBench comprehensively includes the most algorithms and datasets across image, tabular, and text modalities, as well as model architectures that encompass large language models.**
| Benchmark | Coverage | Data type |  |  |  |  |  |
| --- | --- | --- | --- | --- | --- | --- | --- |
| # algo | # datasets | # architectures | LLM | image | tabular | text |  |
| He et al. () | 9 | 6 | 4 | ✗ | ✓ | ✗ | ✗ |
| Niu et al. () | 15 | 7 | 7 | ✗ | ✓ | ✓ | ✗ |
| Duan et al. () | 5 | 8 | 8 | ✓ | ✗ | ✗ | ✓ |
| TDDBench (ours) | 21 | 13 | 11 | ✓ | ✓ | ✓ | ✓ |

Dataset. Table 2 presents a summary of the datasets in TDDBench. It includes three data modalities: image, tabular, and text. TDDBench incorporates datasets commonly used to evaluate TDD algorithms in previous literatures (Truex et al., ; Hui et al., ), such as CIFAR-10 and Purchase. We also compile new datasets that potentially contain private or copyright-sensitive information, including CelebA (human faces), BloodMNIST (medical), Adult (personal income), and Tweet (social networks), which are more likely to necessitate TDD for tasks like copyright verification and unlearning confirmation. Additionally, WIKIMIA is a dataset specifically designed to evaluate TDD algorithms on large language models.

**Table 2: Benchmarking datasets used in TDDBench.**
| Modality | Dataset | #Samples | #Classes | Brief description |
| --- | --- | --- | --- | --- |
| Image | CIFAR-10 (Krizhevsky et al., ) | 60,000 | 10 | General dataset |
| CIFAR-100 (Krizhevsky et al., ) | 60,000 | 100 | General dataset |  |
| BloodMNIST (Yang et al., ) | 17,092 | 8 | Medical image |  |
| CelebA (Liu et al., ) | 202,599 | 2 | Human face |  |
| Tabular | Purchase (Shokri et al., ) | 197,324 | 100 | Purchase record |
| Texas (Shokri et al., ) | 67,330 | 100 | Hospital discharge data |  |
| Adult (Asuncion et al., ) | 48,842 | 2 | Personal income |  |
| Student (Cortez & Silva, ) | 4,424 | 3 | Education information |  |
| Text | Rotten Tomatoes (Pang & Lee, ) | 10,662 | 2 | Movie reviews |
| Tweet Eval (Barbieri et al., ) | 12,970 | 2 | User tweets |  |
| GLUE-CoLA (Wang et al., ) | 9,594 | 2 | Books and journal articles |  |
| ECtHR Articles (Chalkidis et al., ) | 5,063 | 13 | Legal texts |  |
| WIKIMIA (Shi et al., ) | 1,650 | 2 | General dataset |  |

Target Models. We select various model architectures for each data modality. Specifically, for image datasets, we train WRN28-2 (Zagoruyko, ), ResNet18 (He et al., ), VGG11 (Simonyan & Zisserman, ), and MobileNet-v2 (Sandler et al., ). For the tabular datasets, we employ Multilayer Perceptron (Rumelhart et al., ), CatBoost (Dorogush et al., ), and Logistic Regression (Hosmer Jr et al., ).

For textual datasets, except WIKIMIA, in contrast to the target models for the image and tabular modalities, which are trained from scratch, we fine-tune the open-source pre-trained language models DistilBERT (Sanh et al., ), RoBERTa (Liu et al., ), and Flan-T5 (Chung et al., ) on the text datasets, enabling us to detect fine-tuned data using the TDD algorithm. Finally, for the WIKIMIA dataset, we use it to perform TDD on large language models, specifically focusing on the open-sourced Pythia(Biderman et al., ). Training details of the target models are presented in Appendix A.3.

In summary, we implement different target models for each data modality. Since we have four image datasets, each with four target models, the total combination is 16 target image models. Similarly, for tabular data, four datasets and three target models give us a total of 12 target tabular models. For text data, four datasets and three target models provide a total of 12 target text models. Finally, Pythia is used as the target model to examine TDD on the WIKIMIA text dataset. In total, we have 41 target models, which to our knowledge, is one of the most comprehensive benchmarks for TDD.

### 2.3 TDD Algorithms

We implement 21 state-of-the-art TDD algorithms in TDDBench. To facilitate comparison and discussion, we categorize these TDD algorithms into four groups based on the algorithm’s design paradigm: metric-based, learning-based, model-based, and query-based algorithms. Table 3 provides an overview of the implemented TDD algorithms in TDDBench, outlining their categories and detection criteria.
These TDD algorithms are discussed in detail in Appendix A.1.

**Table 3: Summary of training data detection methods in TDDBench.**
| Algorithm type | Algorithm | Detection criterion |
| --- | --- | --- |
| Metric-based | Metric-loss (Yeom et al., ) | Loss |
| Metric-conf (Song et al., ) | Confidence |  |
| Metric-corr (Leino & Fredrikson, ) | Correctness |  |
| Metric-ent (Shokri et al., ; Song & Mittal, ) | Entropy |  |
| Metric-ment (Song & Mittal, ) | Modified prediction entropy |  |
| Learning-based | Learn-original (Shokri et al., ) | Prediction vector |
| Learn-top3 (Salem et al., ) | Top3 confidence |  |
| Learn-sorted (Salem et al., ) | Sorted prediction vector |  |
| Learn-label (Nasr et al., ) | Prediction vector, true label |  |
| Learn-merge (Amit et al., ) | Merging of various detection criteria |  |
| Model-based | Model-loss (Sablayrolles et al., ) | Loss |
| Model-calibration (Watson et al., ) | Loss |  |
| Model-lira (Carlini et al., ) | Scaled logit |  |
| Model-fpr (Ye et al., ) | Scaled logit |  |
| Model-robust (Zarifzadeh et al., ) | Confidence |  |
| Query-based | Query-augment (Choquette-Choo et al., ) | Correctness |
| Query-transfer (Li & Zhang, ) | Loss from surrogate model |  |
| Query-adv (Li & Zhang, ; Choquette-Choo et al., ) | Distance from the decision boundary |  |
| Query-neighbor (Jayaraman et al., ; Mattern et al., ) | Loss |  |
| Query-qrm (Bertran et al., ) | Scaled logit |  |
| Query-ref (Wen et al., ) | Scaled logit |  |

Metric-based methods rely on the analysis of certain statistical properties of a target model’s output, such as confidence scores, prediction probabilities, or loss values, to distinguish between training data and non-training data. Specifically, Metric-loss (Yeom et al., ) is the first metric-based detection method, predicting that data points with a loss below a certain threshold are part of the training data for the target model. Similarly, other works have proposed using the maximum confidence of the target model output (denoted as Metric-conf (Song et al., )), the correctness of the target model output (denoted as Metric-corr (Leino & Fredrikson, )), the entropy of prediction probability distributions (denoted as Metric-ent (Shokri et al., ; Song & Mittal, )), and modified entropy of the prediction (denoted as Metric-ment (Song & Mittal, )).

Learning-based methods involve training an auxiliary classifier (meta-classifier) to distinguish between training data and non-training data. In the literature, neural networks (NNs) are often employed as the auxiliary classifier. The primary differences between learning-based TDD methods lie in the choice of input features for the auxiliary classifier. Earlier work (Shokri et al., ) has proposed using the original prediction vector of the target model (denoted as Learn-original). Other works have suggested using the top-3 prediction confidences (denoted as Learn-top3 (Salem et al., )) , the sorted prediction vector (denoted as Learn-sorted (Salem et al., )) , the true label of the example combined with the prediction vector (denoted as Learn-label (Nasr et al., )) , and a mix of different detection metrics (denoted as Learn-merge (Amit et al., )). In black-box TDD scenarios, a shadow model is constructed to mimic the behavior of the target model, providing the necessary data to train the auxiliary classifier.

Model-based methods involve building multiple reference models, some of which are trained with the focal data point $x$, while others are trained without it. The detection method then analyzes the characteristics (such as loss distribution) of data points when they are included in training versus when they are not. The target model’s output on the focal data point is then compared to the reference models’ characteristics to determine whether it was used in training. Compared to metric-based and learning-based methods, model-based methods do not solely rely on the target model’s output, but can compare it with reference models. These methods have gained significant attention in recent years due to their superior performance. In the literature, different model-based methods utilize reference models in various ways, including learning the loss distribution of data points (denoted as Model-loss (Sablayrolles et al., ) and Model-calibration (Watson et al., )), transforming TDD into a likelihood ratio problem based on the scaled logits of prediction results (denoted as Model-lira (Carlini et al., )), designing TDD that satisfies different false positive ratios (denoted as Model-fpr (Ye et al., )), and creating more robust TDD methods (denoted as Model-robust (Zarifzadeh et al., )).

Query-based methods involve using additional data instances, particularly those close to the focal data point $x$, to query the target model. Compared to the other three types of detection methods, query-based methods leverage more output information from the target model to estimate the likelihood that the focal data point was used in model training. Specifically, we consider a data augmentation-based query method (denoted as Query-augment (Choquette-Choo et al., )), a neighbor-based method (denoted as Query-neighbor (Jayaraman et al., ; Mattern et al., )), a surrogate model-based method (denoted as Query-transfer (Li & Zhang, )), an adversarial learning-based method (denoted as Query-adv (Li & Zhang, ; Choquette-Choo et al., )), a quantile regression model- based method (denoted as Query-qrm (Bertran et al., )), and a reference-model-based query method (denoted as Query-ref (Wen et al., )).

It is also worth noting that different types of TDD methods may have varying requirements and assumptions for executing the detection. For example, metric-based methods have the fewest assumptions, relying solely on the target model’s output for prediction. In contrast, some model-based and query-based methods require additional auxiliary data to build reference models for prediction. In TDDBench, to ensure a fair comparison, we provide auxiliary data for methods that need it, ensuring that each method achieves its best possible detection performance.

## 3 Experiment Results and Analyses

Having compiled TDDBench, we now benchmark the performance of TDD algorithms. Since TDD algorithms can largely be categorized into four types based on their design paradigms, our experimental analysis is conducted at the category level. This allows us to systematically compare the strengths and weaknesses of each type of TDD algorithm.

We conduct experiments in three modalities including image, tabular, and text, to answer the following questions:
Q1: What is the overall performance of the TDD algorithm across different datasets and model architectures?
Q2: How does the target model impact the performance of the TDD algorithm, including model size and training-data memorization?
Q3: How does the TDD algorithm perform when knowledge about the target model architecture is limited?
Q4: How does the TDD algorithm perform in terms of overall performance, practicality, efficiency, and other factors?

### 3.1 Experiment Setting

Evaluation Protocol. We follow prior literatures in TDD evaluation (Carlini et al., ; Ye et al., ). Specifically, given a dataset in TDDBench, we divide the dataset into a target dataset and an auxiliary dataset in a 50:50 ratio. For the target dataset, we further split it into two halves, where the first half serves as the training dataset to train the target model (e.g., an image classifier), and the remaining half is not used in training the target model. Therefore, the training dataset serves as the positive examples for training data detection, while the remaining data serves as the negative examples.

For TDD algorithms, such as model-based and learning-based methods that require training reference models or shadow models, we follow the approach in (Carlini et al., ; Wen et al., ) by randomly partitioning the target dataset multiple times to train various reference and shadow models. The auxiliary dataset, also referred to as the population dataset in (Ye et al., ) and the shadow dataset in (Shokri et al., ), is available at the user’s discretion for use in the TDD algorithms. The auxiliary and target datasets do not overlap, ensuring that the auxiliary data is not accidentally used in training the target model. This characteristic allows for the training of quantile regression model and reference model that exclude the focal data point $x$, which are utilized in certain TDD algorithms.

Target Model Implementation. We implement target models as described in Section 2.2. Techniques such as early stopping, data augmentation, and dropout are utilized to maximize the target model’s predictive accuracy (e.g., for tasks like image classification or sentiment analysis). The training and test accuracy of the target models, along with detailed training information, can be found in Appendix A.3.

TDD Method Implementation. For the metric-based TDD methods, as they rely solely on the target model’s prediction outcome, the implementation is straightforward. For the learning-based TDD methods, we construct a two-layer neural network with 64 and 32 hidden units as the auxiliary classifier. The learning rate is set to 0.001, using the Adam optimizer, and training continues until the validation accuracy does not improve for 30 epochs or until a maximum of 500 epochs is reached. For the model-based TDD methods, we train 16 reference models. Finally, for the query-based TDD methods, including Query-neighbor, Query-augment, and Query-ref, we limit the detection algorithms to a maximum of 10 additional queries per data point.

Evaluation Metrics, Mean, and Standard Deviation. TDD is framed as a binary classification problem that determines whether a data point was used in training the target model. Accordingly, we primarily use AUROC to evaluate the performance of TDD algorithms. Additionally, we include nine supplementary metrics, such as Precision, Accuracy, and TPR@1% FPR, with detailed experimental results provided in Appendix A.6. To ensure the robustness of the experimental results, we perform multiple random partitions for each dataset and independently repeat the experiments five times. We then report the average performance of all TDD algorithms. Standard deviations across the five repeated experiments are also measured, and due to page limitations, the complete standard deviation results are reported in Appendix A.5.

### 3.2 Overall detection performance across different datasets and models

The main results from benchmarking TDD algorithms are presented in Tables 4 and 5. Specifically, Table 4 reports the average performance of TDD methods across different datasets, controlling for the same target model architecture within each modality. Table 5, on the other hand, presents the average performance of TDD methods across different target model architectures, benchmarked on CIFAR10 for image data, Purchase for tabular data, and Rotten Tomatoes for text data. Additionally, results involving large language models are illustrated in Figure 3(c) in Section 3.3.2. The results lead to several key findings:

Overall performance is not satisfactory. In most experimental settings, the AUC scores range between 0.5 and 0.6. From an AUC perspective, this is clearly unsatisfactory, as it indicates a high rate of false negatives and false positives. In other words, data points used by the target model are frequently misclassified as not being used, and vice versa. This is concerning and highlights the urgent need for advancing the performance of TDD methods. For model-based and query-based methods, whose results may be influenced by the number of reference models and queries, we perform a robust analysis by varying the number of reference models and queries. The results remain largely consistent across different configurations, as shown in Appendix A.4.

Model-based TDD methods achieve generally better detection performance. Across datasets and target models, model-based detection algorithms consistently outperform other methods. For instance, as shown in Table 4, all five model-based algorithms achieve an average performance near or above 0.65 across all 12 datasets, whereas the performance of metric-based and learning-based methods is substantially lower. Overall, these results highlight the performance advantage of model-based TDD methods.

Data’s task label information is useful. Some TDD methods leverage the focal data point’s ground truth label (e.g., image class label or sentiment class), while others do not. Experimental results demonstrate that incorporating label information significantly improves detection performance. For instance, Metric-ment consistently outperforms Metric-ent by utilizing data labels. Similar improvements are observed with Learn-label compared to Learn-original, where the former benefits from leveraging the label information while the latter does not.

Hybrid method has potential. Notably, Query-ref, which generates crafted query data for the image modality, achieves the best performance among all 21 TDD algorithms. While categorized as query-based, this method also trains reference models, similar to model-based methods, making it a hybrid of query-based and model-based approaches. This highlights the potential of combining the merits of different methods to enhance detection accuracy.

**Table 4: AUROC of TDD algorithms across different datasets. WRN28-2, Multilayer Perceptron, and DistilBERT are trained on image, tabular, and text datasets, respectively. The last column of each table displays the average performance of the corresponding TDD algorithm across different datasets. Complete results with standard deviations are provided in Table 11 in the Appendix A.5.**
| Modality | Image | Tabular | Text | Avg. |  |  |  |  |  |  |  |  |  |
| --- | --- | --- | --- | --- | --- | --- | --- | --- | --- | --- | --- | --- | --- |
| Dataset | CIFAR-10 | CIFAR-100 | BloodMNIST | CelebA | Purchase | Texas | Adult | Student | Rotten | Tweet | CoLA | ECtHR |  |
| Metric-loss | 0.635 | 0.858 | 0.527 | 0.509 | 0.619 | 0.629 | 0.500 | 0.566 | 0.582 | 0.566 | 0.571 | 0.521 | 0.590 |
| Metric-conf | 0.635 | 0.858 | 0.527 | 0.509 | 0.619 | 0.629 | 0.500 | 0.566 | 0.582 | 0.566 | 0.571 | 0.521 | 0.590 |
| Metric-corr | 0.552 | 0.708 | 0.517 | 0.507 | 0.551 | 0.610 | 0.501 | 0.560 | 0.557 | 0.550 | 0.550 | 0.519 | 0.557 |
| Metric-ent | 0.628 | 0.848 | 0.525 | 0.508 | 0.616 | 0.563 | 0.498 | 0.520 | 0.561 | 0.528 | 0.519 | 0.507 | 0.568 |
| Metric-ment | 0.635 | 0.858 | 0.527 | 0.509 | 0.620 | 0.630 | 0.500 | 0.566 | 0.582 | 0.566 | 0.571 | 0.522 | 0.591 |
| Learn-original | 0.631 | 0.870 | 0.508 | 0.503 | 0.652 | 0.597 | 0.502 | 0.531 | 0.558 | 0.529 | 0.568 | 0.506 | 0.580 |
| Learn-top3 | 0.628 | 0.851 | 0.526 | 0.503 | 0.677 | 0.573 | 0.500 | 0.520 | 0.561 | 0.528 | 0.531 | 0.502 | 0.575 |
| Learn-sorted | 0.628 | 0.850 | 0.529 | 0.508 | 0.666 | 0.573 | 0.501 | 0.520 | 0.561 | 0.528 | 0.510 | 0.501 | 0.573 |
| Learn-label | 0.633 | 0.882 | 0.515 | 0.507 | 0.656 | 0.669 | 0.503 | 0.590 | 0.584 | 0.570 | 0.622 | 0.517 | 0.604 |
| Learn-merge | 0.656 | 0.893 | 0.523 | 0.507 | 0.684 | 0.686 | 0.502 | 0.595 | 0.584 | 0.569 | 0.620 | 0.530 | 0.612 |
| Model-loss | 0.664 | 0.852 | 0.560 | 0.522 | 0.725 | 0.767 | 0.509 | 0.670 | 0.773 | 0.756 | 0.752 | 0.655 | 0.684 |
| Model-calibration | 0.639 | 0.763 | 0.553 | 0.520 | 0.684 | 0.718 | 0.508 | 0.648 | 0.695 | 0.714 | 0.699 | 0.638 | 0.648 |
| Model-lira | 0.690 | 0.937 | 0.536 | 0.512 | 0.755 | 0.753 | 0.503 | 0.634 | 0.753 | 0.728 | 0.737 | 0.604 | 0.679 |
| Model-fpr | 0.647 | 0.852 | 0.552 | 0.516 | 0.697 | 0.723 | 0.507 | 0.641 | 0.679 | 0.722 | 0.708 | 0.635 | 0.657 |
| Model-robust | 0.635 | 0.889 | 0.552 | 0.520 | 0.711 | 0.762 | 0.509 | 0.669 | 0.766 | 0.745 | 0.746 | 0.621 | 0.677 |
| Query-augment | 0.573 | 0.761 | 0.517 | 0.502 | 0.612 | 0.612 | 0.500 | 0.560 | 0.570 | 0.551 | 0.561 | 0.518 | 0.570 |
| Query-transfer | 0.522 | 0.622 | 0.503 | 0.502 | 0.529 | 0.581 | 0.499 | 0.522 | 0.530 | 0.530 | 0.526 | 0.510 | 0.531 |
| Query-adv | 0.615 | 0.838 | 0.508 | 0.514 | 0.620 | 0.579 | 0.500 | 0.563 | 0.571 | 0.551 | 0.568 | 0.519 | 0.579 |
| Query-neighbor | 0.511 | 0.553 | 0.497 | 0.501 | 0.533 | 0.612 | 0.500 | 0.535 | 0.533 | 0.556 | 0.550 | 0.522 | 0.534 |
| Query-qrm | 0.532 | 0.574 | 0.510 | 0.505 | 0.523 | 0.530 | 0.500 | 0.526 | 0.524 | 0.521 | 0.511 | 0.512 | 0.522 |
| Query-ref | 0.735 | 0.941 | 0.566 | 0.526 | N/A | N/A | N/A | N/A | N/A | N/A | N/A | N/A | 0.692 |

**Table 5: AUROC of TDD algorithms across different target model architectures. MLP stands for Multilayer Perceptron, and LR stands for Logistic Regression. The last column of each table displays the average performance of the corresponding TDD algorithm across different model architectures. Complete results with standard deviations are provided in Table 12 in the Appendix A.5.**
| Dataset | CIFAR10(Image) | Purchase(Tabular) | Rotten-tomatoes(Text) | Avg. |  |  |  |  |  |  |  |
| --- | --- | --- | --- | --- | --- | --- | --- | --- | --- | --- | --- |
| Target model | WRN28-2 | ResNet18 | VGG11 | MobileNet-v2 | MLP | CatBoost | LR | DistilBERT | RoBERTa | Flan-T5 |  |
| Metric-loss | 0.635 | 0.659 | 0.684 | 0.592 | 0.619 | 0.948 | 0.640 | 0.582 | 0.571 | 0.517 | 0.645 |
| Metric-conf | 0.635 | 0.659 | 0.684 | 0.592 | 0.619 | 0.948 | 0.640 | 0.582 | 0.571 | 0.517 | 0.645 |
| Metric-corr | 0.552 | 0.557 | 0.574 | 0.548 | 0.551 | 0.636 | 0.622 | 0.557 | 0.542 | 0.513 | 0.565 |
| Metric-ent | 0.628 | 0.654 | 0.680 | 0.582 | 0.616 | 0.943 | 0.594 | 0.561 | 0.555 | 0.509 | 0.632 |
| Metric-ment | 0.635 | 0.659 | 0.685 | 0.592 | 0.620 | 0.950 | 0.642 | 0.582 | 0.571 | 0.517 | 0.645 |
| Learn-original | 0.631 | 0.623 | 0.694 | 0.533 | 0.652 | 0.935 | 0.644 | 0.558 | 0.546 | 0.515 | 0.633 |
| Learn-top3 | 0.628 | 0.653 | 0.680 | 0.582 | 0.677 | 0.967 | 0.660 | 0.561 | 0.555 | 0.509 | 0.647 |
| Learn-sorted | 0.628 | 0.654 | 0.680 | 0.578 | 0.666 | 0.963 | 0.661 | 0.561 | 0.555 | 0.509 | 0.646 |
| Learn-label | 0.633 | 0.612 | 0.707 | 0.557 | 0.656 | 0.954 | 0.701 | 0.584 | 0.565 | 0.520 | 0.649 |
| Learn-merge | 0.656 | 0.628 | 0.727 | 0.528 | 0.684 | 0.968 | 0.716 | 0.584 | 0.566 | 0.518 | 0.657 |
| Model-loss | 0.664 | 0.709 | 0.729 | 0.607 | 0.725 | 0.975 | 0.776 | 0.773 | 0.656 | 0.602 | 0.721 |
| Model-calibration | 0.639 | 0.671 | 0.690 | 0.595 | 0.684 | 0.865 | 0.719 | 0.695 | 0.622 | 0.592 | 0.677 |
| Model-lira | 0.690 | 0.749 | 0.780 | 0.601 | 0.755 | 0.995 | 0.761 | 0.753 | 0.630 | 0.569 | 0.728 |
| Model-fpr | 0.647 | 0.684 | 0.712 | 0.619 | 0.697 | 0.976 | 0.724 | 0.679 | 0.623 | 0.589 | 0.695 |
| Model-robust | 0.635 | 0.677 | 0.704 | 0.602 | 0.711 | 0.983 | 0.796 | 0.766 | 0.639 | 0.574 | 0.709 |
| Query-augment | 0.573 | 0.575 | 0.633 | 0.542 | 0.612 | 0.696 | 0.664 | 0.570 | 0.546 | 0.512 | 0.592 |
| Query-transfer | 0.522 | 0.522 | 0.533 | 0.507 | 0.529 | 0.587 | 0.574 | 0.530 | 0.515 | 0.506 | 0.533 |
| Query-adv | 0.615 | 0.621 | 0.666 | 0.583 | 0.620 | 0.727 | 0.662 | 0.571 | 0.552 | 0.516 | 0.613 |
| Query-neighbor | 0.511 | 0.512 | 0.509 | 0.509 | 0.533 | 0.820 | 0.530 | 0.533 | 0.527 | 0.504 | 0.549 |
| Query-qrm | 0.532 | 0.537 | 0.541 | 0.530 | 0.523 | 0.946 | 0.632 | 0.524 | 0.528 | 0.506 | 0.580 |
| Query-ref | 0.735 | 0.800 | 0.843 | 0.656 | N/A | N/A | N/A | N/A | N/A | N/A | 0.759 |

### 3.3 The impact of target model

In this section, we examine the impact of the target model on TDD detection performance.

#### 3.3.1 Data Memorization and Overfitting

A common view is that the effectiveness of TDD is closely tied to the level of training-data memorization or overfitting exhibited by the target model during training (Yeom et al., ; Long et al., ). The disparity between the target model’s accuracy on the training set and the test set, known as the train-test accuracy gap, serves as an indicator of data memorization in prior literature (Carlini et al., ). In our experiments, we document the train-test gaps of 12 distinct target models in Table 4, along with the corresponding performance of all detection methods. Specifically, the training of target models is repeated five times with different random training samples. Figure 2 illustrates the performance of various detection algorithms across different train-test gaps, with error bars representing 95% confidence intervals obtained from five independent trials. It is evident that the performance of all TDD methods is positively correlated with the train-test accuracy gap of the target model. When the train-test gap is maximal, indicating the target model may overfit to the training data, identifying whether a data point was used by the target model becomes relatively easier. In contrast, when the target model has a minimal train-test gap, the TDD algorithms often fall short in detection performance. Therefore, it is crucial for future TDD research to examine the generalizability of target models and report TDD performance when the train-test gap is small.

Figure: (a) Image
Refer to caption: x2.png

#### 3.3.2 Target model size

We examine the impact of target model size on TDD performance. Specifically, in our experiment, we vary the number of layers in the ResNet architecture for image data, the number of hidden units in the MLP for tabular data, and the parameter sizes of the large language model Pythia for text data.

Due to limitations in computing resources, TDD on large models often does not involve the creation of shadow models or reference models. Drawing from prior studies (Shi et al., ; Duan et al., ), we utilize multiple detection methods suitable for pretrained large language models. Detailed descriptions of these detection methods can be found in Appendix A.2. Additionally, due to the lack of specific information regarding the training data used for large language models, we utilize the WIKIMIA (Shi et al., ), which collects training and non-training data for the large language model based on the model’s release timeline to evaluate the TDD method in large language models.

The results of the experiments are illustrated in Figure 3. It is observed that, in most cases, the performance of the detection method improves as the size of the model increases. This aligns with the expectation that an increase in model size typically enhances model memorization (Carlini et al., ; Arpit et al., ). However, an exception occurs when the number of layers in the ResNet model is expanded from 34 to 50, resulting in a decline in the detection method’s performance. One potential explanation for this anomaly is that the integration of residual connections in ResNet helps alleviate issues related to excessive memorization stemming from the increased depth of the model.

Figure: (a) Image:ResNet
Refer to caption: x5.png

### 3.4 Performance when Knowledge about the Target Model is Limited

In the above experiments, we assumed that despite the black-box setting, TDD algorithms had some knowledge about the target model’s training algorithm. However, in real-world scenarios, it is possible that the data owner may lack detailed knowledge about the target model’s architecture, leading to significant differences between the reference and shadow models constructed by the TDD method and the actual target model. To explore this issue, we assess the performance of TDD when the reference and shadow models differ from the target model.

The results, presented in Table 6, show a noticeable decline in detection performance when the data owner has limited knowledge about the target model’s architecture. For example, Learn-original exhibits a 5.3% performance decline when using ResNet18 as the shadow model. This performance degradation can be attributed to discrepancies between the shadow and target models, which result in biased input features for training the auxiliary classifier. Moreover, the overall AUC performance remains unsatisfactory, highlighting the ineffectiveness of TDD algorithms in most datasets when knowledge about the target model is limited.

**Table 6: TDD algorithm performance (AUROC) when the reference or shadow models are different from the target model (i.e., when knowledge about the target model is limited). Complete results with standard deviations are provided in Table 13 in Appendix A.5.**
| Target model | WRN28-2(CIFAR-10) | MLP(Purchase) | DistilBERT(Rotten-tomatoes) | Avg. |  |  |  |  |  |  |  |
| --- | --- | --- | --- | --- | --- | --- | --- | --- | --- | --- | --- |
| Shadow/reference model | WRN28-2 | ResNet18 | VGG11 | MobileNet-v2 | MLP | CatBoost | LR | DistilBERT | RoBERTa | Flan-T5 |  |
| Learn-original | 0.631 | 0.578 | 0.632 | 0.539 | 0.652 | 0.651 | 0.564 | 0.558 | 0.560 | 0.546 | 0.591 |
| Learn-top3 | 0.628 | 0.628 | 0.628 | 0.628 | 0.677 | 0.646 | 0.515 | 0.561 | 0.561 | 0.561 | 0.603 |
| Learn-sorted | 0.628 | 0.629 | 0.628 | 0.629 | 0.666 | 0.656 | 0.532 | 0.561 | 0.561 | 0.561 | 0.605 |
| Learn-label | 0.633 | 0.591 | 0.644 | 0.563 | 0.656 | 0.651 | 0.551 | 0.584 | 0.584 | 0.580 | 0.604 |
| Learn-merge | 0.656 | 0.581 | 0.651 | 0.509 | 0.684 | 0.517 | 0.595 | 0.584 | 0.584 | 0.580 | 0.594 |
| Model-loss | 0.664 | 0.657 | 0.641 | 0.632 | 0.725 | 0.608 | 0.611 | 0.773 | 0.607 | 0.589 | 0.651 |
| Model-calibration | 0.639 | 0.634 | 0.617 | 0.614 | 0.684 | 0.579 | 0.588 | 0.695 | 0.595 | 0.587 | 0.623 |
| Model-lira | 0.690 | 0.659 | 0.666 | 0.610 | 0.755 | 0.686 | 0.588 | 0.753 | 0.602 | 0.553 | 0.656 |
| Model-fpr | 0.647 | 0.668 | 0.638 | 0.664 | 0.697 | 0.645 | 0.643 | 0.679 | 0.557 | 0.567 | 0.641 |
| Model-robust | 0.635 | 0.639 | 0.633 | 0.621 | 0.711 | 0.632 | 0.625 | 0.766 | 0.624 | 0.591 | 0.648 |
| Query-augment | 0.573 | 0.555 | 0.575 | 0.552 | 0.612 | 0.612 | 0.612 | 0.570 | 0.569 | 0.565 | 0.580 |
| Query-transfer | 0.522 | 0.529 | 0.518 | 0.518 | 0.529 | 0.535 | 0.529 | 0.530 | 0.529 | 0.514 | 0.525 |
| Query-qrm | 0.532 | 0.532 | 0.533 | 0.532 | 0.523 | 0.625 | 0.622 | 0.524 | 0.524 | 0.528 | 0.548 |
| Query-ref | 0.735 | 0.740 | 0.722 | 0.708 | N/A | N/A | N/A | N/A | N/A | N/A | 0.726 |

### 3.5 Performance tradeoff of TDD algorithms

**Table 7: Quantitative evaluation of different types of TDD algorithms including the average and best AUROC, maximum runtime and memory usage.**
| Algorithm type | Average performance | Best performance | Running time(s) | Memory usage(MB) |
| --- | --- | --- | --- | --- |
| Metric-based | 0.626 | 0.645 | 232 | 0 |
| Learning-based | 0.646 | 0.657 | 2,107 | 855 |
| Model-based | 0.706 | 0.728 | 4,089 | 13,680 |
| Query-based | 0.604 | 0.759 | 40,963 | 13,680 |

Most evaluations of TDD algorithms primarily focus on detection accuracy. However, other factors, such as computational efficiency, are equally important in real-world applications. For instance, model-based methods, which require building numerous reference models, may be too costly in terms of time and memory when applied to large AI models. Therefore, in TDDBench, we emphasize the computational complexity of running different TDD algorithms. Specifically, we document the maximum runtime and memory usage for each type of TDD algorithm. This provides a holistic evaluation beyond detection accuracy. We present an overall assessment of the four types of TDD algorithms in Table 7. Evidently, each type of TDD algorithm has its own advantages and disadvantages. While model-based methods offer the best average performance, they come with significantly higher running time and memory usage. Therefore, data owners performing TDD must strike a balance between practicality, resource utilization, and detection accuracy, depending on their specific scenario. For instance, in resource-constrained environments, metric-based methods are a suitable choice for TDD, as they require minimal computational resources and fewer assumptions compared to other methods.

## 4 Related Work

Training data detection (TDD) is commonly employed to assess privacy risks in machine learning models (Murakonda & Shokri, ). It has been applied across various domains, including image classification (Hui et al., ; Rezaei & Liu, ), text generation (Shejwalkar et al., ; Oh et al., ), graph neural networks (Wu et al., ; Olatunji et al., ), and recommendation systems (Zhang et al., ; Wang et al., ).
TDD has a wide range of applications such as dataset copyright protection (Maini et al., ) and for verifying machine unlearning (Chen et al., ).

Shokri et al., introduce the first TDD algorithm, utilizing shadow models to help identifying differences in the model’s predictions for training data versus other data. Yeom et al., demonstrate that satisfactory results could be achieved by utilizing only the loss of the target model on the sample. Building on this foundation, subsequent studies employ the entropy (Song & Mittal, ) and maximum confidence (Song et al., ) of the target model’s predictions as additional detection criteria. Carlini et al., criticize methods based solely on the target model’s output, arguing that they overlook the inherent characteristics of the data, which can lead to biased estimates regarding whether a sample belongs to the training dataset. They propose training multiple reference models to better understand how the sample’s characteristics influence metrics like loss. Additionally, increasing the number of queries (Rahimian et al., ) to the target model is a viable improvement for TDD algorithms, with newly added query samples typically being neighbors of the data under detection, enriching the detection criterion. In short, there is a rapidly growing body of literature on TDD methods, and we provide a brief summarization in Section 2.3.

Existing benchmarking works. He et al., evaluate 9 TDD algorithms on image data, while Niu et al., expand the evaluation to 15 algorithms, focusing on how sample differences within datasets affect TDD performance. Duan et al., investigate five TDD algorithms on large language models (LLMs) and find that current TDD algorithms perform poorly in this context. In summary, existing TDD benchmarks have limited coverage of data modalities and algorithms, underscoring the need for a more comprehensive analysis of TDD algorithms. This paper, along with the developed TDDBench, aims to address this gap by providing in-depth insights into the development and performance tradeoff in state-of-the-art TDD algorithms.

## 5 Conclusion

In this article, we introduce TDDBench, a novel and comprehensive training data detection benchmark. Unlike existing benchmarks, TDDBench extends evaluations across multiple data modalities, including image, tabular, and text. It also implements large-scale target models and benchmarks 21 state-of-the-art TDD algorithms. Our comprehensive evaluation sheds critical light on the development of TDD algorithms and helps both researchers and practitioners reconsider the trade-offs involved in using TDD algorithms. For example, our evaluation shows that model-based TDD algorithms outperform others but at the cost of higher time and memory complexity. Additionally, all existing TDD algorithms experience performance degradation when the target model avoids overfitting. Based on our findings with TDDBench, we believe future work on TDD algorithms should focus on, but not be limited to: (1) designing TDD algorithms robust to target models less prone to overfitting, (2) creating TDD algorithms that require minimal knowledge of the target model architecture, (3) achieving a better balance between performance and practical considerations such as computational complexity, and (4) developing algorithms tailored to specific application contexts or data modalities, such as training data detection for large language models and recommendation systems.

## Appendix A Appendix

### A.1 Details of the algorithms included in TDDBench

#### A.1.1 Metric-based dectection

Several studies (Carlini et al., ; ) indicate that models retain a certain degree of memory regarding the training data during the learning process. This memorization can result in significant differences between the model’s predictions on training and test data, which can be leveraged as a decision basis for TDD algorithms. Specifically, Metric-loss (Yeom et al., ) utilizes the loss of the target model’s prediction on data points as the detection criterion. Since the target model is instructed to minimize training loss during optimization, a training data point typically exhibits a lower loss than a test data point. Similarly, Metric-conf (Metric-confidence) (Song et al., ) identifies that the maximum confidence of the target model’s predictions can also serve as the detection criterion. Metric-corr (Metric-correctness) (Leino & Fredrikson, ) further demonstrates that even without access to the model’s prediction confidence or logits for a specific data point, comparing the predicted label with the true label can provide an effective detection basis. Metric-corr achieves training data detection (TDD) with fewer assumptions than both Metric-loss and Metric-conf.

Beyond individual prediction values, the distribution of prediction results can also serve as a valuable detection criterion. Metric-ent (Metric-entropy) (Shokri et al., ; Song & Mittal, ) posits that the target model exhibits greater confidence in its predictions for training data, as evidenced by a more concentrated distribution of prediction confidences across different classes. Building on this, entropy is utilized as the detection criterion for Metric-ent. Metric-ment (Metric-modified entropy) (Song & Mittal, ) further incorporates the true label of the data point into Metric-ent to prevent the detection algorithm from predicting data points where the target model has misclassified as its training data.

#### A.1.2 Learning-based dectection

The metric-based algorithms design various metrics to extract detection basis from the prediction results of the target model. However, manually designed metrics may not accurately capture the differences between the predicted results of training and test data. A more robust approach is to use neural networks to automatically extract training data detection (TDD)-friendly information from the target model’s predictions, known as learning-based detection algorithms. Learn-original (Shokri et al., ) is the first algorithm to propose building an auxiliary classifier for TDD. It inputs the prediction vector of the target model into the auxiliary classifier, aiming to directly produce a detection result. To train the auxiliary classifier, Learn-original employs a shadow model similar to the target model, utilizing shadow training techniques. Since the shadow model’s training process is conducted by the detectors, they can obtain both the training and test data of the shadow model. The predictions made by the shadow model on its training and test data are utilized to facilitate the training of the auxiliary classifier.

Different learning-based detection algorithms primarily differ in the input features of their auxiliary classifiers. For instance, Learn-top3 and Learn-sorted (Salem et al., ) utilize the top-3 prediction confidences and ranked prediction vectors as input features, respectively. Building on Learn-original, Learn-label (Nasr et al., ) supplements the input features with the true label of the data point. Learn-merge (Amit et al., ) further incorporates the entropy, loss, and predicted label into the input features. It is noteworthy that while Learn-original builds multiple shadow models, most learning-based methods utilize only one. Moreover, previous work demonstrates that training data detection (TDD) with a single shadow model achieves performance comparable to that of multiple shadow models. Therefore, to ensure a fair comparison among learning-based methods, we standardize the number of shadow models to one for all learning-based approaches.

#### A.1.3 Model-based dectection

The two lines of TDD methods discussed above rely solely on the prediction results of the target model, overlooking the inherent characteristics of the data points, which may introduce bias into the detection criteria. For example, abnormal training data may exhibit higher losses than normal test data due to inherent data characteristics (Carlini et al., ; ), making it challenging for Metric-loss to detect these data. Therefore, the design of the detection criterion must consider data characteristics to eliminate bias.

Model-based algorithms address this issue by utilizing a set of reference models that share a similar architecture to the target model. These reference models are used to obtain predictions for data point $x$ across different models, which helps to de-bias the detection criteria of the target model. To elaborate, Model-loss (Sablayrolles et al., ) calculates the mean loss of data point $x$ across all reference models and then subtracts the loss from the target model to eliminate bias induced by data characteristics. In contrast, Model-calibration (Watson et al., ) uses only reference models that exclude data point $x$ from its training data, allowing it to implement model-based TDD for any new data point. Moreover, Model-lira (Carlini et al., ) treats the detection process as a likelihood ratio test, determining whether the rescaled logit value of data point $x$ in the target model originates from models trained on $x$. Building on Model-lira, Model-fpr (Ye et al., ) designs a detection method that meets the specified arbitrary false positive ratio. Model-robust (Zarifzadeh et al., ) introduces a robust TDD method that utilizes only one reference model.

#### A.1.4 Query-based dectection

The motivation for query-based algorithms stems from two main reasons. Firstly, some of them aim to implement label-only training data detection (TDD), where the target model provides only predicted labels. In such cases, the detector can depend solely on prediction correctness as the detection criterion, which limits the ability to acquire more intricate and effective detection information. To address this limitation, Query-augment(Query-augmentmentation) (Choquette-Choo et al., ) proposes obtaining multiple neighbors of a data point $x$ through data augmentation. The correctness of the target model on these augmented data points is then combined to form input features for the auxiliary classifier in the learning-based algorithm. Query-transfer (Li & Zhang, ), on the other hand, suggests training a surrogate model based on the prediction labels of the target model. The surrogate model is expected to closely resemble the target model and subsequently replace it to provide more detailed prediction results for arbitrary data points, enabling the generation of a more intricate detection criterion. Moreover, Query-adv(Query-adversarial) (Li & Zhang, ; Choquette-Choo et al., ) considers the distance of a data point from the target model’s decision boundary as a detection criterion with the aid of adversarial tools. This is based on the assumption that training data will generally be farther from the decision boundary than test data.

Another type of query-based algorithm does not assume that the target model only returns prediction labels. In these algorithms, additional queries are introduced to provide more information to aid in detection. For example, Query-neighbor (Jayaraman et al., ; Mattern et al., ) adds random noise to the data point $x$ and uses the difference between the loss of the target model on $x$ and its average loss on the neighboring points of $x$ as the detection criterion. Query-qrm (Bertran et al., ) collects a large amount of data that is explicitly not from the target model’s training data and obtains the scaled logits of the target model on these samples to train a quantile regression model. This quantile regression model can determine the likelihood that $x$ is not part of the target model’s training data. Additionally, Query-ref(Query-reference) (Wen et al., ) makes extra queries for adversarial samples of $x$ generated based on reference models. These samples help to better reflect the differences in the predicted results of $x$ when $x$ is training data versus when it is not.

Remark. Query-ref is categorized as a query-based method rather than a reference-based method because of its innovative query sample generation strategy. It is specifically designed to generate suitable query data for image datasets, rather than for tabular or text data.

### A.2 Training data detection algorithms in large language models

In this section, we provide a brief introduction to the TDD algorithms for large language models; please refer to the related works for specific details.

Loss (Yeom et al., ) refers to the Metric-loss mentioned in the article. Instead of using cross-entropy as in classification models, the log likelihood of each text under the target model serves as the basis for detection in pretraining language models.

Zlib (Carlini et al., ) calibrates the sample’s loss under the target model using the sample’s zlib compression size.

MIN-K% (Shi et al., ) utilizes the k% of tokens with the lowest likelihoods as the detection basis, rather than average loss.

MIN-K++ (Zhang et al., ) is based on the observation that training samples tend to be local maxima of the modeled distribution along each input dimension due to maximum likelihood training, effectively translating the TDD problem into the identification of local maxima.

Reference (Carlini et al., ) borrows from model-based approaches, utilizing the reference model to help correct the detection basis derived from the prediction results of the target model. Reference models for TDD in large language models are typically open-source and have architectures similar to the target model, thus avoiding the significant computational cost of training the reference model from scratch.

Neighbor, or Query-neighbor (Mattern et al., ), supplements the detection information provided by the sample point $x$ with the loss of the target model on the neighboring samples of $x$.

### A.3 Performances and training details of target models

**Table 8: Training accuracy and test accuracy of target models trained on different datasets (corresponds to Table 4 in Section 3.2). WRN28-2, Multilayer Perceptron, and DistilBERT are trained on image, table, and text datasets, respectively. Typically, target models trained on datasets with more categories exhibit smaller test accuracy and greater train-test accuracy gaps.**
| Modality | Dataset | # Classes | Train accuracy | Test accuracy | Train-test accuracy gap |
| --- | --- | --- | --- | --- | --- |
| Image | CIFAR-10 | 10 | 0.981 | 0.877 | 0.104 |
| CIFAR-100 | 100 | 1.000 | 0.583 | 0.417 |  |
| BloodMNIST | 8 | 0.989 | 0.955 | 0.034 |  |
| CelebA | 2 | 0.988 | 0.976 | 0.013 |  |
| Tabular | Purchase | 100 | 1.000 | 0.897 | 0.103 |
| Texas | 100 | 0.766 | 0.546 | 0.220 |  |
| Adult | 2 | 0.831 | 0.830 | 0.001 |  |
| Student | 3 | 0.855 | 0.735 | 0.121 |  |
| Text | Rotten tomatoes | 2 | 0.947 | 0.833 | 0.113 |
| Tweet Eval | 2 | 0.840 | 0.739 | 0.101 |  |
| GLUE-CoLA | 2 | 0.864 | 0.763 | 0.100 |  |
| ECtHR Articles | 13 | 0.476 | 0.438 | 0.038 |  |

**Table 9: Training accuracy and test accuracy of target models trained with different architectures(corresponds to Table 5 in Section 3.2). CIFAR-10 and Purchase datasets were used to train image models and tabular models from scratch, respectively. The Rotten Tomatoes dataset was used to fine-tune the pre-trained text models.**
| Dataset | Target model | Train accuracy | Test accuracy | Train-test accuracy gap |
| --- | --- | --- | --- | --- |
| CIFAR-10 | WRN28-2 | 0.981 | 0.877 | 0.104 |
| ResNet18 | 0.992 | 0.880 | 0.112 |  |
| VGG11 | 1.000 | 0.853 | 0.147 |  |
| MobileNet-v2 | 0.934 | 0.845 | 0.089 |  |
| Purchase | Multilayer Perceptron | 1.000 | 0.897 | 0.103 |
| CatBoost | 1.000 | 0.725 | 0.276 |  |
| Logistic Regression | 0.999 | 0.755 | 0.244 |  |
| Rotten tomatoes | DistilBERT | 0.947 | 0.833 | 0.113 |
| RoBERTa | 0.964 | 0.881 | 0.083 |  |
| Flan-T5 | 0.911 | 0.886 | 0.025 |  |

**Table 10: Training details for various model architectures, including learning rate, weight decay, maximum training epochs, and more. MLP stands for Multilayer Perceptron, and LR stands for Logistic Regression. ’N/A’ indicates that the model does not require consideration of the corresponding hyperparameter.**
| Modality | Target model | Learning rate | Weight decay | Maximum epochs | Optimizer | Learning rate schedule | Batch size |
| --- | --- | --- | --- | --- | --- | --- | --- |
| Image | WRN28-2 | 0.1 | 0.0005 | 200 | SGD | Cosine Annealing | 256 |
| ResNet18 | 0.1 | 0.0005 | 200 | SGD | Cosine Annealing | 256 |  |
| VGG11 | 0.1 | 0.0005 | 200 | SGD | Cosine Annealing | 256 |  |
| MobileNet-v2 | 0.1 | 0.0005 | 200 | SGD | Cosine Annealing | 256 |  |
| Tabular | MLP | 0.001 | 0.0001 | 200 | Adam | N/A | 256 |
| CatBoost | 0.05 | N/A | 10,000 | N/A | N/A | N/A |  |
| LR | N/A | N/A | 100 | N/A | N/A | N/A |  |
| Text | DistilBERT | 0.00002 | 0.01 | 10 | AdamW | N/A | 32 |
| RoBERTa | 0.00002 | 0.01 | 10 | AdamW | N/A | 32 |  |
| Flan-T5 | 0.00002 | 0.01 | 10 | AdamW | N/A | 32 |  |

### A.4 Detection performance with more queries and reference models.

In this part, we analyze the potential of the two latest types of TDD algorithms: reference-based and query-based. Specifically, we expand the number of reference models that the reference-based algorithm can create and the number of queries that the query-based algorithm can handle. Due to limitations in computational resources, we opt to decrease rather than increase the number of reference models in the text modality. We anticipate that similar conclusions can be drawn with a larger number of reference models. The results in Figure 4 demonstrate that increasing the number of queries or reference models can improve the TDD algorithm’s performance. Adjusting the number of reference models has a more significant impact on the algorithm’s performance compared to increasing the number of queries. Nevertheless, enhancing computational resources does not close the gap in algorithm design. For instance, in tabular data, even with 256 reference models, other algorithms fail to outperform LiRA with 16 reference models.

Figure: (a) Image
Refer to caption: x8.png

### A.5 Complete version of the experimental results

**Table 11: Complete version of TDD performance across different datasets (corresponds to Table 4). WRN28-2, Multilayer Perceptron, and DistilBERT are trained on image, table, and text datasets, respectively.**
| Modality | Image | Tabular | Text | Avg. |  |  |  |  |  |  |  |  |  |
| --- | --- | --- | --- | --- | --- | --- | --- | --- | --- | --- | --- | --- | --- |
| Dataset | CIFAR-10 | CIFAR-100 | BloodMNIST | CelebA | Purchase | Texas | Adult | Student | Rotten | Tweet | CoLA | ECtHR |  |
| Metric-loss | 0.635 | 0.858 | 0.527 | 0.509 | 0.619 | 0.629 | 0.500 | 0.566 | 0.582 | 0.566 | 0.571 | 0.521 | 0.590 |
| ($\pm$ 0.053) | ($\pm$ 0.004) | ($\pm$ 0.018) | ($\pm$ 0.012) | ($\pm$ 0.003) | ($\pm$ 0.013) | ($\pm$ 0.003) | ($\pm$ 0.032) | ($\pm$ 0.007) | ($\pm$ 0.005) | ($\pm$ 0.003) | ($\pm$ 0.007) | ($\pm$ 0.094) |  |
| Metric-conf | 0.635 | 0.858 | 0.527 | 0.509 | 0.619 | 0.629 | 0.500 | 0.566 | 0.582 | 0.566 | 0.571 | 0.521 | 0.590 |
| ($\pm$ 0.053) | ($\pm$ 0.004) | ($\pm$ 0.018) | ($\pm$ 0.012) | ($\pm$ 0.003) | ($\pm$ 0.013) | ($\pm$ 0.003) | ($\pm$ 0.032) | ($\pm$ 0.007) | ($\pm$ 0.005) | ($\pm$ 0.003) | ($\pm$ 0.007) | ($\pm$ 0.094) |  |
| Metric-corr | 0.552 | 0.708 | 0.517 | 0.507 | 0.551 | 0.610 | 0.501 | 0.560 | 0.557 | 0.550 | 0.550 | 0.519 | 0.557 |
| ($\pm$ 0.009) | ($\pm$ 0.002) | ($\pm$ 0.006) | ($\pm$ 0.002) | ($\pm$ 0.001) | ($\pm$ 0.012) | ($\pm$ 0.002) | ($\pm$ 0.022) | ($\pm$ 0.006) | ($\pm$ 0.005) | ($\pm$ 0.007) | ($\pm$ 0.005) | ($\pm$ 0.055) |  |
| Metric-ent | 0.628 | 0.848 | 0.525 | 0.508 | 0.616 | 0.563 | 0.498 | 0.520 | 0.561 | 0.528 | 0.519 | 0.507 | 0.568 |
| ($\pm$ 0.058) | ($\pm$ 0.004) | ($\pm$ 0.018) | ($\pm$ 0.010) | ($\pm$ 0.003) | ($\pm$ 0.010) | ($\pm$ 0.003) | ($\pm$ 0.018) | ($\pm$ 0.007) | ($\pm$ 0.005) | ($\pm$ 0.003) | ($\pm$ 0.016) | ($\pm$ 0.096) |  |
| Metric-ment | 0.635 | 0.858 | 0.527 | 0.509 | 0.620 | 0.630 | 0.500 | 0.566 | 0.582 | 0.566 | 0.571 | 0.522 | 0.591 |
| ($\pm$ 0.053) | ($\pm$ 0.004) | ($\pm$ 0.018) | ($\pm$ 0.012) | ($\pm$ 0.003) | ($\pm$ 0.013) | ($\pm$ 0.003) | ($\pm$ 0.031) | ($\pm$ 0.007) | ($\pm$ 0.005) | ($\pm$ 0.003) | ($\pm$ 0.007) | ($\pm$ 0.094) |  |
| Learn-original | 0.631 | 0.870 | 0.508 | 0.503 | 0.652 | 0.597 | 0.502 | 0.531 | 0.558 | 0.529 | 0.568 | 0.506 | 0.580 |
| ($\pm$ 0.064) | ($\pm$ 0.003) | ($\pm$ 0.010) | ($\pm$ 0.007) | ($\pm$ 0.002) | ($\pm$ 0.011) | ($\pm$ 0.005) | ($\pm$ 0.024) | ($\pm$ 0.009) | ($\pm$ 0.003) | ($\pm$ 0.009) | ($\pm$ 0.007) | ($\pm$ 0.103) |  |
| Learn-top3 | 0.628 | 0.851 | 0.526 | 0.503 | 0.677 | 0.573 | 0.500 | 0.520 | 0.561 | 0.528 | 0.531 | 0.502 | 0.575 |
| ($\pm$ 0.057) | ($\pm$ 0.004) | ($\pm$ 0.016) | ($\pm$ 0.002) | ($\pm$ 0.003) | ($\pm$ 0.012) | ($\pm$ 0.004) | ($\pm$ 0.018) | ($\pm$ 0.007) | ($\pm$ 0.005) | ($\pm$ 0.020) | ($\pm$ 0.015) | ($\pm$ 0.100) |  |
| Learn-sorted | 0.628 | 0.850 | 0.529 | 0.508 | 0.666 | 0.573 | 0.501 | 0.520 | 0.561 | 0.528 | 0.510 | 0.501 | 0.573 |
| ($\pm$ 0.057) | ($\pm$ 0.004) | ($\pm$ 0.016) | ($\pm$ 0.010) | ($\pm$ 0.028) | ($\pm$ 0.011) | ($\pm$ 0.004) | ($\pm$ 0.018) | ($\pm$ 0.007) | ($\pm$ 0.005) | ($\pm$ 0.022) | ($\pm$ 0.016) | ($\pm$ 0.100) |  |
| Learn-label | 0.633 | 0.882 | 0.515 | 0.507 | 0.656 | 0.669 | 0.503 | 0.590 | 0.584 | 0.570 | 0.622 | 0.517 | 0.604 |
| ($\pm$ 0.056) | ($\pm$ 0.005) | ($\pm$ 0.011) | ($\pm$ 0.007) | ($\pm$ 0.005) | ($\pm$ 0.016) | ($\pm$ 0.003) | ($\pm$ 0.042) | ($\pm$ 0.009) | ($\pm$ 0.006) | ($\pm$ 0.010) | ($\pm$ 0.010) | ($\pm$ 0.104) |  |
| Learn-merge | 0.656 | 0.893 | 0.523 | 0.507 | 0.684 | 0.686 | 0.502 | 0.595 | 0.584 | 0.569 | 0.620 | 0.530 | 0.612 |
| ($\pm$ 0.065) | ($\pm$ 0.004) | ($\pm$ 0.017) | ($\pm$ 0.001) | ($\pm$ 0.003) | ($\pm$ 0.017) | ($\pm$ 0.002) | ($\pm$ 0.040) | ($\pm$ 0.009) | ($\pm$ 0.005) | ($\pm$ 0.010) | ($\pm$ 0.004) | ($\pm$ 0.108) |  |
| Model-loss | 0.664 | 0.852 | 0.560 | 0.522 | 0.725 | 0.767 | 0.509 | 0.670 | 0.773 | 0.756 | 0.752 | 0.655 | 0.684 |
| ($\pm$ 0.050) | ($\pm$ 0.004) | ($\pm$ 0.017) | ($\pm$ 0.004) | ($\pm$ 0.002) | ($\pm$ 0.011) | ($\pm$ 0.006) | ($\pm$ 0.068) | ($\pm$ 0.020) | ($\pm$ 0.010) | ($\pm$ 0.017) | ($\pm$ 0.012) | ($\pm$ 0.107) |  |
| Model-calibration | 0.639 | 0.763 | 0.553 | 0.520 | 0.684 | 0.718 | 0.508 | 0.648 | 0.695 | 0.714 | 0.699 | 0.638 | 0.648 |
| ($\pm$ 0.040) | ($\pm$ 0.005) | ($\pm$ 0.016) | ($\pm$ 0.004) | ($\pm$ 0.002) | ($\pm$ 0.011) | ($\pm$ 0.006) | ($\pm$ 0.063) | ($\pm$ 0.012) | ($\pm$ 0.006) | ($\pm$ 0.014) | ($\pm$ 0.011) | ($\pm$ 0.082) |  |
| Model-lira | 0.690 | 0.937 | 0.536 | 0.512 | 0.755 | 0.753 | 0.503 | 0.634 | 0.753 | 0.728 | 0.737 | 0.604 | 0.679 |
| ($\pm$ 0.085) | ($\pm$ 0.002) | ($\pm$ 0.009) | ($\pm$ 0.002) | ($\pm$ 0.003) | ($\pm$ 0.007) | ($\pm$ 0.002) | ($\pm$ 0.063) | ($\pm$ 0.024) | ($\pm$ 0.007) | ($\pm$ 0.014) | ($\pm$ 0.014) | ($\pm$ 0.126) |  |
| Model-fpr | 0.647 | 0.852 | 0.552 | 0.516 | 0.697 | 0.723 | 0.507 | 0.641 | 0.679 | 0.722 | 0.708 | 0.635 | 0.657 |
| ($\pm$ 0.056) | ($\pm$ 0.002) | ($\pm$ 0.020) | ($\pm$ 0.007) | ($\pm$ 0.004) | ($\pm$ 0.015) | ($\pm$ 0.005) | ($\pm$ 0.073) | ($\pm$ 0.041) | ($\pm$ 0.008) | ($\pm$ 0.029) | ($\pm$ 0.011) | ($\pm$ 0.099) |  |
| Model-robust | 0.635 | 0.889 | 0.552 | 0.520 | 0.711 | 0.762 | 0.509 | 0.669 | 0.766 | 0.745 | 0.746 | 0.621 | 0.677 |
| ($\pm$ 0.030) | ($\pm$ 0.004) | ($\pm$ 0.016) | ($\pm$ 0.003) | ($\pm$ 0.002) | ($\pm$ 0.017) | ($\pm$ 0.006) | ($\pm$ 0.061) | ($\pm$ 0.022) | ($\pm$ 0.008) | ($\pm$ 0.014) | ($\pm$ 0.013) | ($\pm$ 0.112) |  |
| Query-augment | 0.573 | 0.761 | 0.517 | 0.502 | 0.612 | 0.612 | 0.500 | 0.560 | 0.570 | 0.551 | 0.561 | 0.518 | 0.570 |
| ($\pm$ 0.025) | ($\pm$ 0.010) | ($\pm$ 0.008) | ($\pm$ 0.002) | ($\pm$ 0.001) | ($\pm$ 0.011) | ($\pm$ 0.002) | ($\pm$ 0.022) | ($\pm$ 0.007) | ($\pm$ 0.006) | ($\pm$ 0.015) | ($\pm$ 0.006) | ($\pm$ 0.070) |  |
| Query-transfer | 0.522 | 0.622 | 0.503 | 0.502 | 0.529 | 0.581 | 0.499 | 0.522 | 0.530 | 0.530 | 0.526 | 0.510 | 0.531 |
| ($\pm$ 0.008) | ($\pm$ 0.028) | ($\pm$ 0.010) | ($\pm$ 0.003) | ($\pm$ 0.004) | ($\pm$ 0.011) | ($\pm$ 0.003) | ($\pm$ 0.012) | ($\pm$ 0.011) | ($\pm$ 0.008) | ($\pm$ 0.005) | ($\pm$ 0.006) | ($\pm$ 0.036) |  |
| Query-adv | 0.615 | 0.838 | 0.508 | 0.514 | 0.620 | 0.579 | 0.500 | 0.563 | 0.571 | 0.551 | 0.568 | 0.519 | 0.579 |
| ($\pm$ 0.038) | ($\pm$ 0.015) | ($\pm$ 0.008) | ($\pm$ 0.007) | ($\pm$ 0.003) | ($\pm$ 0.008) | ($\pm$ 0.003) | ($\pm$ 0.024) | ($\pm$ 0.007) | ($\pm$ 0.006) | ($\pm$ 0.014) | ($\pm$ 0.005) | ($\pm$ 0.089) |  |
| Query-neighbor | 0.511 | 0.553 | 0.497 | 0.501 | 0.533 | 0.612 | 0.500 | 0.535 | 0.533 | 0.556 | 0.550 | 0.522 | 0.534 |
| ($\pm$ 0.003) | ($\pm$ 0.006) | ($\pm$ 0.004) | ($\pm$ 0.004) | ($\pm$ 0.001) | ($\pm$ 0.007) | ($\pm$ 0.005) | ($\pm$ 0.015) | ($\pm$ 0.004) | ($\pm$ 0.005) | ($\pm$ 0.010) | ($\pm$ 0.014) | ($\pm$ 0.032) |  |
| Query-qrm | 0.532 | 0.574 | 0.510 | 0.505 | 0.523 | 0.530 | 0.500 | 0.526 | 0.524 | 0.521 | 0.511 | 0.512 | 0.522 |
| ($\pm$ 0.072) | ($\pm$ 0.163) | ($\pm$ 0.017) | ($\pm$ 0.012) | ($\pm$ 0.057) | ($\pm$ 0.080) | ($\pm$ 0.003) | ($\pm$ 0.048) | ($\pm$ 0.038) | ($\pm$ 0.028) | ($\pm$ 0.031) | ($\pm$ 0.009) | ($\pm$ 0.060) |  |
| Query-ref | 0.735 | 0.941 | 0.566 | 0.526 | N/A | N/A | N/A | N/A | N/A | N/A | N/A | N/A | 0.692 |
| ($\pm$ 0.108) | ($\pm$ 0.008) | ($\pm$ 0.022) | ($\pm$ 0.017) | N/A | N/A | N/A | N/A | N/A | N/A | N/A | N/A | ($\pm$ 0.176) |  |

**Table 12: Complete version of TDD performance across different target model architectures (corresponds to Table 5). MLP stands for Multilayer Perceptron, and LR stands for Logistic Regression.**
| Dataset | CIFAR10(Image) | Purchase(Tabular) | Rotten-tomatoes(Text) | Avg. |  |  |  |  |  |  |  |
| --- | --- | --- | --- | --- | --- | --- | --- | --- | --- | --- | --- |
| Target model | WRN28-2 | ResNet18 | VGG11 | MobileNet-v2 | MLP | CatBoost | LR | DistilBERT | RoBERTa | Flan-T5 |  |
| Metric-loss | 0.635 | 0.659 | 0.684 | 0.592 | 0.619 | 0.948 | 0.640 | 0.582 | 0.571 | 0.517 | 0.645 |
| ($\pm$ 0.053) | ($\pm$ 0.042) | ($\pm$ 0.004) | ($\pm$ 0.057) | ($\pm$ 0.003) | ($\pm$ 0.009) | ($\pm$ 0.003) | ($\pm$ 0.007) | ($\pm$ 0.019) | ($\pm$ 0.004) | ($\pm$ 0.115) |  |
| Metric-conf | 0.635 | 0.659 | 0.684 | 0.592 | 0.619 | 0.948 | 0.640 | 0.582 | 0.571 | 0.517 | 0.645 |
| ($\pm$ 0.053) | ($\pm$ 0.042) | ($\pm$ 0.004) | ($\pm$ 0.057) | ($\pm$ 0.003) | ($\pm$ 0.009) | ($\pm$ 0.003) | ($\pm$ 0.007) | ($\pm$ 0.019) | ($\pm$ 0.004) | ($\pm$ 0.115) |  |
| Metric-corr | 0.552 | 0.557 | 0.574 | 0.548 | 0.551 | 0.636 | 0.622 | 0.557 | 0.542 | 0.513 | 0.565 |
| ($\pm$ 0.009) | ($\pm$ 0.006) | ($\pm$ 0.003) | ($\pm$ 0.019) | ($\pm$ 0.001) | ($\pm$ 0.001) | ($\pm$ 0.002) | ($\pm$ 0.006) | ($\pm$ 0.014) | ($\pm$ 0.002) | ($\pm$ 0.036) |  |
| Metric-ent | 0.628 | 0.654 | 0.680 | 0.582 | 0.616 | 0.943 | 0.594 | 0.561 | 0.555 | 0.509 | 0.632 |
| ($\pm$ 0.058) | ($\pm$ 0.048) | ($\pm$ 0.004) | ($\pm$ 0.060) | ($\pm$ 0.003) | ($\pm$ 0.012) | ($\pm$ 0.003) | ($\pm$ 0.007) | ($\pm$ 0.015) | ($\pm$ 0.003) | ($\pm$ 0.118) |  |
| Metric-ment | 0.635 | 0.659 | 0.685 | 0.592 | 0.620 | 0.950 | 0.642 | 0.582 | 0.571 | 0.517 | 0.645 |
| ($\pm$ 0.053) | ($\pm$ 0.042) | ($\pm$ 0.004) | ($\pm$ 0.056) | ($\pm$ 0.003) | ($\pm$ 0.009) | ($\pm$ 0.003) | ($\pm$ 0.007) | ($\pm$ 0.019) | ($\pm$ 0.004) | ($\pm$ 0.116) |  |
| Learn-original | 0.631 | 0.623 | 0.694 | 0.533 | 0.652 | 0.935 | 0.644 | 0.558 | 0.546 | 0.515 | 0.633 |
| ($\pm$ 0.064) | ($\pm$ 0.031) | ($\pm$ 0.004) | ($\pm$ 0.017) | ($\pm$ 0.002) | ($\pm$ 0.092) | ($\pm$ 0.008) | ($\pm$ 0.009) | ($\pm$ 0.015) | ($\pm$ 0.007) | ($\pm$ 0.121) |  |
| Learn-top3 | 0.628 | 0.653 | 0.680 | 0.582 | 0.677 | 0.967 | 0.660 | 0.561 | 0.555 | 0.509 | 0.647 |
| ($\pm$ 0.057) | ($\pm$ 0.047) | ($\pm$ 0.005) | ($\pm$ 0.059) | ($\pm$ 0.003) | ($\pm$ 0.019) | ($\pm$ 0.008) | ($\pm$ 0.007) | ($\pm$ 0.015) | ($\pm$ 0.003) | ($\pm$ 0.124) |  |
| Learn-sorted | 0.628 | 0.654 | 0.680 | 0.578 | 0.666 | 0.963 | 0.661 | 0.561 | 0.555 | 0.509 | 0.646 |
| ($\pm$ 0.057) | ($\pm$ 0.048) | ($\pm$ 0.004) | ($\pm$ 0.057) | ($\pm$ 0.028) | ($\pm$ 0.019) | ($\pm$ 0.008) | ($\pm$ 0.007) | ($\pm$ 0.015) | ($\pm$ 0.003) | ($\pm$ 0.124) |  |
| Learn-label | 0.633 | 0.612 | 0.707 | 0.557 | 0.656 | 0.954 | 0.701 | 0.584 | 0.565 | 0.520 | 0.649 |
| ($\pm$ 0.056) | ($\pm$ 0.011) | ($\pm$ 0.005) | ($\pm$ 0.025) | ($\pm$ 0.005) | ($\pm$ 0.044) | ($\pm$ 0.005) | ($\pm$ 0.009) | ($\pm$ 0.020) | ($\pm$ 0.007) | ($\pm$ 0.121) |  |
| Learn-merge | 0.656 | 0.628 | 0.727 | 0.528 | 0.684 | 0.968 | 0.716 | 0.584 | 0.566 | 0.518 | 0.657 |
| ($\pm$ 0.065) | ($\pm$ 0.017) | ($\pm$ 0.006) | ($\pm$ 0.005) | ($\pm$ 0.003) | ($\pm$ 0.024) | ($\pm$ 0.006) | ($\pm$ 0.009) | ($\pm$ 0.021) | ($\pm$ 0.005) | ($\pm$ 0.128) |  |
| Model-loss | 0.664 | 0.709 | 0.729 | 0.607 | 0.725 | 0.975 | 0.776 | 0.773 | 0.656 | 0.602 | 0.721 |
| ($\pm$ 0.050) | ($\pm$ 0.028) | ($\pm$ 0.005) | ($\pm$ 0.045) | ($\pm$ 0.002) | ($\pm$ 0.015) | ($\pm$ 0.002) | ($\pm$ 0.020) | ($\pm$ 0.034) | ($\pm$ 0.017) | ($\pm$ 0.106) |  |
| Model-calibration | 0.639 | 0.671 | 0.690 | 0.595 | 0.684 | 0.865 | 0.719 | 0.695 | 0.622 | 0.592 | 0.677 |
| ($\pm$ 0.040) | ($\pm$ 0.015) | ($\pm$ 0.005) | ($\pm$ 0.036) | ($\pm$ 0.002) | ($\pm$ 0.029) | ($\pm$ 0.002) | ($\pm$ 0.012) | ($\pm$ 0.028) | ($\pm$ 0.011) | ($\pm$ 0.078) |  |
| Model-lira | 0.690 | 0.749 | 0.780 | 0.601 | 0.755 | 0.995 | 0.761 | 0.753 | 0.630 | 0.569 | 0.728 |
| ($\pm$ 0.085) | ($\pm$ 0.066) | ($\pm$ 0.005) | ($\pm$ 0.053) | ($\pm$ 0.003) | ($\pm$ 0.003) | ($\pm$ 0.005) | ($\pm$ 0.024) | ($\pm$ 0.028) | ($\pm$ 0.009) | ($\pm$ 0.120) |  |
| Model-fpr | 0.647 | 0.684 | 0.712 | 0.619 | 0.697 | 0.976 | 0.724 | 0.679 | 0.623 | 0.589 | 0.695 |
| ($\pm$ 0.056) | ($\pm$ 0.033) | ($\pm$ 0.006) | ($\pm$ 0.071) | ($\pm$ 0.004) | ($\pm$ 0.013) | ($\pm$ 0.003) | ($\pm$ 0.041) | ($\pm$ 0.059) | ($\pm$ 0.008) | ($\pm$ 0.109) |  |
| Model-robust | 0.635 | 0.677 | 0.704 | 0.602 | 0.711 | 0.983 | 0.796 | 0.766 | 0.639 | 0.574 | 0.709 |
| ($\pm$ 0.030) | ($\pm$ 0.011) | ($\pm$ 0.006) | ($\pm$ 0.041) | ($\pm$ 0.002) | ($\pm$ 0.011) | ($\pm$ 0.003) | ($\pm$ 0.022) | ($\pm$ 0.025) | ($\pm$ 0.017) | ($\pm$ 0.115) |  |
| Query-augment | 0.573 | 0.575 | 0.633 | 0.542 | 0.612 | 0.696 | 0.664 | 0.570 | 0.546 | 0.512 | 0.592 |
| ($\pm$ 0.025) | ($\pm$ 0.009) | ($\pm$ 0.005) | ($\pm$ 0.024) | ($\pm$ 0.001) | ($\pm$ 0.002) | ($\pm$ 0.004) | ($\pm$ 0.007) | ($\pm$ 0.020) | ($\pm$ 0.009) | ($\pm$ 0.057) |  |
| Query-transfer | 0.522 | 0.522 | 0.533 | 0.507 | 0.529 | 0.587 | 0.574 | 0.530 | 0.515 | 0.506 | 0.533 |
| ($\pm$ 0.008) | ($\pm$ 0.007) | ($\pm$ 0.014) | ($\pm$ 0.003) | ($\pm$ 0.004) | ($\pm$ 0.001) | ($\pm$ 0.003) | ($\pm$ 0.011) | ($\pm$ 0.011) | ($\pm$ 0.004) | ($\pm$ 0.027) |  |
| Query-adv | 0.615 | 0.621 | 0.666 | 0.583 | 0.620 | 0.727 | 0.662 | 0.571 | 0.552 | 0.516 | 0.613 |
| ($\pm$ 0.038) | ($\pm$ 0.026) | ($\pm$ 0.031) | ($\pm$ 0.042) | ($\pm$ 0.003) | ($\pm$ 0.002) | ($\pm$ 0.005) | ($\pm$ 0.007) | ($\pm$ 0.020) | ($\pm$ 0.007) | ($\pm$ 0.063) |  |
| Query-neighbor | 0.511 | 0.512 | 0.509 | 0.509 | 0.533 | 0.820 | 0.530 | 0.533 | 0.527 | 0.504 | 0.549 |
| ($\pm$ 0.003) | ($\pm$ 0.004) | ($\pm$ 0.004) | ($\pm$ 0.003) | ($\pm$ 0.001) | ($\pm$ 0.015) | ($\pm$ 0.002) | ($\pm$ 0.004) | ($\pm$ 0.004) | ($\pm$ 0.005) | ($\pm$ 0.092) |  |
| Query-qrm | 0.532 | 0.537 | 0.541 | 0.530 | 0.523 | 0.946 | 0.632 | 0.524 | 0.528 | 0.506 | 0.580 |
| ($\pm$ 0.072) | ($\pm$ 0.083) | ($\pm$ 0.088) | ($\pm$ 0.060) | ($\pm$ 0.057) | ($\pm$ 0.007) | ($\pm$ 0.004) | ($\pm$ 0.038) | ($\pm$ 0.035) | ($\pm$ 0.007) | ($\pm$ 0.136) |  |
| Query-ref | 0.735 | 0.800 | 0.843 | 0.656 | N/A | N/A | N/A | N/A | N/A | N/A | 0.759 |
| ($\pm$ 0.108) | ($\pm$ 0.085) | ($\pm$ 0.013) | ($\pm$ 0.103) | N/A | N/A | N/A | N/A | N/A | N/A | ($\pm$ 0.107) |  |

**Table 13: Complete version of TDD performance across various shadow and reference models (corresponds to Table 6). MLP stands for Multilayer Perceptron, and LR stands for Logistic Regression.**
| Target model | WRN28-2(CIFAR-10) | MLP(Purchase) | DistilBERT(Rotten-tomatoes) | Avg. |  |  |  |  |  |  |  |
| --- | --- | --- | --- | --- | --- | --- | --- | --- | --- | --- | --- |
| Shadow/reference model | WRN28-2 | ResNet18 | VGG11 | MobileNet-v2 | MLP | CatBoost | LR | DistilBERT | RoBERTa | Flan-T5 |  |
| Learn-original | 0.631 | 0.578 | 0.632 | 0.539 | 0.652 | 0.651 | 0.564 | 0.558 | 0.560 | 0.546 | 0.591 |
| ($\pm$ 0.064) | ($\pm$ 0.030) | ($\pm$ 0.061) | ($\pm$ 0.015) | ($\pm$ 0.002) | ($\pm$ 0.005) | ($\pm$ 0.008) | ($\pm$ 0.009) | ($\pm$ 0.007) | ($\pm$ 0.006) | ($\pm$ 0.051) |  |
| Learn-top3 | 0.628 | 0.628 | 0.628 | 0.628 | 0.677 | 0.646 | 0.515 | 0.561 | 0.561 | 0.561 | 0.603 |
| ($\pm$ 0.057) | ($\pm$ 0.057) | ($\pm$ 0.057) | ($\pm$ 0.057) | ($\pm$ 0.003) | ($\pm$ 0.023) | ($\pm$ 0.007) | ($\pm$ 0.007) | ($\pm$ 0.007) | ($\pm$ 0.007) | ($\pm$ 0.059) |  |
| Learn-sorted | 0.628 | 0.629 | 0.628 | 0.629 | 0.666 | 0.656 | 0.532 | 0.561 | 0.561 | 0.561 | 0.605 |
| ($\pm$ 0.057) | ($\pm$ 0.058) | ($\pm$ 0.057) | ($\pm$ 0.058) | ($\pm$ 0.028) | ($\pm$ 0.019) | ($\pm$ 0.048) | ($\pm$ 0.007) | ($\pm$ 0.007) | ($\pm$ 0.007) | ($\pm$ 0.058) |  |
| Learn-label | 0.633 | 0.591 | 0.644 | 0.563 | 0.656 | 0.651 | 0.551 | 0.584 | 0.584 | 0.580 | 0.604 |
| ($\pm$ 0.056) | ($\pm$ 0.024) | ($\pm$ 0.060) | ($\pm$ 0.014) | ($\pm$ 0.005) | ($\pm$ 0.005) | ($\pm$ 0.009) | ($\pm$ 0.009) | ($\pm$ 0.009) | ($\pm$ 0.007) | ($\pm$ 0.045) |  |
| Learn-merge | 0.656 | 0.581 | 0.651 | 0.509 | 0.684 | 0.517 | 0.595 | 0.584 | 0.584 | 0.580 | 0.594 |
| ($\pm$ 0.065) | ($\pm$ 0.022) | ($\pm$ 0.063) | ($\pm$ 0.017) | ($\pm$ 0.003) | ($\pm$ 0.019) | ($\pm$ 0.025) | ($\pm$ 0.009) | ($\pm$ 0.009) | ($\pm$ 0.008) | ($\pm$ 0.061) |  |
| Model-loss | 0.664 | 0.657 | 0.641 | 0.632 | 0.725 | 0.608 | 0.611 | 0.773 | 0.607 | 0.589 | 0.651 |
| ($\pm$ 0.050) | ($\pm$ 0.045) | ($\pm$ 0.039) | ($\pm$ 0.025) | ($\pm$ 0.002) | ($\pm$ 0.001) | ($\pm$ 0.002) | ($\pm$ 0.020) | ($\pm$ 0.008) | ($\pm$ 0.008) | ($\pm$ 0.061) |  |
| Model-calibration | 0.639 | 0.634 | 0.617 | 0.614 | 0.684 | 0.579 | 0.588 | 0.695 | 0.595 | 0.587 | 0.623 |
| ($\pm$ 0.040) | ($\pm$ 0.033) | ($\pm$ 0.032) | ($\pm$ 0.019) | ($\pm$ 0.002) | ($\pm$ 0.001) | ($\pm$ 0.002) | ($\pm$ 0.012) | ($\pm$ 0.007) | ($\pm$ 0.007) | ($\pm$ 0.043) |  |
| Model-lira | 0.690 | 0.659 | 0.666 | 0.610 | 0.755 | 0.686 | 0.588 | 0.753 | 0.602 | 0.553 | 0.656 |
| ($\pm$ 0.085) | ($\pm$ 0.064) | ($\pm$ 0.067) | ($\pm$ 0.025) | ($\pm$ 0.003) | ($\pm$ 0.003) | ($\pm$ 0.002) | ($\pm$ 0.024) | ($\pm$ 0.009) | ($\pm$ 0.010) | ($\pm$ 0.075) |  |
| Model-fpr | 0.647 | 0.668 | 0.638 | 0.664 | 0.697 | 0.645 | 0.643 | 0.679 | 0.557 | 0.567 | 0.641 |
| ($\pm$ 0.056) | ($\pm$ 0.061) | ($\pm$ 0.053) | ($\pm$ 0.056) | ($\pm$ 0.004) | ($\pm$ 0.003) | ($\pm$ 0.003) | ($\pm$ 0.041) | ($\pm$ 0.007) | ($\pm$ 0.008) | ($\pm$ 0.055) |  |
| Model-robust | 0.635 | 0.639 | 0.633 | 0.621 | 0.711 | 0.632 | 0.625 | 0.766 | 0.624 | 0.591 | 0.648 |
| ($\pm$ 0.030) | ($\pm$ 0.036) | ($\pm$ 0.034) | ($\pm$ 0.022) | ($\pm$ 0.002) | ($\pm$ 0.001) | ($\pm$ 0.002) | ($\pm$ 0.022) | ($\pm$ 0.010) | ($\pm$ 0.006) | ($\pm$ 0.053) |  |
| Query-augment | 0.573 | 0.555 | 0.575 | 0.552 | 0.612 | 0.612 | 0.612 | 0.570 | 0.569 | 0.565 | 0.580 |
| ($\pm$ 0.025) | ($\pm$ 0.019) | ($\pm$ 0.025) | ($\pm$ 0.016) | ($\pm$ 0.001) | ($\pm$ 0.002) | ($\pm$ 0.001) | ($\pm$ 0.007) | ($\pm$ 0.008) | ($\pm$ 0.011) | ($\pm$ 0.026) |  |
| Query-transfer | 0.522 | 0.529 | 0.518 | 0.518 | 0.529 | 0.535 | 0.529 | 0.530 | 0.529 | 0.514 | 0.525 |
| ($\pm$ 0.008) | ($\pm$ 0.018) | ($\pm$ 0.013) | ($\pm$ 0.017) | ($\pm$ 0.004) | ($\pm$ 0.001) | ($\pm$ 0.001) | ($\pm$ 0.011) | ($\pm$ 0.010) | ($\pm$ 0.006) | ($\pm$ 0.012) |  |
| Query-qrm | 0.532 | 0.532 | 0.533 | 0.532 | 0.523 | 0.625 | 0.622 | 0.524 | 0.524 | 0.528 | 0.548 |
| ($\pm$ 0.072) | ($\pm$ 0.072) | ($\pm$ 0.074) | ($\pm$ 0.070) | ($\pm$ 0.057) | ($\pm$ 0.003) | ($\pm$ 0.003) | ($\pm$ 0.038) | ($\pm$ 0.037) | ($\pm$ 0.039) | ($\pm$ 0.061) |  |
| Query-ref | 0.735 | 0.740 | 0.722 | 0.708 | N/A | N/A | N/A | N/A | N/A | N/A | 0.726 |
| ($\pm$ 0.108) | ($\pm$ 0.100) | ($\pm$ 0.093) | ($\pm$ 0.074) | N/A | N/A | N/A | N/A | N/A | N/A | ($\pm$ 0.088) |  |

### A.6 Performance under different metrics

**Table 14: TDD performance across different metrics on WRN28-2 trained on CIFAR-10 dataset. MA(membership advantage) (Jayaraman et al., ) equals the difference between the true positive rate and the false positive rate. For all metrics except for FPR and FNR, higher values indicate better performance of the corresponding TDD algorithm.**
| Algorithm | Precision | Recall | F1-score | Acc | FNR $\downarrow$ | FPR $\downarrow$ | MA | TPR@1%FPR | TPR@10%FPR | AUROC |
| --- | --- | --- | --- | --- | --- | --- | --- | --- | --- | --- |
| Metric-loss | 0.579 | 0.905 | 0.706 | 0.623 | 0.095 | 0.659 | 0.246 | 0.009 | 0.132 | 0.635 |
| ($\pm$ 0.030) | ($\pm$ 0.039) | ($\pm$ 0.033) | ($\pm$ 0.045) | ($\pm$ 0.039) | ($\pm$ 0.057) | ($\pm$ 0.091) | ($\pm$ 0.005) | ($\pm$ 0.014) | ($\pm$ 0.053) |  |
| Metric-conf | 0.579 | 0.905 | 0.706 | 0.623 | 0.095 | 0.659 | 0.246 | 0.009 | 0.131 | 0.635 |
| ($\pm$ 0.030) | ($\pm$ 0.039) | ($\pm$ 0.033) | ($\pm$ 0.045) | ($\pm$ 0.039) | ($\pm$ 0.057) | ($\pm$ 0.091) | ($\pm$ 0.005) | ($\pm$ 0.014) | ($\pm$ 0.053) |  |
| Metric-corr | 0.528 | 0.982 | 0.687 | 0.552 | 0.018 | 0.877 | 0.105 | 0.000 | 0.000 | 0.552 |
| ($\pm$ 0.004) | ($\pm$ 0.041) | ($\pm$ 0.013) | ($\pm$ 0.009) | ($\pm$ 0.041) | ($\pm$ 0.025) | ($\pm$ 0.018) | ($\pm$ 0.000) | ($\pm$ 0.000) | ($\pm$ 0.009) |  |
| Metric-ent | 0.574 | 0.878 | 0.694 | 0.614 | 0.122 | 0.649 | 0.229 | 0.012 | 0.133 | 0.628 |
| ($\pm$ 0.032) | ($\pm$ 0.073) | ($\pm$ 0.046) | ($\pm$ 0.050) | ($\pm$ 0.073) | ($\pm$ 0.028) | ($\pm$ 0.101) | ($\pm$ 0.001) | ($\pm$ 0.015) | ($\pm$ 0.058) |  |
| Metric-ment | 0.579 | 0.901 | 0.705 | 0.624 | 0.099 | 0.654 | 0.247 | 0.010 | 0.133 | 0.635 |
| ($\pm$ 0.029) | ($\pm$ 0.044) | ($\pm$ 0.035) | ($\pm$ 0.046) | ($\pm$ 0.044) | ($\pm$ 0.050) | ($\pm$ 0.091) | ($\pm$ 0.005) | ($\pm$ 0.015) | ($\pm$ 0.053) |  |
| Learn-original | 0.570 | 0.872 | 0.688 | 0.611 | 0.128 | 0.650 | 0.222 | 0.015 | 0.162 | 0.631 |
| ($\pm$ 0.031) | ($\pm$ 0.128) | ($\pm$ 0.065) | ($\pm$ 0.051) | ($\pm$ 0.128) | ($\pm$ 0.031) | ($\pm$ 0.102) | ($\pm$ 0.005) | ($\pm$ 0.031) | ($\pm$ 0.064) |  |
| Learn-top3 | 0.576 | 0.877 | 0.695 | 0.616 | 0.123 | 0.645 | 0.232 | 0.012 | 0.130 | 0.628 |
| ($\pm$ 0.033) | ($\pm$ 0.071) | ($\pm$ 0.046) | ($\pm$ 0.051) | ($\pm$ 0.071) | ($\pm$ 0.033) | ($\pm$ 0.102) | ($\pm$ 0.001) | ($\pm$ 0.013) | ($\pm$ 0.057) |  |
| Learn-sorted | 0.575 | 0.875 | 0.694 | 0.615 | 0.125 | 0.645 | 0.230 | 0.012 | 0.132 | 0.628 |
| ($\pm$ 0.032) | ($\pm$ 0.070) | ($\pm$ 0.046) | ($\pm$ 0.050) | ($\pm$ 0.070) | ($\pm$ 0.032) | ($\pm$ 0.101) | ($\pm$ 0.001) | ($\pm$ 0.014) | ($\pm$ 0.057) |  |
| Learn-label | 0.573 | 0.925 | 0.708 | 0.618 | 0.075 | 0.689 | 0.237 | 0.015 | 0.156 | 0.633 |
| ($\pm$ 0.027) | ($\pm$ 0.061) | ($\pm$ 0.038) | ($\pm$ 0.045) | ($\pm$ 0.061) | ($\pm$ 0.036) | ($\pm$ 0.090) | ($\pm$ 0.005) | ($\pm$ 0.031) | ($\pm$ 0.056) |  |
| Learn-merge | 0.580 | 0.908 | 0.708 | 0.625 | 0.092 | 0.658 | 0.250 | 0.020 | 0.178 | 0.656 |
| ($\pm$ 0.030) | ($\pm$ 0.039) | ($\pm$ 0.034) | ($\pm$ 0.046) | ($\pm$ 0.039) | ($\pm$ 0.054) | ($\pm$ 0.093) | ($\pm$ 0.008) | ($\pm$ 0.041) | ($\pm$ 0.065) |  |
| Model-loss | 0.581 | 0.811 | 0.676 | 0.611 | 0.189 | 0.589 | 0.222 | 0.050 | 0.211 | 0.664 |
| ($\pm$ 0.026) | ($\pm$ 0.053) | ($\pm$ 0.022) | ($\pm$ 0.032) | ($\pm$ 0.053) | ($\pm$ 0.086) | ($\pm$ 0.065) | ($\pm$ 0.015) | ($\pm$ 0.027) | ($\pm$ 0.050) |  |
| Model-calibration | 0.566 | 0.835 | 0.674 | 0.597 | 0.165 | 0.641 | 0.193 | 0.040 | 0.173 | 0.639 |
| ($\pm$ 0.020) | ($\pm$ 0.040) | ($\pm$ 0.018) | ($\pm$ 0.027) | ($\pm$ 0.040) | ($\pm$ 0.069) | ($\pm$ 0.054) | ($\pm$ 0.011) | ($\pm$ 0.013) | ($\pm$ 0.040) |  |
| Model-lira | 0.591 | 0.810 | 0.682 | 0.622 | 0.190 | 0.567 | 0.243 | 0.120 | 0.300 | 0.690 |
| ($\pm$ 0.039) | ($\pm$ 0.036) | ($\pm$ 0.026) | ($\pm$ 0.049) | ($\pm$ 0.036) | ($\pm$ 0.108) | ($\pm$ 0.097) | ($\pm$ 0.059) | ($\pm$ 0.107) | ($\pm$ 0.085) |  |
| Model-fpr | 0.624 | 0.520 | 0.566 | 0.605 | 0.480 | 0.310 | 0.210 | 0.074 | 0.261 | 0.647 |
| ($\pm$ 0.042) | ($\pm$ 0.074) | ($\pm$ 0.060) | ($\pm$ 0.038) | ($\pm$ 0.074) | ($\pm$ 0.027) | ($\pm$ 0.076) | ($\pm$ 0.034) | ($\pm$ 0.064) | ($\pm$ 0.056) |  |
| Model-robust | 0.553 | 0.882 | 0.676 | 0.583 | 0.118 | 0.716 | 0.167 | 0.070 | 0.221 | 0.635 |
| ($\pm$ 0.015) | ($\pm$ 0.135) | ($\pm$ 0.039) | ($\pm$ 0.018) | ($\pm$ 0.135) | ($\pm$ 0.123) | ($\pm$ 0.036) | ($\pm$ 0.023) | ($\pm$ 0.032) | ($\pm$ 0.030) |  |
| Query-augment | 0.539 | 0.917 | 0.679 | 0.567 | 0.083 | 0.783 | 0.135 | 0.004 | 0.071 | 0.573 |
| ($\pm$ 0.013) | ($\pm$ 0.039) | ($\pm$ 0.019) | ($\pm$ 0.022) | ($\pm$ 0.039) | ($\pm$ 0.027) | ($\pm$ 0.045) | ($\pm$ 0.003) | ($\pm$ 0.046) | ($\pm$ 0.025) |  |
| Query-transfer | 0.519 | 0.943 | 0.670 | 0.535 | 0.057 | 0.873 | 0.070 | 0.006 | 0.101 | 0.522 |
| ($\pm$ 0.005) | ($\pm$ 0.033) | ($\pm$ 0.012) | ($\pm$ 0.008) | ($\pm$ 0.033) | ($\pm$ 0.019) | ($\pm$ 0.017) | ($\pm$ 0.003) | ($\pm$ 0.003) | ($\pm$ 0.008) |  |
| Query-adv | 0.583 | 0.916 | 0.712 | 0.631 | 0.084 | 0.654 | 0.261 | 0.002 | 0.084 | 0.615 |
| ($\pm$ 0.035) | ($\pm$ 0.027) | ($\pm$ 0.031) | ($\pm$ 0.042) | ($\pm$ 0.027) | ($\pm$ 0.074) | ($\pm$ 0.084) | ($\pm$ 0.004) | ($\pm$ 0.050) | ($\pm$ 0.038) |  |
| Query-neighbor | 0.514 | 0.417 | 0.452 | 0.510 | 0.583 | 0.396 | 0.021 | 0.000 | 0.074 | 0.511 |
| ($\pm$ 0.005) | ($\pm$ 0.115) | ($\pm$ 0.079) | ($\pm$ 0.001) | ($\pm$ 0.115) | ($\pm$ 0.116) | ($\pm$ 0.003) | ($\pm$ 0.000) | ($\pm$ 0.011) | ($\pm$ 0.003) |  |
| Query-qrm | 0.518 | 0.583 | 0.522 | 0.529 | 0.417 | 0.525 | 0.057 | 0.000 | 0.000 | 0.532 |
| ($\pm$ 0.037) | ($\pm$ 0.301) | ($\pm$ 0.150) | ($\pm$ 0.058) | ($\pm$ 0.301) | ($\pm$ 0.252) | ($\pm$ 0.116) | ($\pm$ 0.000) | ($\pm$ 0.000) | ($\pm$ 0.072) |  |
| Query-ref | 0.649 | 0.776 | 0.696 | 0.666 | 0.224 | 0.445 | 0.331 | 0.152 | 0.355 | 0.735 |
| ($\pm$ 0.075) | ($\pm$ 0.140) | ($\pm$ 0.050) | ($\pm$ 0.062) | ($\pm$ 0.140) | ($\pm$ 0.180) | ($\pm$ 0.123) | ($\pm$ 0.084) | ($\pm$ 0.152) | ($\pm$ 0.108) |  |

**Table 15: TDD performance across different metrics on MLP trained on Purchase dataset. MA(membership advantage) (Jayaraman et al., ) equals the difference between the true positive rate and the false positive rate. For all metrics except for FPR and FNR, higher values indicate better performance of the corresponding TDD algorithm.**
| Algorithm | Precision | Recall | F1-score | Acc | FNR $\downarrow$ | FPR $\downarrow$ | MA | TPR@1%FPR | TPR@10%FPR | AUROC |
| --- | --- | --- | --- | --- | --- | --- | --- | --- | --- | --- |
| Metric-loss | 0.587 | 0.956 | 0.727 | 0.642 | 0.044 | 0.672 | 0.283 | 0.010 | 0.108 | 0.619 |
| ($\pm$ 0.002) | ($\pm$ 0.006) | ($\pm$ 0.003) | ($\pm$ 0.003) | ($\pm$ 0.006) | ($\pm$ 0.005) | ($\pm$ 0.007) | ($\pm$ 0.000) | ($\pm$ 0.001) | ($\pm$ 0.003) |  |
| Metric-conf | 0.587 | 0.956 | 0.727 | 0.642 | 0.044 | 0.672 | 0.283 | 0.010 | 0.108 | 0.619 |
| ($\pm$ 0.002) | ($\pm$ 0.006) | ($\pm$ 0.003) | ($\pm$ 0.003) | ($\pm$ 0.006) | ($\pm$ 0.005) | ($\pm$ 0.007) | ($\pm$ 0.000) | ($\pm$ 0.001) | ($\pm$ 0.003) |  |
| Metric-corr | 0.527 | 1.000 | 0.690 | 0.551 | 0.000 | 0.897 | 0.103 | 0.000 | 0.000 | 0.551 |
| ($\pm$ 0.002) | ($\pm$ 0.000) | ($\pm$ 0.001) | ($\pm$ 0.001) | ($\pm$ 0.000) | ($\pm$ 0.002) | ($\pm$ 0.002) | ($\pm$ 0.000) | ($\pm$ 0.000) | ($\pm$ 0.001) |  |
| Metric-ent | 0.582 | 0.948 | 0.721 | 0.634 | 0.052 | 0.680 | 0.268 | 0.010 | 0.108 | 0.616 |
| ($\pm$ 0.002) | ($\pm$ 0.008) | ($\pm$ 0.004) | ($\pm$ 0.004) | ($\pm$ 0.008) | ($\pm$ 0.006) | ($\pm$ 0.007) | ($\pm$ 0.000) | ($\pm$ 0.001) | ($\pm$ 0.003) |  |
| Metric-ment | 0.587 | 0.962 | 0.729 | 0.643 | 0.038 | 0.676 | 0.286 | 0.010 | 0.108 | 0.620 |
| ($\pm$ 0.002) | ($\pm$ 0.005) | ($\pm$ 0.003) | ($\pm$ 0.003) | ($\pm$ 0.005) | ($\pm$ 0.008) | ($\pm$ 0.007) | ($\pm$ 0.000) | ($\pm$ 0.001) | ($\pm$ 0.003) |  |
| Learn-original | 0.582 | 0.956 | 0.724 | 0.635 | 0.044 | 0.686 | 0.270 | 0.016 | 0.151 | 0.652 |
| ($\pm$ 0.003) | ($\pm$ 0.010) | ($\pm$ 0.002) | ($\pm$ 0.002) | ($\pm$ 0.010) | ($\pm$ 0.011) | ($\pm$ 0.005) | ($\pm$ 0.002) | ($\pm$ 0.005) | ($\pm$ 0.002) |  |
| Learn-top3 | 0.585 | 0.956 | 0.726 | 0.639 | 0.044 | 0.678 | 0.279 | 0.018 | 0.175 | 0.677 |
| ($\pm$ 0.002) | ($\pm$ 0.006) | ($\pm$ 0.003) | ($\pm$ 0.003) | ($\pm$ 0.006) | ($\pm$ 0.005) | ($\pm$ 0.007) | ($\pm$ 0.001) | ($\pm$ 0.010) | ($\pm$ 0.003) |  |
| Learn-sorted | 0.586 | 0.949 | 0.725 | 0.640 | 0.051 | 0.670 | 0.280 | 0.018 | 0.167 | 0.666 |
| ($\pm$ 0.003) | ($\pm$ 0.007) | ($\pm$ 0.003) | ($\pm$ 0.004) | ($\pm$ 0.007) | ($\pm$ 0.007) | ($\pm$ 0.008) | ($\pm$ 0.004) | ($\pm$ 0.034) | ($\pm$ 0.028) |  |
| Learn-label | 0.585 | 0.965 | 0.728 | 0.640 | 0.035 | 0.685 | 0.280 | 0.016 | 0.153 | 0.656 |
| ($\pm$ 0.002) | ($\pm$ 0.003) | ($\pm$ 0.002) | ($\pm$ 0.003) | ($\pm$ 0.003) | ($\pm$ 0.006) | ($\pm$ 0.006) | ($\pm$ 0.000) | ($\pm$ 0.004) | ($\pm$ 0.005) |  |
| Learn-merge | 0.589 | 0.953 | 0.728 | 0.644 | 0.047 | 0.665 | 0.288 | 0.020 | 0.187 | 0.684 |
| ($\pm$ 0.002) | ($\pm$ 0.005) | ($\pm$ 0.002) | ($\pm$ 0.003) | ($\pm$ 0.005) | ($\pm$ 0.006) | ($\pm$ 0.006) | ($\pm$ 0.001) | ($\pm$ 0.003) | ($\pm$ 0.003) |  |
| Model-loss | 0.611 | 0.843 | 0.708 | 0.653 | 0.157 | 0.537 | 0.306 | 0.056 | 0.276 | 0.725 |
| ($\pm$ 0.006) | ($\pm$ 0.028) | ($\pm$ 0.006) | ($\pm$ 0.001) | ($\pm$ 0.028) | ($\pm$ 0.030) | ($\pm$ 0.003) | ($\pm$ 0.002) | ($\pm$ 0.003) | ($\pm$ 0.002) |  |
| Model-calibration | 0.590 | 0.842 | 0.693 | 0.628 | 0.158 | 0.586 | 0.256 | 0.040 | 0.206 | 0.684 |
| ($\pm$ 0.011) | ($\pm$ 0.063) | ($\pm$ 0.016) | ($\pm$ 0.002) | ($\pm$ 0.063) | ($\pm$ 0.065) | ($\pm$ 0.003) | ($\pm$ 0.002) | ($\pm$ 0.002) | ($\pm$ 0.002) |  |
| Model-lira | 0.614 | 0.913 | 0.734 | 0.670 | 0.087 | 0.573 | 0.340 | 0.134 | 0.378 | 0.755 |
| ($\pm$ 0.003) | ($\pm$ 0.022) | ($\pm$ 0.006) | ($\pm$ 0.002) | ($\pm$ 0.022) | ($\pm$ 0.021) | ($\pm$ 0.004) | ($\pm$ 0.009) | ($\pm$ 0.005) | ($\pm$ 0.003) |  |
| Model-fpr | 0.636 | 0.658 | 0.646 | 0.640 | 0.342 | 0.377 | 0.281 | 0.073 | 0.296 | 0.697 |
| ($\pm$ 0.009) | ($\pm$ 0.035) | ($\pm$ 0.012) | ($\pm$ 0.002) | ($\pm$ 0.035) | ($\pm$ 0.032) | ($\pm$ 0.005) | ($\pm$ 0.008) | ($\pm$ 0.012) | ($\pm$ 0.004) |  |
| Model-robust | 0.599 | 0.839 | 0.697 | 0.638 | 0.161 | 0.564 | 0.275 | 0.094 | 0.289 | 0.711 |
| ($\pm$ 0.012) | ($\pm$ 0.074) | ($\pm$ 0.018) | ($\pm$ 0.002) | ($\pm$ 0.074) | ($\pm$ 0.074) | ($\pm$ 0.005) | ($\pm$ 0.003) | ($\pm$ 0.003) | ($\pm$ 0.002) |  |
| Query-augment | 0.563 | 0.963 | 0.711 | 0.609 | 0.037 | 0.745 | 0.218 | 0.000 | 0.000 | 0.612 |
| ($\pm$ 0.002) | ($\pm$ 0.002) | ($\pm$ 0.002) | ($\pm$ 0.001) | ($\pm$ 0.002) | ($\pm$ 0.003) | ($\pm$ 0.003) | ($\pm$ 0.000) | ($\pm$ 0.000) | ($\pm$ 0.001) |  |
| Query-transfer | 0.523 | 0.981 | 0.682 | 0.544 | 0.019 | 0.893 | 0.088 | 0.010 | 0.100 | 0.529 |
| ($\pm$ 0.003) | ($\pm$ 0.008) | ($\pm$ 0.004) | ($\pm$ 0.005) | ($\pm$ 0.008) | ($\pm$ 0.002) | ($\pm$ 0.009) | ($\pm$ 0.000) | ($\pm$ 0.002) | ($\pm$ 0.004) |  |
| Query-adv | 0.569 | 0.879 | 0.690 | 0.607 | 0.121 | 0.665 | 0.214 | 0.000 | 0.000 | 0.620 |
| ($\pm$ 0.005) | ($\pm$ 0.038) | ($\pm$ 0.008) | ($\pm$ 0.003) | ($\pm$ 0.038) | ($\pm$ 0.038) | ($\pm$ 0.006) | ($\pm$ 0.000) | ($\pm$ 0.000) | ($\pm$ 0.003) |  |
| Query-neighbor | 0.524 | 0.527 | 0.522 | 0.524 | 0.473 | 0.480 | 0.048 | 0.000 | 0.115 | 0.533 |
| ($\pm$ 0.006) | ($\pm$ 0.094) | ($\pm$ 0.046) | ($\pm$ 0.001) | ($\pm$ 0.094) | ($\pm$ 0.093) | ($\pm$ 0.002) | ($\pm$ 0.000) | ($\pm$ 0.002) | ($\pm$ 0.001) |  |
| Query-qrm | 0.516 | 0.313 | 0.282 | 0.529 | 0.687 | 0.255 | 0.058 | 0.000 | 0.000 | 0.523 |
| ($\pm$ 0.037) | ($\pm$ 0.421) | ($\pm$ 0.313) | ($\pm$ 0.064) | ($\pm$ 0.421) | ($\pm$ 0.313) | ($\pm$ 0.128) | ($\pm$ 0.000) | ($\pm$ 0.000) | ($\pm$ 0.057) |  |

**Table 16: TDD performance across different metrics on DistilBERT trained on Rotten-tomatoes dataset. MA(membership advantage) (Jayaraman et al., ) equals the difference between the true positive rate and the false positive rate. For all metrics except for FPR and FNR, higher values indicate better performance of the corresponding TDD algorithm.**
| Algorithm | Precision | Recall | F1-score | Acc | FNR $\downarrow$ | FPR $\downarrow$ | MA | TPR@1%FPR | TPR@10%FPR | AUROC |
| --- | --- | --- | --- | --- | --- | --- | --- | --- | --- | --- |
| Metric-loss | 0.549 | 0.828 | 0.660 | 0.578 | 0.172 | 0.672 | 0.156 | 0.011 | 0.121 | 0.582 |
| ($\pm$ 0.010) | ($\pm$ 0.031) | ($\pm$ 0.013) | ($\pm$ 0.009) | ($\pm$ 0.031) | ($\pm$ 0.017) | ($\pm$ 0.019) | ($\pm$ 0.003) | ($\pm$ 0.010) | ($\pm$ 0.007) |  |
| Metric-conf | 0.549 | 0.828 | 0.660 | 0.578 | 0.172 | 0.672 | 0.156 | 0.011 | 0.121 | 0.582 |
| ($\pm$ 0.010) | ($\pm$ 0.031) | ($\pm$ 0.013) | ($\pm$ 0.009) | ($\pm$ 0.031) | ($\pm$ 0.017) | ($\pm$ 0.019) | ($\pm$ 0.003) | ($\pm$ 0.010) | ($\pm$ 0.007) |  |
| Metric-corr | 0.529 | 0.947 | 0.678 | 0.557 | 0.053 | 0.833 | 0.113 | 0.000 | 0.000 | 0.557 |
| ($\pm$ 0.009) | ($\pm$ 0.008) | ($\pm$ 0.008) | ($\pm$ 0.006) | ($\pm$ 0.008) | ($\pm$ 0.006) | ($\pm$ 0.011) | ($\pm$ 0.000) | ($\pm$ 0.000) | ($\pm$ 0.006) |  |
| Metric-ent | 0.536 | 0.766 | 0.629 | 0.555 | 0.234 | 0.655 | 0.110 | 0.011 | 0.119 | 0.561 |
| ($\pm$ 0.009) | ($\pm$ 0.061) | ($\pm$ 0.020) | ($\pm$ 0.007) | ($\pm$ 0.061) | ($\pm$ 0.050) | ($\pm$ 0.013) | ($\pm$ 0.003) | ($\pm$ 0.010) | ($\pm$ 0.007) |  |
| Metric-ment | 0.549 | 0.828 | 0.660 | 0.578 | 0.172 | 0.672 | 0.156 | 0.011 | 0.121 | 0.582 |
| ($\pm$ 0.010) | ($\pm$ 0.031) | ($\pm$ 0.013) | ($\pm$ 0.009) | ($\pm$ 0.031) | ($\pm$ 0.017) | ($\pm$ 0.019) | ($\pm$ 0.003) | ($\pm$ 0.010) | ($\pm$ 0.007) |  |
| Learn-original | 0.533 | 0.780 | 0.633 | 0.552 | 0.220 | 0.675 | 0.105 | 0.012 | 0.120 | 0.558 |
| ($\pm$ 0.010) | ($\pm$ 0.053) | ($\pm$ 0.017) | ($\pm$ 0.009) | ($\pm$ 0.053) | ($\pm$ 0.050) | ($\pm$ 0.017) | ($\pm$ 0.003) | ($\pm$ 0.007) | ($\pm$ 0.009) |  |
| Learn-top3 | 0.536 | 0.766 | 0.629 | 0.555 | 0.234 | 0.655 | 0.110 | 0.011 | 0.119 | 0.561 |
| ($\pm$ 0.009) | ($\pm$ 0.061) | ($\pm$ 0.020) | ($\pm$ 0.007) | ($\pm$ 0.061) | ($\pm$ 0.050) | ($\pm$ 0.013) | ($\pm$ 0.003) | ($\pm$ 0.009) | ($\pm$ 0.007) |  |
| Learn-sorted | 0.536 | 0.766 | 0.629 | 0.555 | 0.234 | 0.655 | 0.110 | 0.011 | 0.119 | 0.561 |
| ($\pm$ 0.009) | ($\pm$ 0.061) | ($\pm$ 0.020) | ($\pm$ 0.007) | ($\pm$ 0.061) | ($\pm$ 0.050) | ($\pm$ 0.013) | ($\pm$ 0.003) | ($\pm$ 0.010) | ($\pm$ 0.007) |  |
| Learn-label | 0.546 | 0.866 | 0.670 | 0.578 | 0.134 | 0.711 | 0.155 | 0.011 | 0.122 | 0.584 |
| ($\pm$ 0.009) | ($\pm$ 0.023) | ($\pm$ 0.012) | ($\pm$ 0.009) | ($\pm$ 0.023) | ($\pm$ 0.019) | ($\pm$ 0.018) | ($\pm$ 0.003) | ($\pm$ 0.007) | ($\pm$ 0.009) |  |
| Learn-merge | 0.547 | 0.862 | 0.669 | 0.578 | 0.138 | 0.705 | 0.157 | 0.012 | 0.122 | 0.584 |
| ($\pm$ 0.010) | ($\pm$ 0.017) | ($\pm$ 0.011) | ($\pm$ 0.009) | ($\pm$ 0.017) | ($\pm$ 0.018) | ($\pm$ 0.019) | ($\pm$ 0.002) | ($\pm$ 0.009) | ($\pm$ 0.009) |  |
| Model-loss | 0.683 | 0.707 | 0.694 | 0.691 | 0.293 | 0.324 | 0.383 | 0.148 | 0.385 | 0.773 |
| ($\pm$ 0.011) | ($\pm$ 0.061) | ($\pm$ 0.025) | ($\pm$ 0.015) | ($\pm$ 0.061) | ($\pm$ 0.033) | ($\pm$ 0.030) | ($\pm$ 0.021) | ($\pm$ 0.034) | ($\pm$ 0.020) |  |
| Model-calibration | 0.606 | 0.777 | 0.680 | 0.639 | 0.223 | 0.500 | 0.277 | 0.106 | 0.234 | 0.695 |
| ($\pm$ 0.008) | ($\pm$ 0.045) | ($\pm$ 0.015) | ($\pm$ 0.006) | ($\pm$ 0.045) | ($\pm$ 0.041) | ($\pm$ 0.013) | ($\pm$ 0.011) | ($\pm$ 0.019) | ($\pm$ 0.012) |  |
| Model-lira | 0.631 | 0.813 | 0.710 | 0.671 | 0.187 | 0.471 | 0.342 | 0.183 | 0.374 | 0.753 |
| ($\pm$ 0.007) | ($\pm$ 0.043) | ($\pm$ 0.018) | ($\pm$ 0.017) | ($\pm$ 0.043) | ($\pm$ 0.032) | ($\pm$ 0.035) | ($\pm$ 0.026) | ($\pm$ 0.048) | ($\pm$ 0.024) |  |
| Model-fpr | 0.671 | 0.506 | 0.573 | 0.630 | 0.494 | 0.245 | 0.260 | 0.141 | 0.340 | 0.679 |
| ($\pm$ 0.006) | ($\pm$ 0.094) | ($\pm$ 0.062) | ($\pm$ 0.025) | ($\pm$ 0.094) | ($\pm$ 0.048) | ($\pm$ 0.049) | ($\pm$ 0.024) | ($\pm$ 0.043) | ($\pm$ 0.041) |  |
| Model-robust | 0.651 | 0.797 | 0.715 | 0.684 | 0.203 | 0.429 | 0.368 | 0.162 | 0.375 | 0.766 |
| ($\pm$ 0.031) | ($\pm$ 0.063) | ($\pm$ 0.008) | ($\pm$ 0.021) | ($\pm$ 0.063) | ($\pm$ 0.101) | ($\pm$ 0.042) | ($\pm$ 0.028) | ($\pm$ 0.027) | ($\pm$ 0.022) |  |
| Query-augment | 0.537 | 0.878 | 0.666 | 0.565 | 0.122 | 0.747 | 0.131 | 0.000 | 0.000 | 0.570 |
| ($\pm$ 0.008) | ($\pm$ 0.019) | ($\pm$ 0.003) | ($\pm$ 0.006) | ($\pm$ 0.019) | ($\pm$ 0.026) | ($\pm$ 0.012) | ($\pm$ 0.000) | ($\pm$ 0.000) | ($\pm$ 0.007) |  |
| Query-transfer | 0.525 | 0.933 | 0.672 | 0.549 | 0.067 | 0.835 | 0.098 | 0.008 | 0.090 | 0.530 |
| ($\pm$ 0.011) | ($\pm$ 0.009) | ($\pm$ 0.011) | ($\pm$ 0.007) | ($\pm$ 0.009) | ($\pm$ 0.007) | ($\pm$ 0.014) | ($\pm$ 0.005) | ($\pm$ 0.040) | ($\pm$ 0.011) |  |
| Query-adv | 0.537 | 0.881 | 0.667 | 0.566 | 0.119 | 0.750 | 0.131 | 0.000 | 0.000 | 0.571 |
| ($\pm$ 0.008) | ($\pm$ 0.031) | ($\pm$ 0.010) | ($\pm$ 0.005) | ($\pm$ 0.031) | ($\pm$ 0.040) | ($\pm$ 0.011) | ($\pm$ 0.000) | ($\pm$ 0.000) | ($\pm$ 0.007) |  |
| Query-neighbor | 0.525 | 0.909 | 0.665 | 0.548 | 0.091 | 0.813 | 0.096 | 0.011 | 0.091 | 0.533 |
| ($\pm$ 0.008) | ($\pm$ 0.021) | ($\pm$ 0.010) | ($\pm$ 0.004) | ($\pm$ 0.021) | ($\pm$ 0.018) | ($\pm$ 0.007) | ($\pm$ 0.003) | ($\pm$ 0.005) | ($\pm$ 0.004) |  |
| Query-qrm | 0.509 | 0.722 | 0.597 | 0.525 | 0.278 | 0.672 | 0.049 | 0.000 | 0.000 | 0.524 |
| ($\pm$ 0.023) | ($\pm$ 0.079) | ($\pm$ 0.042) | ($\pm$ 0.035) | ($\pm$ 0.079) | ($\pm$ 0.035) | ($\pm$ 0.070) | ($\pm$ 0.000) | ($\pm$ 0.000) | ($\pm$ 0.038) |  |

## Peer Reviews

confidence: 4
contribution: 2
initial_rating: 5
presentation: 3
questions: - in Table 4, the authors do not explain why performance on CIFAR-100 is noticeably higher than on other datasets
- in Table 5, there is no explanation for why the target model CatBoost outperforms others on tabular data
- in Figure 2, the results indicate that the train-test gap is higher, and the target model performs better at detecting the data used in training. The authors attribute this to overfitting of the training data, which supposedly results in high performance when the train-test gap is large. However, wouldn’t overfitting typically degrade performance since the test set is independent? 
- in Fig. 3(a), the authors state that residual connections in ResNet help alleviate excessive memorization issues, thus decreasing the detection method’s performance. This explanation is confusing.
- In Table 6, the authors claim that detection performance declines when the reference model has limited knowledge of the target model’s architecture. However, for the learn-top3 method, all types of reference models show similar performance without any decline. The explanation appears inconsistent.
- The paper also lacks necessary explanations of the TDD protocols, such as the training-test protocol when using a reference model, or how the query model operates.
- Why the evaluation omits Transformer-based models, especially for image datasets
- Why chose Pythia instead of LLaMa as the most widely used public LLM
soundness: 2
strengths: This work provides a comprehensive evaluation of training data detection (TDD). The evaluation spans multiple data modalities, including image, tabular, and text, and implements 21 state-of-the-art TDD algorithms, including large language models such as Pythia-12B. This is the largest benchmark for TDD to date, offering a valuable reference for future research on this problem.
summary: This work provides a comprehensive evaluation of training data detection (TDD). The evaluation spans multiple data modalities, including image, tabular, and text, and implements 21 state-of-the-art TDD algorithms, including large language models such as Pythia-12B. This is the largest benchmark for TDD to date, offering a valuable reference for future research on this problem.

Although this evaluation extensively covers multiple data modalities and four main types of TDD methods, the insights from the evaluation are limited. It lacks in-depth analysis of the results. For example, in Table 4, the authors do not explain why performance on CIFAR-100 is noticeably higher than on other datasets, and similarly in Table 5, there is no explanation for why the target model CatBoost outperforms others on tabular data. Some explanations of the results are also unclear. For instance, in Figure 2, the results indicate that the train-test gap is higher, and the target model performs better at detecting the data used in training. The authors attribute this to overfitting of the training data, which supposedly results in high performance when the train-test gap is large. However, wouldn’t overfitting typically degrade performance since the test set is independent? This explanation should be clarified. Additionally, in Fig. 3(a), the authors state that residual connections in ResNet help alleviate excessive memorization issues, thus decreasing the detection method’s performance. This explanation is also somewhat confusing. In Table 6, the authors claim that detection performance declines when the reference model has limited knowledge of the target model’s architecture. However, this is not always true; for the learn-top3 method, all types of reference models show similar performance without any decline. Thus, the explanation appears inconsistent.

The paper also lacks necessary explanations of the TDD protocols, such as the training-test protocol when using a reference model, or how the query model operates, which would make it easier for readers to follow.

The evaluation also omits Transformer-based models, which are widely used today. For image-based applications in particular, Transformer-based methods are among the most commonly used models, making it worthwhile to evaluate these models for TDD. For large language models, instead of focusing on the Pythia models, evaluating LLaMa, as the most widely used public LLM, would be highly relevant for TDD and aligns better with real-world scenarios where TDD is a concern.

In summary, this work presents an extensive evaluation across different datasets and multiple models, providing a complementary benchmark for future TDD research. However, the lack of in-depth analysis and necessary explanations of training-test protocols across TDD methods weakens the work’s overall quality. Additionally, the omission of evaluations on Transformer-based models and widely used public LLMs, such as the LLaMa models, limits the value of this work.
weaknesses: Although this evaluation extensively covers multiple data modalities and four main types of TDD methods, the insights from the evaluation are limited. It lacks in-depth analysis of the results. For example, in Table 4, the authors do not explain why performance on CIFAR-100 is noticeably higher than on other datasets, and similarly in Table 5, there is no explanation for why the target model CatBoost outperforms others on tabular data. Some explanations of the results are also unclear. For instance, in Figure 2, the results indicate that the train-test gap is higher, and the target model performs better at detecting the data used in training. The authors attribute this to overfitting of the training data, which supposedly results in high performance when the train-test gap is large. However, wouldn’t overfitting typically degrade performance since the test set is independent? This explanation should be clarified. Additionally, in Fig. 3(a), the authors state that residual connections in ResNet help alleviate excessive memorization issues, thus decreasing the detection method’s performance. This explanation is also somewhat confusing. In Table 6, the authors claim that detection performance declines when the reference model has limited knowledge of the target model’s architecture. However, this is not always true; for the learn-top3 method, all types of reference models show similar performance without any decline. Thus, the explanation appears inconsistent.

The paper also lacks necessary explanations of the TDD protocols, such as the training-test protocol when using a reference model, or how the query model operates, which would make it easier for readers to follow.

The evaluation also omits Transformer-based models, which are widely used today. For image-based applications in particular, Transformer-based methods are among the most commonly used models, making it worthwhile to evaluate these models for TDD. For large language models, instead of focusing on the Pythia models, evaluating LLaMa, as the most widely used public LLM, would be highly relevant for TDD and aligns better with real-world scenarios where TDD is a concern.

---

confidence: 3
contribution: 2
initial_rating: 5
presentation: 3
questions: Please refer to the above weaknesses
soundness: 3
strengths: 1. The paper analyzes the latest methods' limitations and strengths carefully. Particularly, the correlation between train-test performance gap and TDD performance is interesting.
summary: Motivated by the limitations of the previous benchmarks on training data detection (TDD), the paper proposes a new benchmark that covers three different modalities (tabular, text, and image) and includes evaluation of latest methods. The paper gives several insights on the recent TDD methods with respect to TDD.
weaknesses: 1. The proposed benchmark is not truly large-scale as it considers regular-sized datasets.

2. The proposed benchmark is not significantly different from the existing ones except that it covers different modalities. It would be more interesting if the paper does not follow data split rules from the existing benchmarks but explore something completely new. Another missing aspect is the size of data points that are evaluated under TDD.

3. The claim that TDD outperformance comes at a cost of computational expense is a bit controversial particularly due to Fig. 2(a); learning-based can achieve the best performance although it requires significantly less computation than the model-based ones.

Overall, it is valuable that the pepper gives an extensive amount of experimental results, but the paper is a bit lacking in novel aspects.

---

confidence: 3
contribution: 3
initial_rating: 6
presentation: 3
questions: If a model is initialized with ImageNet pretrain  weights and then optimized with a different dataset  A. When using an  image example from ImageNet dataset as the input of the TDD  method to  test a model, should the  outcome to  be Yes  or  No?
soundness: 3
strengths: The benchmark is comprehensive. The TDDBench covers multiple data modalities and includes a wide range of datasets, algorithms, and model architectures, making it one of the most inclusive TDD benchmarks available.

The writing and formatting are decent.
summary: The paper introduces TDDBench, a comprehensive benchmark designed for Training Data Detection (TDD), which is for determining if specific data was used to train a model. This can be used to address issues like data privacy, copyright verification, and model unlearning. TDDBench includes 13 datasets covering image, tabular, and text data and benchmarks 21 TDD methods across four detection paradigms. The findings in this paper highlight that model-based methods outperform others but require significantly more resources. The benchmark reveals that existing TDD methods struggle, particularly with text and tabular data, and emphasizes the need for further research to enhance TDD effectiveness.

-----

Have gone through the responses, and I appreciate the authors' efforts. I keep my ratings.
weaknesses: It will be great if the authors can propose a novel method to improve the performance based  on the analysis in their benchmark experiments.

It  lacks  an important discussion about the TDD and different training methods. There are supervised learning, self-supervised learning, unsupervised learning. Would  TDD method generalizable to different training methods?

---

confidence: 3
contribution: 3
initial_rating: 6
presentation: 3
questions: see above
soundness: 3
strengths: 1. The paper is written well and is easy to understand.

2. The studied problem is very important.

3. The benchmark is comprehensive
summary: The paper benchmarks 21 different TDD methods across four detection paradigms and evaluate their performance from five perspectives: average detection performance, best detection performance, memory consumption, and computational efficiency in both time and memory.
weaknesses: 1. It might be useful to include some discussion and experimental results on the defense of the training data detection algorithms

2. The authors can consider elaborating more on the takeaway messages from their experimentations and include more discussions on the current limations of the detection algorithms and the possible future directions at the end of the paper

3. It might be better to discuss the differences between the proposed benchmark with an existing benchmark. That will be useful.

[1] Ye et al., Data Contamination Calibration for Black-box LLMs

Meta-review: This paper was reviewed by 4 experts in the field and received 6, 5, 6, 6 as the ratings. The reviewers agreed that the paper presents a comprehensive evaluation of 21 training data detection (TDD) algorithms on multiple data modalities (image, text and tabular). The paper studies a problem of interest, it is well-written and easy to understand.  

Reviewer 64P6 raised a concern that the benchmark is not large-scale, as it considers regular sized datasets; in the rebuttal, the authors have evaluated the performance of the TDD algorithms on the ImageNet-1k and the FairJob datasets, both containing over 1 million samples. This reviewer also raised a concern about the size of the dataset that are evaluated under TDD; to address this, additional experiments were conducted by the authors in the rebuttal, by varying the size of the target dataset. While reviewer 64P6 has a slightly negative opinion about the paper, the AC feels that all his concerns were convincingly addressed by the authors in the rebuttal. The reviewer has also acknowledged the extensiveness of the experiments in the paper. 

Reviewer KoC1 mentioned that the evaluations do not include transformer based models; in response, the authors have conducted experiments on the ViT and Swin transformer models. A question was raised whether the TDD method will be generalizable to different training methods; as part of the response, the authors have conducted experiments on CIFAR-10 and CIFAR-100 to assess the suitability of TDD algorithms for semi-supervised and self-supervised learning, in addition to supervised learning. Concerns were also raised about the defense of the training data detection algorithms, explanations of the TDD protocols and current limitations of the detection algorithms, all of which were addressed convincingly by the authors. 

The reviewers, in general, have a positive opinion about the paper and its contributions. Most of the reviewers' concerns have been addressed convincingly by the authors, with supporting experimental results. Based on the reviewers' feedback, the decision is to recommend the paper for acceptance to ICLR 2025. The reviewers have provided some valuable comments, such as the novelty of the proposed research, further discussions on TDD algorithms tailored for specific training methods (particularly semi-supervised and self-supervised learning) and adaptive approaches to address the potential defense mechanisms of the target model. The authors are encouraged to address these in the final version of their paper. We congratulate the authors on the acceptance of their paper!

Decision: Accept (Poster)