Probing Language Models for Pre-training Data Detection (2024)

Abstract: Large Language Models (LLMs) have shown their impressive capabilities, while also raising concerns about the data contamination problems due to privacy issues and leakage of benchmark datasets in the pre-training phase. Therefore, it is vital to detect the contamination by checking whether an LLM has been pre-trained on the target texts. Recent studies focus on the generated texts and compute perplexities, which are superficial features and not reliable. In this study, we propose to utilize the probing technique for pre-training data detection by examining the model’s internal activations. Our method is simple yet effective and leads to more trustworthy pre-training data detection. Additionally, we propose ArxivMIA, a new challenging benchmark comprising arxiv abstracts from Computer Science and Mathematics categories. Our experiments demonstrate that our method outperforms all the baselines, and achieves state-of-the-art performance on both WikiMIA and ArxivMIA, with additional experiments confirming its efficacy1.

heading: 1 Introduction
text: Large language models (LLMs) trained on massive corpora of texts demonstrate extraordinary abilities to understand, reason, and generate following natural language instructions (Brown et al., 2020; Anil et al., 2023). Meanwhile, the open-source of LLMs has significantly contributed to the advancement and collaborative development within the LLM community (Zhang et al., 2022; Touvron et al., 2023b; Biderman et al., 2023; Bai et al., 2023; Team, 2023; LLaMA-MoE Team, 2023). Despite this progress, the lack of transparency raises ethical and legal questions, particularly about the use of potentially private data sourced from the internet, and threatens the reliability of benchmark evaluations
* Corresponding author 1Our code and dataset are available at https://github.
com/zhliu0106/probing-lm-data
due to the risk of leaking test data. Therefore, determining if certain texts have been utilized during the pre-training phase of the target LLM becomes a critical task.
Recent efforts to detect pre-training data in LLMs have attracted significant attention. Several studies have been proposed to investigate dataset contamination, including prompting LLMs to generate data-specific examples or using statistical methods to detect contamination in test sets (Sainz et al., 2023; Golchin and Surdeanu, 2023; Oren et al., 2023). Concurrently, Membership Inference Attacks (MIAs) in Natural Language Processing have been extensively explored for their potential to discern whether specific data was used in LLMs’ pre-training (Carlini et al., 2021; Mireshghallah et al., 2022; Mattern et al., 2023; Shi et al., 2023). The above solutions have achieved a certain success. However, they all rely on the model’s superficial features, such as generated texts or loss metrics, which may not always be reliable indicators of pretraining data.
Different from these conventional approaches, we propose a simple yet effective pre-training data detection method that utilizes the probing technique to examine the model’s internal activations. This approach is based on the assumption: Texts that have been seen during the model’s pre-training phase are represented differently in the model’s internal activations compared to texts that have not been seen, so we could train a linear probe classifier to distinguish them.
As illustrated in Figure 1, our method consists of three main steps: (1) We initiate our process by gathering a training dataset that the LLM has not previously been trained on, splitting the data into member and non-member subsets. We then inject the member data into the target model through a fine-tuning process on the member data alone. This step enables us to create a proxy model that retains the memory of the member data from the training
1576
dataset. (2) Subsequently, we input both member and non-member subsets from the training dataset into the proxy model and extract the model’s internal activations. These activations are employed to train a probe classifier that can distinguish between member and non-member data. (3) Finally, given a target text, we can input it to the target model, extract the internal activations, and let the probe classifier infer whether the text is member data. In other words, the probe classifier could assess whether the target text has been seen during the pre-training phase.
In order to evaluate various pre-training data detection methods in a more challenging scenario, we introduce ArxivMIA, a difficult benchmark in the academic domain. In contrast to the existing WikiMIA (Shi et al., 2023) benchmark, which primarily utilizes event data from Wikipedia pages, ArxivMIA presents a more challenging scenario. The academic abstracts within ArxivMIA are rarer on the internet compared to Wikipedia content, naturally resulting in a lower duplication rate. Furthermore, the inherent complexity of texts targeted at researchers adds another layer of difficulty for model memorization. This combination of rarity and complexity makes it exceedingly challenging for large models to memorize such content during the pre-training process, making its detection through conventional methods markedly tougher. Therefore, ArxivMIA stands as an especially rigorous benchmark, highlighting the need for more sophisticated pre-training data detection methods.
Our contributions can be summarized as follows:
• We propose a novel pre-training data detection method that utilizes the probing technique. To the best of our knowledge, this is the first work to examine LLMs’ internal activations to determine whether a given text was included in the pre-training data.
• We propose ArxivMIA, a new benchmark in the academic domain to assess pre-training data detection methods. With a low duplication rate and the inherent complexity of texts, ArxivMIA presents a more challenging scenario and highlights the need for more sophisticated pre-training data detection methods.
• We conduct extensive experiments on WikiMIA and ArxivMIA benchmarks. In addition, we also evaluate various detection methods on a downstream task datasets contamination challenge. Through comprehensive experimentation, we demonstrate that our proposed method outperforms all baselines, and achieves state-of-the-art performance.

heading: 2 Related Work
text: Related work involves membership inference attacks in NLP and dataset contamination.
Membership Inference Attacks in NLP. Membership Inference Attacks (MIAs) are designed to identify if a particular data sample was included in the training dataset of a machine learning model (Shokri et al., 2017; Yeom et al., 2018; Hu et al.,
2022b). Most MIAs take a black-box setting, assuming that the adversary only has access to the model confidence or loss scores (Yeom et al., 2018; Sablayrolles et al., 2019; Jayaraman et al., 2021; Watson et al., 2021). Unlike it, similar to Leino and Fredrikson (2020), we consider a white-box setting where the adversary has access to the model weights and activations. Specifically in NLP, a lot of studies have been proposed (Carlini et al., 2021; Mireshghallah et al., 2022; Mattern et al., 2023; Shi et al., 2023). Carlini et al. (2021) and Mireshghallah et al. (2022) separately investigated Likelihood Ratio Attacks for causal language models and masked language models. Mattern et al. (2023) proposed a neighbor attack that compares model loss for a given sample to losses of synthetically generated neighbor texts. Shi et al. (2023) measured the likelihood of outlier words in a given text, thereby assessing whether the text was likely part of a model’s pre-training corpora. Similar to Shi et al. (2023), we aim to detect pre-training data in LLMs. However, different from existing attacks that rely on the model’s superficial features, we focus on the LLMs’ internal activations, and the experiments show that our method outperforms existing attacks.
Dataset Contamination. The dataset contamination in LLMs has been widely studied since benchmark datasets are intentionally or unintentionally included in pre-training corpora. The n-gram based overlap comparison methods not only require access to training corpora but take a long time to compute (Gao et al., 2020; Brown et al., 2020; Dodge et al., 2021; Chowdhery et al., 2023; Anil et al., 2023; Touvron et al., 2023a,b). Without access to pre-training corpora, there are also some methods to detect dataset contamination. Sainz et al. (2023) prompted LLMs to generate verbatim examples of a dataset split. Golchin and Surdeanu (2023) proposed the ’Data Contamination Quiz’, which employs a multiple-choice format to assess a model’s ability to recognize original dataset instances among perturbed versions. Oren et al. (2023) presented a statistical test to demonstrate test set contamination in language models, leveraging the concept of exchangeability in benchmark datasets and comparing model log probabilities against shuffled dataset permutations.

heading: 3 Methodology

heading: 3.1 Overview
text: Different from conventional detection methods in MIA that rely on the model’s superficial features, we directly analyze the model’s internal activations, providing a deeper insight into its pre-training history. Our method employs the probe technique originally proposed by Alain and Bengio (2016). This technique hypothesizes that the internal representations of a model inherently contain specific encoded properties, so we could train a linear probe classifier with logistic regression for the detection of these properties. In our context, we are interested in determining whether a text sample was included in the model’s pre-training dataset. The framework of our method is illustrated in Figure 1.

heading: 3.2 Task Definition
text: The task of pre-training data detection follows a white-box setting of MIAs where the adversary has access to the model M’s architecture and weights, but not the pre-training data. The adversary aims to determine whether a sample s was included in the pre-training data of the model M with an attack method A: AM(s) → {0, 1}, where 1 represents member (seen) data, 0 denotes non-member (unseen) data. Usually, we have a scoring function f , then can get the confidence score f(s) ∈ [0, 1], which represents the probability of the sample being a part of the pre-training dataset. Then we can classify the sample as a member or non-member based on a threshold γ :
AM(s) = 1 [f(s) < γ]

heading: 3.3 Training Proxy Model
text: Training a probe classifier needs a dataset {⟨xi, yi⟩}, where xi represents the sample’s activation, and yi is a binary label indicating whether the sample is a member or non-member data. However, the absence of pre-training data makes it impossible to obtain the activations of the member or non-member samples. Therefore, we first gather a training dataset that the LLM has not previously been trained on, splitting the data into member and non-member subsets, and inject the member data into the proxy model to simulate data contamination, as detailed in subsection 4.2. The training dataset is D = {⟨si, yi⟩}, where si represents the sample, and yi is a binary label indicating whether the sample is a member or non-member.
Prompt Template for Sample. To standardize the input for consistency, each sample of D is processed using a prompt template. In the subsequent experiments, we use the following prompt template: "Here is a statement: [SAMPLE] \n Is the above statement correct? Answer:". Intuitively, we hope the last token of the sample can capture the information of it being member or non-member, and we believe the prompt template can help the target model to focus on this information. Further prompt template selection will be discussed in Appendix B.
Training Proxy Model. Next we need to inject the member samples Dmember = {si | ⟨si, yi⟩ ∈ D, yi = 1} into the model M to let it memorize the member data. This injection is accomplished by fine-tuning the model on Dmember. This step aims to make the model M memorize the member data, and the fine-tuning process is used to simulate the pre-training process. After this, we can get the proxy model M′, which retains the memory of Dmember. The proxy model M′ is then used to generate the member and non-member sample activations x for the dataset D.

heading: 3.4 Training Probe Classifier
text: The probe classifier takes the form Pθ(x) = σ(Wx), where σ denotes the sigmoid function and W represents the trainable weights. After obtaining the proxy model M′, we construct the training dataset with D for the probe.
In the paper, we focus on causal language models, which are trained to predict the next word given the previous words. In order to capture the representation of the sample, for each sample ⟨si, yi⟩ in D, we extract the activation xl from the final token of the input in layer l of the model M′. Finally, we get the dataset {⟨xli, yi⟩}, which is used to train the probe Pθ with logistic regression.

heading: 3.5 Pre-training Data Detection with Probe
text: Classifier
Given a benchmark, we already trained a probe Pθ, which can be used to detect whether a sample is included in the pre-training data. For a sample s, we preprocess it with the above prompt template, then feed it into the model M to get the activation xl. Finally, we can get the confidence score Pθ(xl), which represents the probability of the sample being a part of the pre-training dataset. This score is then utilized to classify the sample as a member or
non-member based on a threshold γ :
AM(s) = 1 [ Pθ(x l) < γ ]

heading: 4 Data Construction

heading: 4.1 ArxivMIA
text: To evaluate various pre-training data detection methods in a more challenging scenario, we introduce ArxivMIA, a new benchmark comprising abstracts from the fields of Computer Science (CS) and Mathematics (Math) sourced from Arxiv. In contrast to the existing WikiMIA (Shi et al., 2023) benchmark, which primarily utilizes event data from Wikipedia pages—thus prone to higher duplication rates in pre-training datasets—ArxivMIA presents a more challenging scenario.
For dataset construction, abstracts published post-2024 are designated as non-member data, while member data are derived from the Arxiv subset of the RedPajama dataset (Computer, 2023). The RedPajama dataset is the reproduction of the LLaMA (Touvron et al., 2023a) training dataset and is extensively utilized in pre-training LLMs (Zhang et al., 2024; Geng and Liu, 2023). This makes ArxivMIA particularly suited for testing LLMs pre-trained on the RedPajama dataset. Detailed information about ArxivMIA is presented in Table 1. In summary, ArxivMIA has three distinctive features: Firstly, it is a larger dataset with a total of 2000 samples. Secondly, it covers multiple fields, including CS and Math. Lastly, it features a longer average sentence length, with an average of 143.1 tokens per sample. These characteristics make ArxivMIA a more challenging benchmark for evaluating pre-training data detection methodologies.

heading: 4.2 Training Dataset Collection
text: Our method needs to construct a training dataset D = {⟨si, yi⟩} similar to the target benchmark, where si represents the sample, and yi is a binary label indicating whether the sample is a member or non-member. This dataset is pivotal for training the probe to accurately evaluate the likelihood of a sample being included in the pre-training data. However, the construction of a training dataset for the probe is challenging due to the lack of access to the pre-training data. To address this, we propose a heuristic method:
Data Collection. Firstly, we need to collect a dataset D = {si}, and make sure they are not
included in the pre-training data. There are two ways to accomplish it: (1). Real data: We collect the data published after the model release date. (2). Synthetic data: We can also use ChatGPT (Achiam et al., 2023) to synthesize similar data according to the data to be detected. The detailed process is described in Appendix A.
Dataset Split. Next, we randomly label half of the data in D as non-member data, and the other half as member data. Then we get the dataset D = {⟨si, yi⟩}.
We constructed both real and synthetic training datasets for each benchmark respectively, with specifics outlined in Table 1. Notably, for convenience, we directly segregated a subset of 100 non-member data from WikiMIA as real data, and the remaining part will be used in subsequent experiments.

heading: 5 Experiments
text: We evaluate the performance of our method and other baselines against open-source language models trained to predict the next word, including Pythia-2.8B (Biderman et al., 2023), OPT-6.7B (Zhang et al., 2022), TinyLLaMA-1.1B (Zhang et al., 2024) and OpenLLaMA-13B (Geng and Liu, 2023).

heading: 5.1 Datasets
text: WikiMIA proposed by Shi et al. (2023), a dynamic benchmark designed to periodically and automatically evaluate detection methods on any newly released pre-trained LLMs. We use the WikiMIA-32 split in this work, which contains 776 samples with a max length of 32 tokens.
ArxivMIA proposed in this work, a more complex benchmark comprising abstracts in the fields of Computer Science and Mathematics from Arxiv. The details refer to subsection 4.1.
We split each dataset into a validation set and a test set in a ratio of 1:4. The validation set is used to select the best hyperparameters, and the test set is used to evaluate the performance of the detection methods.

heading: 5.2 Evaluation Metrics
text: Following Shi et al. (2023); Carlini et al. (2022); Mattern et al. (2023), we assess the effectiveness of detection methods using these metrics:
Area Under the ROC Curve (AUC). The ROC curve plots the true positive rate (power) against the false positive rate (error) across various thresholds γ, which captures the trade-off between power and error. Therefore, the area under the ROC curve (AUC) serves as a singular, threshold-independent measure to evaluate the effectiveness of the detection method.
True Positive Rate (TPR) under low False Positive Rates (FPR). We report TPR under low FPR by adjusting the threshold value γ. Concretely, we set 5% as the target FPRs, and report the corresponding TPRs.

heading: 5.3 Baselines
text: To compare the performance of Probe Attack, we consider the following reference-free methods:
Loss Attack proposed by Yeom et al. (2018), which assesses the membership of the target sample based on the loss of the target model.
Neighbor Attack proposed by Mattern et al. (2023), which compares model loss for the target sample to losses of synthetically generated neighbor texts. We construct 100 neighbors for each sample using one-word replacement with the RoBERTa-base model (Liu et al., 2019).
Min-K% Prob proposed by Shi et al. (2023), begins by calculating the probability of each token
in the target sample, then selects the k% of tokens with the lowest probabilities to compute their average log-likelihood. A high average log-likelihood suggests that the text is likely part of the pretraining data.
Following Carlini et al. (2021) and Shi et al. (2023), we also consider reference-based methods, which calibrate difficulty by quantifying the intrinsic complexity of a target sample:
Comparing to Zlib Compression. We compute the zlib entropy of the sample, which is the entropy in bits when the sequence is compressed using zlib2. The detection score is then determined by the ratio of the model’s perplexity to the zlib entropy.
Comparing to Lowercased Text. We compute the ratio of the perplexity of the sample before and after converting it to lowercase.
Comparing to Smaller Model. We compute the sample perplexity ratio of the target model to a smaller model pre-trained on the same data.

heading: 5.4 Implementation Details
text: For WikiMIA, we employ Pythia-2.8B (Biderman et al., 2023) and OPT-6.7B (Zhang et al., 2022) as the target model following Shi et al. (2023). For ArxivMIA, we employ TinyLLaMA-1.1B (Zhang et al., 2024) and OpenLLaMA-13B(Geng and Liu, 2023) pre-trained on RedPajama (Computer, 2023) as the target model.
For comparing to smaller model baseline setting, we take Pythia-70M for Pythia-2.8B, OPT350M for OPT-6.7B, and OpenLLaMA-3B for OpenLLaMA-13B. Because there is no smaller model for TinyLLaMA, we leave this baseline out for TinyLLaMA.
For the training of the proxy model, we conducted a grid search hyperparameters on a held-out validation set in order to better inject member data into the model. Based on the performance, the best choice is to put all the data to be injected into one batch and train for 2 epochs. For different models and datasets, we set the best learning rate and activation extraction model layer according to the performance of the validation set. All experiments are conducted with 2 NVIDIA A100 (40GB) GPUs.
2https://github.com/madler/zlib

heading: 6 Results and Analyses
text: In this section, we report our main result and conduct ablation studies to analyze the impact of model size and training data number for our method. We also compare the performance of various detection methods on PubMedQA (Jin et al., 2019) and CommonsenseQA (Talmor et al., 2019) in the contamination detection challenge proposed by Oren et al. (2023).

heading: 6.1 Main Results
text: We present the main results of our experiments in Table 2 and Table 3, where the former shows AUC values and the latter shows true positive rates at 5% false positive rates. The results show that our method consistently outperforms all baselines on both WikiMIA and ArxivMIA benchmarks, achieving state-of-the-art AUC values. We also achieve the state-of-the-art average true positive rates at 5% false positive rates across all detection methods on both benchmarks. We can further observe that:
• The average performance across all detection methods is notably lower on ArxivMIA compared to WikiMIA. This disparity underscores the increased complexity of ArxivMIA as a benchmark. In addition to our method, the Neighbor Attack method exhibits a relatively competent AUC value.
• The performance gap between various detection methods across the two fields of ArxivMIA is notable. Specifically, in the ArxivMIA-CS, the average AUC value is comparatively higher, with our method achieving its best results above 60. In contrast, in the ArxivMIA-Math, the values are only above 50, differing by approximately 10 points. This discrepancy may suggest that mathematical content in academic papers is more challenging for Large Language Models (LLMs) to memorize, and consequently, harder for our method to detect.
• As shown in Table 2, we separately test our method with real and synthetic data. On WikiMIA, the utilization of real data marginally outperforms synthetic data, while the opposite is observed on ArxivMIA. Despite a slight difference, the performance of our method is far superior to other baselines with both real and synthetic data.
Method WikiMIA ArxivMIA ArxivMIA-CS ArxivMIA-Math Pythia OPT TinyL. OpenL. TinyL. OpenL. TinyL. OpenL.
Method WikiMIA ArxivMIA Avg. Pythia OPT TinyL. OpenL.

heading: 6.2 Cross-Domain Evaluation
text: We conducted cross-domain evaluations with TinyLLaMA. As shown in Table 6, we find that in-domain performance is better than cross-domain performance. Training on WikiMIA and then testing on ArxivMIA, AUC values dropped by 5 points. Conversely, training on ArxivMIA and then testing on WikiMIA, AUC values only decreased by 1 point. This supports the finding that ArxivMIA is relatively more challenging than WikiMIA.

heading: 6.3 Ablation Studies
text: We further investigate the impact of model size and training data number for our method:
Model Size. We evaluate our method and neighbor attack on ArxivMIA with different OpenLLaMA sizes (3B/7B/13B). As shown in Figure 2, the AUC values of our method increase with the model size, while the change of neighbor attack is not significant. This result indicates that our method benefits from larger models.
Number of Training Data. We also evaluate our method with different synthetic training data sizes (50, 100, 200, 500 and 1000). We conduct the comparison experiment on ArxivMIA with TinyLLaMA. As illustrated in Figure 3, our method exhibits optimal performance with 200 training data samples. Increasing the number of training data beyond this point results in a slight decline in performance, yet it remains superior to various baselines. This indicates that our method is data efficient.

heading: 6.4 Downstream Task Datasets Contamination Detection Challenge
text: To support the development of further work on detecting pretraining data contamination, Oren et al. (2023) pre-trained a 1.4 billion parameter GPT-2 model (Radford et al., 2019), Contam-1.4b, with intentional downstream task datasets contamination 3. We evaluate various detection methods on PubMedQA (Jin et al., 2019) and CommonsenseQA (Talmor et al., 2019) from this challenge. PubMedQA and CommonsenseQA have different duplication counts (how often the dataset was injected into the pre-training data) with 1 and 2, and detection at this low duplication level is extremely
3https://github.com/tatsu-lab/test_set_ contamination
difficult (Oren et al., 2023).
Experimental Setup. We sampled 1000 examples from the contaminated training data as member data for each task and then sampled 1000 examples from their standard dataset as non-member data. Similar to subsection 5.1, we split each dataset into a validation set and a test set. The validation set will be used to select the best hyperparameters, and the test set for evaluation. For our method, we collected 200 synthetic training data for each task. For comparing to smaller model baseline setting, we choose Contam-Small (124M Params) pre-trained on the same dataset for Contam-1.4b.
Results. The results are shown in Table 4. We observe that our method outperforms other baselines, which demonstrate the effectiveness of our method. Nonetheless, we acknowledge that the overall detection efficacy is unsatisfactory at an extremely low duplication count (1 and 2), corroborating the findings of Oren et al. (2023).

heading: 7 Conclusion
text: In summary, this paper investigates the pre-training data detection problem in large language models. We propose a simple and effective approach that determines whether a target text has been included in a model’s pre-training dataset by analyzing the internal activations using the probe technique. Additionally, we introduce a more challenging benchmark, ArxivMIA. The experiments demonstrate that our method outperforms all baselines across various benchmarks, achieving SOTA performance. We further analyze the impact of target model size and the number of training data on our method. Additionally, we validate the effectiveness of our approach through a downstream task datasets contamination detection challenge. Future work could extend our methods to larger model scales or apply them to multi-modal models.
Limitations
Generalization. One limitation of our study stems from the generalizability of the probe classifier, which necessitates domain-specific training data. This characteristic implies that the training data are not universally applicable across different domains/benchmarks. Consequently, to detect data from varied fields, it becomes imperative to collect distinct sets of training data for each domain.
Computational Resource Requirements. While our method demonstrates superior performance, it necessitates a certain amount of computational resources due to the requirement to train both a proxy model and a probe classifier. We conduct experiments on WikiMIA with LoRA (Hu et al., 2022a), a representative Parameter-Efficient
Fine-Tuning (PEFT) method. The results are presented in Table 5. We observed that the performance of LoRA is inferior to fine-tuning, but it remains competitive with other baselines. For our experimental settings, the training compute consumption is not that high since the member dataset is rather small. However, as the number of data scales, PEFT may be a better choice.

heading: Acknowledgments
text: This work is supported by the National Natural Science Foundation of China (Grant No. 62036004, 62376177) and Provincial Key Laboratory for Computer Information Processing Technology, Soochow University. This work is also supported by Collaborative Innovation Center of Novel Software Technology and Industrialization, the Priority Academic Program Development of Jiangsu Higher Education Institutions. We would also like to thank the anonymous reviewers for their insightful and valuable comments.

heading: A Data Synthesis with ChatGPT
text: Given a target dataset D0, our goal is to utilize ChatGPT to generate a new, similar, domainspecific dataset D. To achieve this, we employ a templated prompt to guide ChatGPT in generating data points that are stylistically and structurally similar to D0, yet unique in content. The prompt template used is shown in Table 7.
To initiate this process, we randomly select 5 examples from D0 and insert them into the prompt. This prompt is then provided to ChatGPT, which generates a specified number of new data points. By iterating through this procedure multiple rounds, we can get a dataset D that is similar to and within the same domain as D0.

heading: B Prompt Template Selection
text: We experiment with different prompt templates on ArxivMIA with TinyLLaMA to determine the optimal one for our task. The results of the ablation study on prompt templates are shown in Table 8. The average AUC value for using a template is 57.9, which is higher than the AUC value of 54.3 for not using a template. This demonstrates the importance of a prompt template.

heading: Future Work
text: Future work could extend our methods to larger model scales or apply them to multi-modal models. Limitations
Generalization. One limitation of our study stems from the generalizability of the probe classifier, which necessitates domain-specific training data. This characteristic implies that the training data are not universally applicable across different domains/benchmarks. Consequently, to detect data from varied fields, it becomes imperative to collect distinct sets of training data for each domain. Computational Resource Requirements. While our method demonstrates superior performance, it necessitates a certain amount of computational resources due to the requirement to train both a proxy model and a probe classifier. We conduct experiments on WikiMIA with LoRA (Hu et al. , 2022a), a representative Parameter-Efficient
Fine-Tuning (PEFT) method. The results are presented in Table 5. We observed that the performance of LoRA is inferior to fine-tuning, but it remains competitive with other baselines. For our experimental settings, the training compute consumption is not that high since the member dataset is rather small. However, as the number of data scales, PEFT may be a better choice