DPDLLM : A Black-box Framework for Detecting Pre-training Data from Large Language Models (2024)

Abstract: The success of large language models (LLM) benefits from large-scale model parameters and large amounts of pre-training data. However, the textual data for training LLM can not be confirmed to be legal because they are crawled from different web sites. For example, there are copyrighted articles, personal reviews and information in the pre-training data for LLM which are illegal. To address the above issue and develop legal LLM, we propose to detect the pre-training data from LLM in a pure black-box way because the existing LLM services only return the generated text. The previous most related works are the membership inference attack (MIA) on machine learning models to detect the training data from them. But the existing methods are based on analyzing the output probabilities of models which are unrealistic to LLM services. To tackle the problem, we firstly construct the benchmark datasets by collecting textual data from different domains as the seen and unseen pre-training data for LLMs. Then, we investigate a black-box framework named DPDLLM, with the only access to the generated texts from LLM for detecting textual data whether was used to train it. In the proposed framework, we exploit GPT-2 as the reference model to fit the textual data and feed the generated text from LLM into it to acquire sequence probabilities as the significant feature for detection. The experimental results on the benchmark datasets demonstrate that DPDLLM is effective on different popular LLMs and outperforms the existing methods.

## introduction language modeling (lm) is a major approach to advance natural language processing systems by modeling the likelihood of sequence of words occurring in a given context @xcite . considering to combine the neural networks with language modeling for generating human-like to overcome the above disadvantages, we propose a black-box framework named dpdllm 1 to detect pre-training data from llms. based on the actual scenario, we require that the black-box approach only obtains the generated texts, without access to losses or probabilities of llms, as depicted in figure 1 . the dpdllm framework consists of two main components: the reference model and the classifier. given texts to be detected, we use the reference model to fit them for memorizing the texts. then, the prefix of the detection text, such as the first half, is fed into llms to obtain the generated text. ultimately, we input them into the reference model to obtain the language modeling sequence probabilities as important features. if the detection text was employed to pre-train llms, the corresponding generated text is similar to the raw text and the reference model will assign the high probabilities to it because of the memorization on it. the features of detection texts are fed into the classifier to predict if they were utilized during pre-training. to assess different detection methods on popular llms, we construct the largescale benchmark datasets by collecting the seen and unseen texts during pre-training llms. the experimental results demonstrate the superiority of 1 when ready, the code will be published at https:// github.com/zovanzhou/dpdllm the dpdllm framework on the proposed benchmark datasets. and the further analysis of different factors on the datasets verifies the robustness of the proposed method. the contributions of the manuscript can be summarized as follows: ## methodology before getting into the details of the proposed framework, we formalize the problem of detecting pre-training data from llms. given the target large language model f θ pre-trained on the seen dataset d s , the task objective is to learn a detector h for classifying the detection text x : h(x, f θ ) → {0, 1} where @xmath0. the training and test sets are denoted as d tr and d te respectively. and we formulate the training set as the dpdllm framework for detecting pretraining data from llms is illustrated in figure 2 . the framework consists of the reference model and the classifier, and is trained to detect pre-training data from target llms. to mimic the realistic scenario, we demand that the llms can be accessed in a black-box way to acquire the generated contents and the reference model can be utilized to calculate the token probabilities in a white-box way. given the texts to be detected, the reference model is trained with them for memorizing the whole texts. to obtain the memorization on detection texts of the target llm, we feed the prefix of the texts, such as the first half, into the llm to generate the postscripts. the trained reference model is employed on the whole generated texts to calculate the language modeling sequence probabilities as significant detection features, and we use the gaussian naive bayes classifier to fit them for detection. ## experimental setup to detect the pre-training data from llms, we investigate the open-source llms including: pythia-2.8b @xcite , llama-7b and llama-13b @xcite as target models. based on the reports of the above llms' training details, we can construct the detection datasets. datasets. to evaluate the different detection methods, @xcite proposed the benchmark datasets wikimia and bookmia. however, the above datasets are limited to the small size and domains. therefore, we propose the new benchmark datasets with large size and more domains. the detailed information of the benchmark datasets is shown in wikimia2: considering that there are only 542 samples in wikimia, we extend it as wikimia2 by collecting newest contents from websites including: wikipedia and news. based on the wikipedia page foot_0 of events occurred in 2023, we crawled the texts and the reference websites of news in it as the unseen data. and we follow the report of llms @xcite to use the previously dumped pages of wikipedia as the seen data which were employed to pre-train them. bookmia: @xcite constructed the dataset by extracting 100 random 512-word snippets from 100 books of the books3 corpus @xcite as the seen data and collecting 50 new books with first editions in 2023 as the unseen data. wiki-spgc: to construct the multi-domain dataset, we select books of gutenberg corpus @xcite as the seen data and take the unseen data of wikimia2 into account. wikimia2-spgc: to mimic the seen data from multi-domains, we merge wikimia2 with the books of gutenberg corpus as the whole dataset. and it is a hard benchmark to evaluate the detection methods under the multi-domain scenario. baselines. we take existing detection methods as our baselines and utilize them to extract detection features for classification. the widely used feature is the perplexity (ppl) @xcite of the sample fed into language models (lm). the other ppl-based methods include comparing the sample ppl to zlib compression entropy (zlib) or to the lowercased sample ppl (lowercase) @xcite . besides, we take the sentence probability from lm as detection feature and apply the average operation on it (average prob). @xcite proposed to select the k% tokens with the minimum probability and compute the average log-likelihood of them as detection feature. in the experiments, we select gpt-2 as the reference model and train it for 15 epochs to memorize detection texts. for different baselines, we extract their specific features to optimize classifiers based on training set and evaluate them on test set. ## conclusion in this paper, we discuss the issue of using potentially illegal pre-training data for large language models (llms) and analyze the disadvantages of existing detection methods. considering to avoid training shadow models and accessing inner of llms, we propose the black-box framework named dpdllm to detect pre-training data from llms. the proposed framework comprises a reference model and a classifier. the reference model is trained to memorize detection texts, while the classifier is optimized using detection features of memorization texts extracted from target llms, based on the reference model. to evaluate detection methods, we construct the benchmark datasets from different domains and the experimental results demonstrate the superiority and robustness of the proposed methods over baselines.