Proving membership in LLM pretraining data via data watermarks (2024)

Abstract: Detecting whether copyright holders’ works were used in LLM pretraining is poised to be an important problem. This work proposes using data watermarks to enable principled detection with only black-box model access, provided that the rightholder contributed multiple training documents and watermarked them before public release. By applying a randomly sampled data watermark, detection can be framed as hypothesis testing, which provides guarantees on the false detection rate. We study two watermarks: one that inserts random sequences, and another that randomly substitutes characters with Unicode lookalikes. We first show how three aspects of watermark design - watermark length, number of duplications, and interference - affect the power of the hypothesis test. Next, we study how a watermark’s detection strength changes under model and dataset scaling: while increasing the dataset size decreases the strength of the watermark, watermarks remain strong if the model size also increases. Finally, we view SHA hashes as natural watermarks and show that we can robustly detect hashes from BLOOM-176B’s training data, as long as they occurred at least 90 times. Together, our results point towards a promising future for data watermarks in real world use.

## introduction many jurisdictions will likely give authors and other copyright holders a right to opt-out their works from machine learning training data. in the eu, such rights are granted by the text and data mining exceptions, 1 which require any non-academics who mine data to respect opt-out requests from the rightholders of that data @xcite . in the u.s., the right to opt-out will be determined by ongoing copyright lawsuits. as the law develops, detecting avg. token loss -6 -4 -2 0 2 z-score ## data watermarks our work proposes the use of data watermarks to detect whether a rightholder's document collection is in the training data of an llm. in the context of opting-out, we make two observations: (1) collections (e.g. news articles) are often centrally accessible (i.e. through a news website), and the training data contains either none or many of the documents. ## relating watermark design to strength in this section, we train many medium-sized language models on watermarked datasets and measure the strength of the watermarks. we further explore how different properties of the watermark affect the power of the hypothesis test, either by increasing the effect size (i.e., the difference between the test statistic and mean of the null distribution) or decreasing the variance of the null distribution (as illustrated in figure 1 ). ## watermarks under scaling both the training datasets and model sizes for popular llms are much larger than those we consider in §4. to build intuition for data watermarks when training large-scale models, we fix a watermarked document collection while scaling both the dataset and model size. we focus on the random sequence watermark here, and present similar findings for the word-level unicode watermark in appendix c.2. ## post-hoc study on natural watermarks to confirm the feasibility of data watermarks in real llms, we conduct a post-hoc study on the detectability of sha and md5 hashes in bloom-176b @xcite . since a good hash function produces hex sequences that are nearly random @xcite , the inclusion of these hashes in training forms a natural experiment. we can detect the hashes as if they were sampled and inserted as random sequence watermarks, where we test the model's loss on seen hashes against randomly sampled hex sequences. some hashes, such as the md5 hash of an empty string, appear in error messages or code and are well duplicated. since most of bloom's training data is publicly available, we can pair observations of the occurrences of a hash with the detection strength of this hash. with these observations, we provide additional empirical guidance on how much duplication is necessary to watermark a document collection. ## future directions further research in several areas of data watermarks will enable its mainstream use. we proposed two watermarks-one that we suggested hiding programmatically, and another that is imperceptible to humans. if the watermark is obvious, malicious model creators could tamper with the watermark. hiding the watermark through word substitutions or semantic paraphrases (in the spirit of @xcite is a natural next step, which requires further study on watermark detectability and erasability (similar to @xcite the main contribution of this work is to relate basic aspects of watermark design to the detection strength. we hope that future work uses our insights as a guide in designing stealthy data watermarks. finally, pretrained language models are often fine-tuned on human feedback @xcite , and whether data watermarks persist after fine-tuning requires additional study. beyond supporting a right to opt-out, data watermarks may also have the potential to meaningfully contribute to the discourse on data stewardship for responsible machine learning @xcite . one important question in this area is how to mitigate the risks of unintended usages of open data @xcite . @xcite propose to establish a public trust for the digital commons, where data watermarks could be used to verify whether models were trained on open data. the trust could then ensure their compliance with standards that align with the public interest. methods that strengthen the relationship between a model and its training data, such as data watermarks, may open up new legal frontiers and present opportunities for training data to play a role in a model's responsible deployment. ## conclusion to support a right to opt-out of language model training, our work proposes the use of data watermarks. rightholders can detect if their data was used for training, by watermarking their data before public release. if the data watermarks are memorized, this serves as statistical evidence on whether rightholders' data has been trained on. by relating aspects of watermark design to the strength of its detection, our insights lay the groundwork for future work on data watermarks. scaling experiments show that data watermarks are stronger for larger models, and a post-hoc study on naturally occurring sha hashes confirms that random sequences watermarks could be detected in bloom-176b if it occurred more than 90 times in the training data. together, our results point towards a promising future for data watermarks in real world use.