SlowGuess/NeurIPS_2023_OCR huggingface dataset:
Additionally we only select datasets with minimal normalization and preprocessing since these are likely to appear in web-scapes. Specifically we use target text from XSum [28], TIFU [20], and AMR2.0 [21, LDC2017T10]. XSum and TIFU are summarization datasets created from news articles and reddit respectively. Labels consist of brief summaries of a larger document. AMR2.0 (Abstract Meaning Representation) is a treebank resource that can be used for graph-to-text generation. Inputs consist of semantic graphs and labels are a target sentence with the same semantics. Some target text is translated to English specifically for this resource and should not appear naturally in a web-scrape absent leakage.
We use an Expected Overlap metric that compares the expected number of n-gram matches in a document
Table 3: Overlap statistics. E.O. is the Expected Overlap metric. Time is in seconds, measuring the total query time for that dataset.
Dataset	%E.O.	Instances	Time
XSum	40.12%	11.3K	3.61
TIFU-short	4.12%	79.7K	1.28
TIFU-long	3.86%	42.1K	10.73
AMR2.0 (LDC)	8.12%	1.4K	0.45
AMR2.0 (Plain)	8.06%	1.4K	0.43
Sum	-	136.0K	16.5
(had the complete document appeared in the Pile) to the observed longest match. Given a strided Bloom filter sketch with width  $w$ , Expected Overlap for a corpus  $T$  is:  $\frac{\sum_{d \in T} \max(chains_d)}{\sum_{d \in T} E(length(d), w)}$  where  $d$  are documents in a test set  $T$ . max(chains_d) is the max chain length i.e. the approximate longest overlapping subsequence between a document and the sketched dataset.  $E()$  is the expected number of matches had  $d$  appeared entirely in the corpus, accounting for boundary issues (see Appendix E). Many other metrics are possible using our sketch (e.g. total match count).
Results are in Table 3. Target summaries from XSum have the highest incidence of overlap with the Pile. This is unsurprising since the dataset is constructed from easily scraped BBC articles [28]. TIFU (short and long) are constructed from summaries of reddit posts. Many of these summaries are too short to guarantee a match in the strided filter we use here for illustration, so these overlap statistics are a lower bound. For AMR2.0, we search for both raw target text as it appears in LDC (Linguistic Data Consortium) distributed files and detokenized plaintext. We find slightly higher overlap for the raw text. This suggests that some LDC files are publicly exposed and scraped into LLM corpora (LDC data licenses typically forbid redistribution).<sup>7</sup> Creating this table is extremely fast using our sketch, taking on the order of 0.1ms per instance (136K total). While the methods are not directly comparable, available full text search tools take on the order of seconds per query when accessed on the web.
<sup>7</sup>We manually found instances of LDC data on GitHub.
Matching Text
Found spans are in grey. The longest span is in blue. Hovering over a character highlights the longest span that includes that character (there may be overlapping shorter spans). Clicking shows the component substrings below.
if !idppc  
/*  
** float q_rsqrt(float number)  
*/  
float Q_rsqrt(float number)  
{  
long i;  
float x2, y;  
const float threehalfs = 1.5F;  
x2 = number * 0.5F;  
y = number;  
i = * (long *) &y; // evil floating point bit level hacking  
i = 0x5f3759df - (i >> 1); // what the fuck?  
y = * (float *) &i;  
y = y * (threehalfs - (x2 * y * y)); // 1st iteration  
// y = y * (threehalfs - (x2 * y * y)); // 2nd iteration, this can be removed  
#ifdef Q3_VM  
#ifdef __linux>  
assert(!isnan(y)); // bk010122 - FPE?  
#endif  
#endif  
return y;
Figure 4: Text for the Fast Inverse Square Root algorithm from the Quake III source code. The blue highlighted span is the longest match, component spans are in green below. Some of the grey spans are preprocessor directives that also appear in the Pile.
4.4 Code Generation
Code generation or text-to-code models are another application of LLMs. GitHub Copilot $^{8}$  is a tool built on the OpenAI Codex model [11]. Some users have found that Copilot will copy famous snippets of code and add a new license. $^{9}$  GitHub has studied plagiarism and has taken steps to avoid copying from existing repos [38]. However, conventions and precedents around language models trained on open source code are rapidly developing. Content creators (i.e. programmers who have published code on GitHub) may wish to check if their code has been included as part of an LLM training corpus. Similarly, a downstream user might want to check if the outputs of a tool like Copilot substantially overlap a collection of data (e.g. for licensing compliance purposes). Figure 4 shows an example of our full front-end on a famous snippet of code. See the demo at pile.dataportraits.org
<sup>8</sup>https://github.com/features/copilot
<sup>9</sup>https://twitter.com/mitsuhiko/status/1410886329924194309
to interact with this example. The highlighted text suggests that the passage appears in the Pile. Specifically, hashes for the contiguous 50-grams are present in the pile (green, below). The Longest Chains section lists the longest sub-strings composed of these n-grams. Since hashing is very fast, users can interact with this interface and see updates at typing speed.
We build a Portrait documenting The Stack in addition to our Pile Portrait. This documentation artifact is available through our web interface (stack.dataportraits.org) and additionally powers a plagiarism checking service exposed to live users. This artifact exactly documents the training data used in StarCoder, a 15.5B parameter code generation model [25]. The StarCoder model has been used in a variety of open source applications and tools including a Copilot-like autocomplete plugin for VSCode. $^{10}$  This plugin implements a plagiarism check by calling our Portrait service. Spans that overlap the model's training set are highlighted in the user's editor. Our Portrait service acts as a lightweight first-pass check for model plagiarism and additional tools from Li et al. [25] can be used to further assess attribution. Our sketch-based Data Portrait is efficient – this service has active users and runs on a single cloud VM with 32GB of RAM. Most queries take only  $\sim$  10s of milliseconds.
To further validate our approach, we return to the fundamental stakeholder question: "is this text in a corpus?" This is a binary classification task: is query document  $d$  in dataset  $C_{\text{pile}}$ ? We design a simple experiment to test how well our method method works as a classifier and compare it to a full text search engine corresponding to the Roots Search Tool of [30].
Data We construct a small test set of text sampled from the Pile and text that is certainly not in the Pile. Without prior access to a membership testing tool, collecting text not included in an LLM corpus can be difficult. We harvest paragraphs from New York Times articles published between August 7-18 2023. Since the Pile was created in 2020, this text is certainly not in the Pile and any overlap should be spurious (e.g. long proper
nouns or repeated quotations). This set consists of  $5.5\mathrm{k}$  wc -w words and is balanced with respect to document length and class label.
Table 4:  $F_{1}$  and latency on our classification task. Both methods can correctly classify Pile text, but ours (BF) is very fast.
Method	F1↑	Sec/Doc↓
BF	1.0	0.015
FTS200	1.0	11.28
FTS50	0.57	1.81
Classifiers The striped Bloom filter approach produces a list of n-grams and a boolean membership value. As described in Section 3.2, we can obtain an approximate longest common substring (lcs) between the query and corpus. Our classification rule is simply:  $\frac{length(lcs)}{length(query)} > 0.9$  where 0.9 is a boundary threshold. We construct a similar classifier using a full text Lucene index on the Pile, where we take the common substring between the query and top result.[11] We find both classifiers can achieve perfect precision and recall on this synthetic task. However, the full text search classifier (FTS in Table 4) is much slower than the Bloom filter based classifier (BF).[12] The retrieval engine's response time is heavily dependent on query length, thus we experiment with only the first  $N$  characters of the document:  $\mathrm{FTS}_N$ . At 200 characters of context, the FTS classifier achieves perfect recall and precision though it takes  $\sim 700x$  more time than the Bloom filter. Queries are much faster with 50 characters of context, but precision and recall suffer. We again emphasize that full indexes provide far more information than a Bloom filter; however, a minimal Bloom filter is sufficient for this classification task.
5.1 False Positives
Our method is probabilistic but the previous sections show that it is still useful for understanding datasets. Here we further analyze false positives. Using the dataset of novel text constructed for the previous section, we record all n-grams for which the Bloom filter returns True. To obtain ground
$^{10}$ https://github.com/huggingface/huggingface- vscode
This search instance did not directly support exact match phrase queries.
Latency measurements are averaged over 5 runs to account for network timing.
truth labels for  $ngram \in C_{pile}$ , we query the Pile Lucene index and record whether any of the top 25 documents contain that n-gram.
We first find that there are no chained matches when querying our system with novel New York Times text. That is, only isolated 50-grams match, rather than a sequence of multiple matching n-grams. Using the ground truth labels from the Pile search, we next find a false positive rate of  $7 \times 10^{-4}$  (slightly less than the intended rate of  $1 \times 10^{-3}$ ). The NYTimes text also contains some true positive matches. For example, "National Counterintelligence and Security Center," is a 50-character string that coincidentally appears in both our novel corpus and the Pile. We found that a resolution of 50 characters is sufficient for document fingerprinting, but this parameter could also be increased to reduce matches of long common strings. We further note that the false positive rate (FPR) is simply a parameter that can be directly set (at construction time) at the cost of space. Decreasing the FPR by an order of magnitude (to  $1 \times 10^{-4}$ ) would increase storage size to about 36 GB and increasing the FPR to  $1 \times 10^{-2}$  would result in an 18 GB structure.
6 Impact and Limitations
We hope that this work will encourage members of the field to adopt membership testing tools as part of a complementary suite of dataset and model documentation artifacts. Many in the field have taken issue with massive opaque datasets. Our web interface and tools are meant to facilitate simple, easy, transparency. We hope that one broader impact of our demo will be enabling both NLP specialists and lay-people in assessing large language models. For example, a non-technical content creator could search for their material in a documented dataset.
Our work is limited in that we focus on studying only a few datasets. We hope our tooling will enable the study of other datasets in the future. We stress a lightweight and efficient hash-based approach to membership testing, but this comes with certain trade-offs such as false positives and the inability to retrieve the context surrounding a match. We also note that while we intend to better document existing datasets, these tools could be used to modify generated outputs. That is, an open plagiarism detection tool could be used to avoid plagiarism detection.
Foundation models come with concerns such as plagiarism, test-set and personal data leakage, data contamination, and data provenance. We have argued that existing documentation practices are not sufficient on their own to address these concerns, and proposed the adoption of Data Portraits: records supporting membership inference on the data used to train a model. After discussing examples of existing solutions, we described our own time and space efficient version based on Bloom filters and host several demos. In future work, we hope to investigate applications of similar efficient data structures for indexing and counting. Our current implementation is minimal in that it aims to satisfy a form of membership testing and nothing more, such as full text indexing. We demonstrate its usefulness in case studies, and call for dataset and model creators to release some form of Data Portrait as part of their essential documentation.
We thank the JHU HLTCOE for compute resources used in developing this work and the BigCode collaboration project for hosting the artifact documenting their Stack dataset. We thank students in the JHU CLSP for their feedback on this project, particularly Orion Weller, Nathaniel Weir, and Kate Sanders. Daniel Khashabi also provided valuable guidance in addressing reviewer feedback.
[28] Shashi Narayan, Shay B. Cohen, and Mirella Lapata. Don't give me the details, just the summary! topic-aware convolutional neural networks for extreme summarization. In Proceedings of the 2018 Conference on Empirical Methods in Natural Language Processing, pages 1797-1807, Brussels, Belgium, October-November 2018. Association for Computational Linguistics. doi: 10.18653/v1/D18-1206. URL https://aclanthology.org/D18-1206.
[29] OpenAI. Chatgpt, 2022. URL https://openai.com/blog/chatgpt.
[30] Aleksandra Piktus, Christopher Akiki, Paulo Villegas, Hugo Laurençon, Gérard Dupont, Sasha Luccioni, Yacine Jernite, and Anna Rogers. The ROOTS search tool: Data transparency for LLMs. In Proceedings of the 61st Annual Meeting of the Association for Computational Linguistics (Volume 3: System Demonstrations), pages 304-314, Toronto, Canada, July 2023. Association for Computational Linguistics. doi: 10.18653/v1/2023.acl-demo.29. URL https://aclanthology.org/2023.acl-demo.29.
[31] Matt Post. A call for clarity in reporting BLEU scores. In Proceedings of the Third Conference on Machine Translation: Research Papers, pages 186-191, Belgium, Brussels, October 2018. Association for Computational Linguistics. URL https://www.aclweb.org/anthology/W18-6319.
[32] Alec Radford, Jeff Wu, Rewon Child, David Luan, Dario Amodei, and Ilya Sutskever. Language models are unsupervised multitask learners. 2019.
[33] Colin Raffel, Noam Shazeer, Adam Roberts, Katherine Lee, Sharan Narang, Michael Matena, Yanqi Zhou, Wei Li, and Peter J. Liu. Exploring the limits of transfer learning with a unified text-to-text transformer. Journal of Machine Learning Research, 21(140):1-67, 2020. URL http://jmlr.org/papers/v21/20-074.html.
[34] David Talbot and Miles Osborne. Randomised language modelling for statistical machine translation. In Proceedings of the 45th Annual Meeting of the Association of Computational Linguistics, pages 512-519, Prague, Czech Republic, June 2007. Association for Computational Linguistics. URL https://aclanthology.org/P07-1065.
[35] Twitter. More on restricted use cases - twitter developers, 2023. URL https://web.archive.org/web/20230212045424/https://developer.twitter.com/en/developer-terms/more-on-restricted-use-cases.
[36] Benjamin Van Durme and Ashwin Lall. Probabilistic counting with randomized storage. In Twenty-First International Joint Conference on Artificial Intelligence. CiteSeer, 2009.
[37] Susan Zhang, Stephen Roller, Naman Goyal, Mikel Artetxe, Moya Chen, Shuhui Chen, Christopher Dewan, Mona Diab, Xian Li, Xi Victoria Lin, et al. OPT: Open pre-trained transformer language models. arXiv preprint arXiv:2205.01068, 2022.
[38] Albert Ziegler. *Github copilot research recitation*, 2021. URL https://github.blog/2021-06-30-github-copilot-research-recitation/.
A Supplemental Documentation
This paper recommends best practices does not construct a new dataset or benchmark; thus not all parts of the recommended supplemental material (e.g. a model card or datasheet) are applicable. However, since we host an implementation and plan to distribute artifacts and code, we address similar relevant concerns:
- URL, landing page, and demo: dataportraits.org
- Code: to be released through the site above. Open source code will be hosted on GitHub.
- Hosting and Preservation: We will maintain the existing interactive web interfaces for at least one year from publication. Code will be available for long-term distribution through GitHub.
B Sketch Misses
In Figure 2, consider the query string: defg. This string lies on the boundary of the n-grams split during corpus processing. No stride 1 width 4 ngrams extracted from that query will match the database. However, if the query string were expanded to length  $2 \cdot width - 1$ , note that it would necessarily intersect at least one of the hashed n-grams. For example, defghi j will match at fghi. In this way, given a long enough query string, our query protocol guarantees that at least one match will be found if the query string of interest does occur in the corpus.
C Adversarial Matches
We describe a protocol for chaining matches together. In Figure 2, the three matches of bcde, fghi, jklm occur separated by width indexes. Therefore we infer the whole string (formed by concatenating the three n-grams) was present in the corpus. However, this might not be true. If an adversary knew the details of the sketch width (and initial offset into the sequence), they could construct a document that embeds n-grams in different locations, such that a query string would appear to be present according to our protocol. For example, the sketch in Figure 2 would falsely infer that the string fghibcde is present in the corpus, since it is composed of two chosen matches. This is very unlikely in practice, given appropriately chosen widths and sketch resolutions. This is essentially a permutation attack, and a similar approach could be used to fool a BM-25 index.
D WMT Documents
Table 5 lists the full test set, doc id, and approximate longest match results from Figure 3.
E Counting Expected Matches
Consider a string of interest  $S$ , with length  $N$  that is embedded in a larger document  $D$ . Matching  $S$  with a strided Bloom filter with width  $w$  will yield a chain of something around  $\frac{N}{w}$  tiles (substrings of length  $w$ ). For example, take a sketch with width  $w = 50$  and a string with  $N = 150$ . If the tiles in  $D$  are perfectly aligned on the boundaries of  $S$ , the Bloom filter will find  $150 / 50 = 3$  matches. Perfectly aligned means that string  $S$  begins at an offset in  $D$  that is a multiple of  $w$  — so when breaking  $D$  into non-overlapping chunks (tiles) of size  $w$ , the start of some tile is also the start of  $S$ .
This perfect alignment might happen by chance, but most likely there will be some parts of  $S$  that hang over the tile boundaries, meaning only some inner part of  $S$  will match the hashed tiles. In Figure 2 the query string  $S = abcdefghijklmn$  is not perfectly aligned.  $a$  and  $n$  hang over and thus the only complete tiles are the inner bcde, fghi, jklm tiles match.
Given the width and length of a string, we can calculate the expected number of matches for any possible alignment of the string. Note that there are  $w$  possible alignments and each is equally likely.
A string of length  $N$  modulo the tile width  $w$  can be written as  $N = aw + b$  where  $a$  is the number of full tiles and  $b$  is the remainder. Consider alignments other than the perfect one boundary. We have
Table 5: Full WMT overlap information.
Test Set	doc_id	Longest
wmt13.en-fr.ref	lemondefr/2012/12/01/275696	700
wmt16.en-fi.ref	kaleva-fi.29723	400
wmt16.en-de.ref	tagesspiegel.de.65447	350
wmt20.en-iu.ref	nunatsiaq-20190930	300
wmt12.en-de.ref	noroeste/2011/11/15/78596.html	250
wmt16.en-de.ref	borkenerzeitung.de.56604	250
wmt14.en-fr.ref	4bb85eb6281e0b19986de1d4f867e3ff	250
wmt15.en-ru.ref	893-kommersant	200
wmt18.en-fi.ref	karjalainen.fi.65284	200
wmt18.en-ru.ref	kommersant.324314	200
wmt14.en-fr.ref	cd085bbb218a7afc1255b2b60a06692a	200
wmt15.en-de.ref	14428-abendzeitung-muenchen.de	200
wmt15.en-ru.ref	115-aif	200
wmt16.en-ru.ref	lgng.30237	150
wmt16.en-ro.ref	ziare.ro.17378	150
wmt15.en-ru.ref	1375-rg.ru	150
wmt14.en-fr.ref	90c566f54bf1076e6f539875d45d673c	150
wmt17.en-ru.ref	izvestiya.51251	150
wmt16.en-ro.ref	hotnews.ro.8884	150
wmt14.en-fr.ref	96e21a07ed57d79665a35a548ef7d841	150
wmt16.en-de.ref	abendzeitung-nuernberg.de.12297	150
wmt17.en-de.ref	dw.47065	150
wmt18.en-de.ref	handelsblatt.com.180784	150
wmt17.en-de.ref	frankfurter-rundschau.70094	150
wmt13.en-fr.ref	cyberpresse/2012/12/01/1564248	100
$b + 1$  alignments that produce  $a$  matching tiles. We will also have  $w - b - 1$  alignments that produce  $a - 1$  tiles. Summing and cancelling terms, we have  $(b + 1)a + (w - b - 1)(a - 1) = aw - w + b + 1$  possible matching tiles. Substituting the length of the string simplifies to  $N - w + 1$  possible matches, and since each  $w$  alignment is equally likely, the expected number of matches is  $E(N,w) = \frac{N - w + 1}{w}$ . The 4 possible alignments with 11 possible matching strings in the Figure 2 example are:
[abcd,efgh，ijkl]（missingmn)
[bcde, fghi, jklm] (missing a, n)
[ cdef, ghij, klmn] (missing ab)
[defg,hijk] (missing abc，lmn)