Blackbox Model Provenance via Palimpsestic Membership Inference
Authors: Rohith Kuditipudi
Jing Huang
Sally Zhu
Diyi Yang
Christopher Potts
Percy Liang
Venue: NeurIPS 2025 spotlight ()
Keywords: provenance
membership inference
model stealing
Primary area: deep_learning
TL;DR: We develop tests for proving that someone is producing text using a particular language model by correlating the text with the order of examples used to train the model.

Abstract: Suppose Alice trains an open-weight language model and Bob uses a blackbox derivative of Alice’s model to produce text. Can Alice prove that Bob is using her model, either by querying Bob’s derivative model (query setting) or from the text alone ( observational setting)? We formulate this question as an independence testing problem—in which the null hypothesis is that Bob’s model or text is independent of Alice’s randomized training run—and investigate it through the lens of palimpsestic memorization in language models: models are more likely to memorize data seen later in training, so we can test whether Bob is using Alice’s model using test statistics that capture correlation between Bob’s model or text and the ordering of training examples in Alice’s training run. If Alice has randomly shuffled her training data, then any significant correlation amounts to exactly quantifiable statistical evidence against the null hypothesis, regardless of the composition of Alice’s training data. In the query setting, we directly estimate (via prompting) the likelihood Bob’s model gives to Alice’s training examples and their training order; we correlate the likelihoods of over 40 fine-tunes of various Pythia and OLMo base models ranging from 1B to 12B parameters with the base model’s training data order, achieving a p-value on the order of at most $1 \times 10^{-8}$ in all but six cases. In the observational setting, we try two approaches based on estimating 1) the likelihood of Bob’s text overlapping with spans of Alice’s training examples and 2) the likelihood of Bob’s text with respect to different versions of Alice’s model we obtain by repeating the last phase (e.g., 1%) of her training run on reshuffled data. The second approach can reliably distinguish Bob’s text from as little as a few hundred tokens; the first does not involve any retraining but requires many more tokens (several hundred thousand) to achieve high power.

Blackbox Model Provenance via Palimpsestic Membership Inference
Rohith Kuditipudi" rohithk@stant ord. edu
Jing Huang" hij @sjasft ord. edu
Sally Zhu' salzhu@stanf ord. edu
Diyi Yang" diyiy0cs. Seiiyyycssstaafore stanford. edu
Christopher Potts cgpotts@stanford. edu
Percy Liang! pal@stanford.co edu
Department of Computer Science
Stanford University
Abstract
Suppose Alice trains an open-weige language model and Bob uses a blackbox derivative of Alice's model to produce text. Can Alice prove that Bob is using her model, either by querying Bob's derivative model (query setting) or from the text alone (observational setting)? We formulate this question testing problem-in which the null hypothesis that Bob's model or text is independent of Alice's randomized training run-and investigate it through the lens of palimpsestic memorization in language models: models are more likely memorize data seen later in training, we Alice's model using test statistics that capture correlation between Bob's model text and the ordering of training examples in Alice's 's training run. If Alice has randomly shuffled her training data, then any significant correlation amounts to exactly quantifiable statistical evidence against the null hypothesis, regardless of the composition of Alice's 's training data. In the query setting, we directly estimate (via prompting) the likelihood Bob's 's model gives to Alice's training examples and their training order; we correlate the likelihoods of over 40 fine-tunes of various Pythia and OL.o base models ranging from 1B 12B parameters with the base model's 'ss training data order, achieving p-value on the order of at most x 10- all but six cases. In observational setting, we try two approaches based on estimating 1) the likelihood of Bob's text overlapping with spans of Alice's training examples and 2) the likelihood of Bob's text with respect different versions Alice's model obtain by repeating the last phase (c.g., 1%) of her training run on reshuffled data. The second approach can reliably distinguish Bob's 'ss text from as little as few hundred tokens; the first does not involve any retraining but requires many more tokens (several hundred thousand) to achieve high power.
independence
whether Bob is using
Introduction
Definition 1. A palimpsest is "writing material (such as a parchment or tablet) used one or more times after earlier writing has been erased" [1].
"qqual contribution.
'Equal advising.
39th Conference on Neural Information Processing Systems (NcurIPS 2025).

---

Non-independent model
Independent model
Negative example log aelllkellhood (normalized)
2
Bob's model
= 0
Negative example log- biglllkklihood (normalized,
y=0
Bob's model
-1
3-value=3.13e-25
p-value=0.91
0.4 0.6
0.8
1.0
1.2
14
0.0
0.2
0.4
0.6
0.8
1.0
1.2
1.4
Alice's training order
Alice's training order
len
(a) Log-likelihoods of pythia-6 9b-deduped.
(b) Log- ligeliiss pythia-6... 6. 9b.
Figure 1: We regress the negative log- lieliieklihoods of pythia-6. 9b-deduped and pythia-6. 9b on pythia-6. 9b-deduped training examples against its training order, which is independent of pythia-6. 6.9b training order. Though the log-likelihoods of individual examples noisy, the overall trend is clear over many examples: pythia-6. bechhai-.-b--dddee deduped exhibits significant correlation, while the independently trained pythia-6 6. 9b exhibits near zero correlation.
Suppose Alice trains an open-weight language model and Bob produces text using derivative of Alice's model. Can Alice prove that Bob is using her model? We ground this question with a few motivating examples. Alice may be model developer who suspects Bob of hosting a chat interface that violates Alice's terms of service." Can Alice prove via querying Bob's interface that it is built (e.g., fine-tuned) from her model? Alternatively. Alice may suspect Bob of operating a social media bot using her model. Can she prove from the account's post history that is using her model?
We formulate the question as an independence testing problem: the goal is to test whether Bob's text (or equivalently, the model producing it) is statistically independent of Alice's training run. The inherently randomized nature language model training-dus in part to the shuffling of training examples-allows us to treat the training outcome
random variable. We assume access to the ordered sequence of training examples from Alice's s training run and consider two variations of the problem depending the type of access permitted Bob's model. In the query setting, to directly prompt Bob's 's model; in the observational setting, we merely observe text generated by Bob's 's model from unknown prompt (or multiple prompts). We aim to design tests that are
able
1. effective-the test should have high power;
2. transparent-it should not rely on keeping model training implementation details private; 3. noninvasive-it should not require Alice modify her original training data model.
In the query setting, prior work fails to achieve these three desiderata, compromising either transparency or noninvasiveness Alice's 's model (e.g., by having it memorize random binary string) [3] enables effective testing (based on whether Bob's model has memorized the canary) but requires modifying Alice's 's model. Using a small held-out test set [4] (and testing if Bob performs worse on the test data than Alice's 's training data) is only effective the test set is private (since otherwise Bob can evade detection by training on the test set). In the observational setting, we not aware of any effective tests (let alone those achieving all three desiderata). We defer a more thorough discussion of related work to Section 2.
order retain effectiveness. For example, inserting canaries into
The main insight underpinning the design of our tests is that language models tend to memorize their training data, and these memorization effects are stronger for more recent training examples. In this sense, language models exhibit palimpsestic memorization: all training examples influence the final trained model, but later examples diminish the effects of previous examples. Thus, we can test whether Bob's 's model or text derives from Alice's by designing test statistics that correlate its behavior with the ordering of examples in Alice's training run (Figure 1). In the query setting, we can directly evaluate the likelihood Bob's 's model gives to Alice's 's training examples (by evaluating his model on these examples to obtain token probabilities) and measure the correlation between example
For example, Meta's terms of service require those using their models within product to acknowledge that the product is "Built with Meta Llama 3" [2].

---

likelihood and training order. In the observational setting, we use various notions of n-gram overlap between training example and Bob's text as proxies for its likelihood under Bob's s model; we also try retraining copies of Alice's model on reshuffled data (from an intermediate checkpoint) to determine whether Bob's text has higher likelihood under Alice's original model versus the retrained copies. In both settings, if Alice has randomly shuffled her training data- as is common practice*- and Bob's model is independent of Alice's 's training run, then there is guaranteed to be no (statistically significant) correlation between likelihood (i.e., either the likelihood of Alice's training examples under Bob's model or the likelihood of Bob's text under Alice's model versus the retrained copies) and training order. Thus, tests yield provable control over false positive errors.
We formalize our problem formulation and the implementation of our tests in Section We empirically validate our tests in Section 4 using the Pythia (trained on pythia and pythi a dedupee,, the deduped and non-deduped Pile datasets used to train Pythia models) and OL (trained on OLMo, OLMo-1. 7, and OL,o-2, the Dolma and OLMo-Mix datasets) model families, as well as small-scale models we train on TinyStories [5, 6. 7]. Finally, we conclude with a discussion of key takeaways and directions for future work in Section 5. We release code and data for reproducing experiments.
2 Related Work
Our work develops methods for establishing the provenance of two types of artifacts that may derive from a language model: the model itself (in the query setting) and text (in the observational setting). We presently focus on the work most relevant to ours; see Oliynyk et al. [8], Wu et al. [9], Jawahar al. [10], and Suvra et al. [11] for a more comprehensive surveys on model and text provenance. We also discuss prior work studying properties of memorization in language models.
Model provenance. Maini al. [4] propose a method they term dataset inference that enables testing whether Bob's model is independent of Alice by measuring the difference in performance of Bob's model on Alice's training set versus a small i.i.d. held-out test set. Crucially, their method is only effective if the test set is kept private, thus precluding transparency. Though in principle it may be possible to achieve transparency by scaling up the size of the test set (e.g.. to be commensurate with Alice's 's training set), doing so would necessarily be more invasive: it would not only require Alice to plan in advance for this specific scenario when launching her training run but moreover would also limit the amount of available training data. Finally, their method does not apply to the observational setting since it requires querying Bob's 's model on the held-out test set.
A line of work under the umbrella of model fingerprinting has developed a multitude of ways to perturb a language model in some subtle way (e.g.. by planting a backdoor trigger [12, 13] or by embedding a pattern into the model's output [14, 15, 16]) so as to enable downstream identification of derivative models. Due to the perturbation step. these methods all fail satisfy our noninvasive criterion. Model fingerprinting is also inapplicable to the observational setting since it requires querying Bob's model on the fingerprint trigger.
Another line of work on model provenance develops heuristic techniques for predicting whether two models are independent or not based on the similarity of their outputs [17, 18]; however, these methods are not always reliable and thus fail to satisfy our effectivity criterion. In particular, we show in Appendix C that independently trained models trained on similar data can behave more similarly to each other than actual derivative models. Finally, we remark that recent work developing independence tests for language models based their weights is inapplicable our setting [19, 20], since we do not assume access to the weights of Bob's model.
Text provenance. The problem of detecting whether text derives from a language model has drawn considerable attention in the literature. While there exist a number of heuristic techniques for predicting whether text derives from language model [21, 22], these heuristic techniques are inexact and often unreliable [10]; furthermore, often the focus is on determining whether a text derives from any language model (i.e., versus human) rather than attributing text particular
'Language models are typically trained on randomly shuffled data; in particular, training consists of a sequence of gradient steps taken on batches of data sampled randomly from some large pool. While it is common to have distinct phases of training (e.g.. pretraining, mid-training, and post-training). so long as data is still shuffled within a particular phase (e.g. pretraining) our tests will still be applicable (by considering the ordering specifically for that phase).
'https:/ / /gi thub. www.coo//////e Ronn/thhittii www.tttttthhhhhac twwtttttittuitiiiiettttp htwpp/p/w///accoooooooooooo tracing.
3

---

language model. In contrast to these heuristic methods, our techniques provide provable control over false positives via exact p-values and enable attribution of text to (derivatives of) particular language model. Finally, we remark that inference-time watermarks [23, 24, 25, 26]-i.e. techniques for detecting text downstream by applying watermarks to generated text-are are inapplicable to our setting since we allow Bob (an untrusted party) to directly sample text himself. Other work has also shown that fine-tuning often destroys watermarks from the base model [27].
Characterizing memorization in language models. Our work crucially leverages properties of memorization in language models inspired by previous work, in particular that language models tend to memorize sequences seen in training and assign high likelihood to these sequences [28. 29]. However, with large-scale pre- training, models also forget examples seen earlier in training [30, 31]. Recent work studies the combined effects of memorization and forgetting by tracking which sequences occurred at a given step across training [31, 32, 33), where the sequence likelihood typically increases sharply after the first exposure and gradually decreases towards the mean. Building on top of these observations, we show that LLMs not only memorize more data seen later in training, but also, to an extent that is detectable, retain a finer-grained pre-training data order. Of particular relevance to our observational setting is that language models tend to regurgitate memorized phrases (from training data) even in contexts different from the ones seen in training [34, 35, 36, 37, 38), making traces of training data and order detectable from samples as well.
3 Methods
3.1 Problem formulation and testing framework
Let X' be the vocabulary. We define language model j E A(X*) as a distribution over strings of text. We abstract training algorithm A (TT) as distribution over transcripts a E 'T, which capture the full execution trace of training run. We adopt the following setup:
Alice runs a training algorithm that produces transcript a, i.c., a A:
2. Bob produces an artifact 33 and
3. Alice tests whether 1 B.
The type of artifact determines the problem setting. In the query setting, Bob produces a language model 3 E A(X²); ); in the observational setring. Bob merely produces text 3 E X'*, Note that the observational setting harder than the query setting since it provides strictly less information to Alice. In particular, if Alice has Bob's model then she can always generate text (given any prompt of her choosing) from the model herself. We will disambiguate the two cases by using M4 E A(x*) to denote Bob's model and _ E X' to denote Bob's text.
Throughout the remainder of the paper, we equate a transcript with a set of strings indexed by their training order, i.e., a {(xx, 44) (x* x [N])" for n, N E N, since this is the only part of the training outcome relevant to our tests. We assume that Alice trains on randomly shuffled data, producing a shuffled transcript (Assumption Al).
Assumption A1. transcript a = {(x( t)) E (X" x [N])* is shuffled if for any permutation a: [N] + [N] the random variables ((x), (4) and (ta))) are identically distributed.
Algorithm abstracts a framework for obtaining exact p-values from arbitrary test statistics under Assumption A1, thus enabling provably exact control over false positive errors (Theorem 1). We will give concrete instantiations of various choices of test statistics for both the query and observational settings in Sections 3.2 and 3.3 respectively. A good test statistic should produce low p-values when Bob's artifact is not independent of Alice's transcript; in particular, we would like Φ6 33) to be abnormally large when a and are not independent. Typically, we do not actually run Algorithm in practice; rather, we efficiently simulate it (either exactly or approximately, depending on the test statistic) for large m to enable obtaining small p-luuues at no additional computational cost.
Theorem 1. Let A satisfy Assumption AI, Let a ~ A and 3 1 a. Then the output of Algorithm I is uniformly distributed over ((i + 1)/(m + 1))
Proof. From our assumption A, it follows that the collection {az )m comprises m exchangeable copies of a. The independence of a and 8 thus implies {(ay, }) comprises m exchangcable

---

Algorithm 1: Obtaining p-values from arbitrary test statistics
Input: Transcript a = [[x", 44)]=1+ artifact
Parameters: test statistic Φ; number of permutations
Output: p-value p E 1]
for j 1,...,7. do
jj Uni
+ [N];: 0j- = (1)))
3
Oj - (
4 p m+1 (1 Ej= 160 < Φ(a, ())) // break ties randomly
5 return p
copies of (a, 3). Because we break ties randomly, by symmetry it follows that Φ(a, 3) will have uniform rank among {Ør { and thus P is uniformly distributed over {
[++1 Im
mm+1 1j=0-
In practice, generally do not the full transcript, which may comprise billions of training examples; instead, to save time we typically randomly subsample elements of the transcript (without replacement) and run tests using the subsampled transcript. So long the subsampled transcript satisfies Definition Al (which will be the case if Alice shuffled her data and we take a random subsample), our tests remain valid.
3.2
Query setting
In the query setting, we assume Alice can directly obtain log-likelihoods from Bob's model μ4g for each training example. (Recall Bob's artifact ,3 jas in the query setting.) She can then correlate these log-likelihoods with the ordering of examples in the transcript independence. In particular, with denoting the Spearman rank correlation coefficient [39), let
{(x', f.)) } to test for
query ax MB) == pl {log pe(x') (t+)i.))
(1)
We expect later-seen training examples to have higher likelihood according to Bob's model derives from Alice's training run, so we expect the statistic Qquery to be positive if Bob's model is not independent and close to zero it is independent of Alice's training run.
The null distribution of the Spearman correlation coefficient is a known quantity particular, if 1 My then á(a. (4a) from equation (1) follows a t-distribution with n 2 degrees of freedom. Thus, in our experiments instead of explicitly running Algorithm 1, we instead directly convert ó to a p-value using the closed form cumulative distribution function for the t-distribution. This enables us to obtain extremely low, exact p-values without being computationally bottlenecked.
We can increase the power of test by controlling for natural variation in likelihoods (some texts are inherently less predictable than others) using an independent reference model foo In particular, let
Querry (a, uo)) := p (log (Po(x')/H((()))))) ()))" (t:
(2)
The idea is to reduce the variance of the test statistic by subtracting the log-likelihood of the independent reference model. We can also regress the log- ligeliikl of Bob's model onto the reference model (and other features of text that may correlate with likeehooodo but are independent of Alice's 's training order) better control for natural variation in text likelihood.
Finally, we can extend all of the test statistics in this section to the case where Alice only receives token predictions from Bob (instead of token probabilities) by first estimating token probabilities from Bob's predictions and then using these probabilities to apply the test statistics.
3.3 Observational setting
In the observational setting, we make no assumptions on how Bob generated the text x that Alice observes. (Recall Bob's artifact 3 xy in the observational setting.) Even if Bob did generate the text using a language model, we do not have access to the model and thus cannot compute its likelihood on her training examples as in the query setting. Instead, what we can do compute the likelihood of Bob's ' text (or proxies thereof) under collection of language models that we construct
5

---

to elicit correlations between Bob's ' text and Alice's is example ordering. We take two approaches to constructing the collection by training models on either partitions or shuffles of Alice's transcript.
For the first approach (Algorithm 2, or @o ) we train language models on different contiguous partitions of Alice's original ordered training data and correlate (some measure of) the likelihood of Bob's text under these models with their relative ordering. Motivated by previous work observing that language models sometimes regurgitate text verbatim from their training data [28, 29] and in n-gram
particular are more likely to regurgitate text more recently in training [31, 32, 33], models and take to be either the likelihood of Bob's 's text under the model or the number of exact n-gram matches among Bob's s text with the n-gram index underlying each model.
Algorithm 2: Training models on partitioned transcript (Pahs)
part
Input:
Transcript a {[x',, (4) text x E X
Parameters: number of models k; metric X
Output: test statistic do (3)
1 Sort examples x by indices
Split sorted into 21, 2g contiguous partitions and train models M1.... MM partitions
return p( [x(sx, x)), (5))-1)
For the second approach (Algorithm 3, dsbull),, train neural language models different shuffles of Alice's original ordered training data to determine whether Bob's text has abnormally high likelihood under the original ordering. In practice, (so that μi is Alice's model itself); we also repeat only small fraction of her training run (e.g., the last 1-10%) ) to reduce computational costs. In our experiments, we take x(4;, to either be the likelihood of Bob's text r under the model μ4i itself or the likelihood after finetuning each μu Bob's text, with the motivation of the latter approach being to improve the robustness of our test to Bob finetuning otherwise modifying Alice's 's model.
the same model architecture Alice
Algorithm 3: Training models on shuffled transcript (hsbulf))
Input: Transcript a = {(xx', T->>>i text x E X'
Parameters: number of models k; metric X
Output:
test statistic désluff (T,
Sort examples by indices
2 Train model μo on x (in sorted order) and models (1, ... on independent reshuffles of x
3 μ (1/k) (1 x(u;, return (x(Po, x0) 11) /
Va/(k- 1)) =*) =1)
Unlike the previous statistics, deshll does not apparently have closed form output distribution under the null hypothesis. Obtaining exact p-values from dishul'' is possible via Algorithm I but would require retraining many models. Instead, in our experiments we report approximate p- p-values by Shhff
treating the output of oobs as z-score, and as a sanity check we report the degree to which these p-values empirically deviate from exact p-values under the null.
4 Experiments
4.1 Setup
Transcript (a). We use the ordered pretraining data from various open-source language models as Alice's transcript. We consider five families of models, each corresponding a different pretraining dataset ranging from 300B to 4T tokens: (1) pythia: The Pile dataset used for training Pythia models [5]; (2) pyduie-deduped: The deduped version of The Pile used for training Pythia-deduped
models; (3) OLMo: Dolma v1.5 dataset used for training OLMo models [6]: (4) OLMo-1. 7: Dolma
v1.7 dataset used for training OLM0-0024 and OLM0-0724 models; and (5) OLMo OLMo-Min used for training the stagel of OLMo- 1124 models [7]. Additionally, we use TinyStories [40] to train small-scale models for ablations that would otherwise prohibitively expensive, such as trying multiple epochs of training. We subsample sequences from each dataset to conduct our test (see Appendices A and regarding sampling details for the query and observational settings respectively).

---

Models fine-tuned from
10"
OLMo
OLMo-1.7
++
OLMo
10-2
Transcrpp
36
10
C.No-1.7
23
5
0
0
p-value
10
OLMo
-4
44
42
23
+3
10-4
10-20
7b- basft-bulu2
mn-insruct-ultro
7D Ap-uR-0444-AM A--uuluu
7 7biinstruct-0422 -uttraur
78:st-0722 78-st-0724-tun 07244-buuu2
7h: 7b-inatract 724-aittra
7he 7h- womp--tommmin doimino
7b Po-dpe-pu prefeereeeeecce
7D-sit-bulu3
7h 7h-instruct-iie
7D-ft-mait
76 70-grpo-math
<0
10'
10"
10'
#Sampled Takens
(a) Evaluating a model on own transcript.
(b) Evaluating OL derivative models.
+-
Models fine tuned from pythia checkpoints +
Models fine-tuned from deduued checkpoints
pythia
15
19
124
26
90
Transcript
deduped
-14
-136
55
-11
65
-69
-08
128
-31
41
-19
-30
-2
-09
-114
.07
epoch1
-33
-12
-23
-50
1.4b-sft-doce
48 Alt-sft-l fin
1.40 Ro--ff-ll Wh
1.40 th
4140-ftt ftt-ft-tulu
4b--dpo Ah
6.90 sff-hh
9.b-sft-tull tull
6.9t-aft 1.0b-aft-uttaeee ultraccutt
po--pp--in hn
6,90 pao-ppo--t hh
120 12b aib-dpo-in hh
oft-hh
1.4b-aft.sl sharettt
it pltesk
1.40 -b-stt-syn
1.4b Ab-stt-uddr
6.96- fll-ft--mm blass
6.9h-ft 6.9b-ft-clinn h-cilnical
6. 2.9th-ft pilesk
6. 6.9b-st-an alpaca
6. 49b-sft-iiä
8.9b art----t oasst
& 2.9---t-syn 6. 98-
ttdr
1 oaist
teer
ab-att-sym
12b-ppo oasst
6. 96 dpo
(c) Evaluating Pythia derivative models. For pythia-deduped, we try restricting queries to each epoch.
We report results (all valuus in log1o) from d for 40 Pythia and OLMo derivative models in the query setting. We label each derivative model as (model (azz)-(poste tranttngainng method} wwwosppptttttee -training dataset).
Figure 2:
Artifact (3). For Bob's artifact 8, we use models that derive from one of the five families above or from our own TinyStories models. We vary the following factors when selecting the derivative models: (1) number of tokens on which the derived model has been finetuned, and (2) the type of finetuning, including supervised fine-tuning, preference optimization, and model souping.
4.2 Query setting
Test statistic (e). We report p-values using Qquery and query as we describe in Section 3.2.
Reference model (fO). To implement models pre-trained on an (almost) identical training dataset as Alice's, s, but using different order, such as using Pythia models trained on the non-deduped version of Pile as references for Pythia models trained on the deduped version; (2) models pre-trained on other training datasets, c.g., using OLMo models references for Pythia models; and (3) an ensemble of models from (1) and (2). In particular, fg must be known to be independent from I', in order to correctly test the indepenedence of μ and T. We present ablations on the type of reference model in Appendix A.5, and we find type (1) reference models yield the lowest p-values.
efet query- we consider three types of reference models M4o: (1)
Results. Figure 2a displays the p-values we obtain over the number of tokens we query from the model (i.c.. the size of the subsampled transcript multiplied by the number of tokens per example) when testing model on its own training data (supposing Bob is using Alice's model without modification). For each of the five families, we use up to IM 64-token sequences randomly sampled from the first epoch as our transcript a and evaluate the 7B-scale model checkpoint at the end of the first epoch. Crucially, though the actual correlation is typically small (between 0.001 and 0.1), the accuracy of the test increases with the number of queries; in other words, the effect size is small but statistically significant. Both Qquery and df obtain extremely small p-values with enough queries, but using a reference model allows us to reduce the query amount by an order of magnitude while still obtaining comparable p-values. Incidentally, our results strongly suggest that pythia-2 2. 8b deduped, 6 model commonly used by the rescarch community to study training
deduped
7

---

dynamics, was actually trained on the non-deduped version of Pile (with p-value of 10 60) rather than the deduped version, contrary to its documentation.
Based on these results, we fix the number of tokens we query to be 100k for pythia and 5M for OLMo, OLMo-1.7, and 0LMo-2. In Figures 2b and 2c, we respectively test 12 HuggingFace models that are fine-tuned from an OLMo checkpoint and 29 models that are fine-tuned from a Pythia checkpoint. These models cover common post-training techniques including supervised finetuning, preference optimization, and model souping (i.e., averaging multiple finetuned model weights); see Appendix A.2 for details. For Pythia finetunes, whose base models were trained for between one and two epochs (with independently shuffled data for each epoch). we test using random subsamples of the full transcript as well as each epoch's transcript. For all but four of these finetunes, we obtain p-values of at most 10 8 with at least one or both epochs (we can straighforwardly correct for multiple-testing by multiplying the p-values by 3). Notably, the four exceptions are all models at the smallest scale we test (1.4B), and the three models with p-values larger than 10-5 all incur significantly higher pretraining loss (ranging from 4.5 to 5.6) relative to their underlying base model (3 0.1); in these cases, Bob evades detection at the cost of significantly degrading the quality of his model relative to Alice. For OLMo finetunes, we obtain p-values less than 10 in every case and less than 10-13 in all but three cases. In all cases, we expect (based on Figure 2a) to be able to obtain even lower p-values with more queries.
Figure 2c already shows we are able to obtain low p-values for various Pythia derivatives by correlating their log- likelihood with the transcript of the first training epoch despite the fact that these models underwent a second (partial) epoch of training on reshuffled data. To further evaluate the effectiveness of our test on models trained for multiple epochs, we conduct a similar test with the OLMo and OLMo-1.7 7B models by using the first epoch's transcript to test the final checkpoints, which have respectively continued pretraining from the first epoch for 544B and 2300B tokens. We obtain a p-value below 10 10 using 64M query tokens for OLMo and below 10 using 256M tokens for 0LMo-1.7. 7. Furthermore, to stress test the effectiveness of our methods on models trained for many epochs, in Appendix A.4 we try training our own models on the Tinystories dataset for up to 10 epochs (reshuffling the data for each epoch). We observe that the final models' log- ligeeliddoood exhibits statistically significant correlation with the transcripts of many (though not all) previous epochs; like a palimpsest, the ordering of previous epochs are inscribed into the model along with later ones.
Finally, we conduct additional experiments to simulate the case where Alice only receives next token predictions from Bob instead of token probabilities (e.g., supposing Bob maintains an API that returns text responses to arbitrary prompts) in Appendix A.6. We find we are able to obtain low p-values even when using just a single query to (roughly) estimate token probabilities, thus incurring minimal overhead in this more challenging setting. We perform a cost analysis of running our tests in Appendix D for various existing model APIs.
4.3 Observational Setting
Recall we consider two approaches in the observational setting based on partitioning (o or reshuffting dbs ) Alice's original training data, training language models on these data and evaluating these models on Bob's text.
4.3.1 Partitioning the transcript
Test statistic (o). We report p-values using ddeet (Algorithm 2) as we describe in Section 3.3. We primarily experiment with a version of spart using n-gram models (n = 8) wherein we let X count the number of exact matches among Bob's 's text with the n- n-gram index underlying each model and let k be the total number of minibatches in Alice's training run; in other words, we count the number of matching n-grams among Bob's text per each minibatch of training examples and correlate these counts with the minibatch order. See Appendix B.1 for additional implementation details.
Sampling text. Because the models we experiment with have limited context windows or are otherwise incapable of generating long, coherent texts, to obtain Bob's text x we independently generate short texts then group these texts together (i.e., we treat x as a collection of distinct documents and use |ar2| to denote the total number of tokens across all documents). We generate these texts as continuations of prefixes from The Pile [41] dataset and vary the sampling temperature. Each prefix has 16 tokens and each continuation is most 128 tokens.
8

---

Results. We experiment with Pythia models, whose g-gram training index (the dictionary mapping n n-grams to batches) we can build with reasonable disk space (1.4TB) and query efficiently with the infini-gram code base [42]. We index a subsample of the transcript for the first 100K training batches and run our test on text we sample from subsequent training checkpoints up to the full 143K batches (treating the later checkpoints as finetunes). Because the test is costly to run for large numbers of tokens, we do not evaluate the full set of Pythia derivative models from earlier.
step
x01 640K
1.28M
3.20M
6.40M
12.8M
19.2M
100K
3.0 10
2.5 x 10 2
2.2 X 10 2
2.0 X 10
6.3 X 10
2.6 x 10"
",,, 10r))
(1.2 5.3x0
LixI)))
(82%
(1.41101 1079)
110K
3.7 x 10-1
x x 10-1
1.3x x 10-1
9.3 x 10-2
6.9 x 10-2
4.4 10-2 (3. 10-2.000.600 10-2)
(L.7X 10"1 7.1 10-1)
06.x 2..
(8
in-)) (29 11x
120K
1.5 x 10
3.0 x 10
----
3.9 x 10 1-
8.4 10
7.4 10-2
8.7 x 10"2"
(1.9 10" 5.9x 10-1)
(1.3 10..001...11 5.1 107) (28x
L7x 10-1) (12
(12x 12x (orl) (8.x 14x 1or1)
130K
1.7 X 10
5.1
10
4.7 x 10
1.7x 10
1.0 x 10
4.4 X 10
(3.3 x 10 45x 15 (0.6%
0.7% 10 85x 1 (10010
42x 10 (88 (18810 15x 10 (11
1 16
143K
10
5.6
10
3.9
X 10
x x 10-1
4. x 10
2.9 x 10
12. 10
10
0.3 10 10 B2X
12 67 124X10 10 (28 .668 16 (20
1,09 10
Table I: We report median p-values and interquartile ranges over 10 trials from applying e to text sampled from pythia-6. 1.....9bddee 9b-deduped checkpoints. Because sampling is expensive, for each trial we resample generations with replacement from an initial sample of 800K generations (i.e., 25.6M tokens), so the reported ranges are likely narrow.
In Table 1, even when testing the checkpoint corresponding to the end of the transcript (i.e., supposing Bob uses Alice's model directly) we require several hundred thousand tokens before we begin to observe consistently low p-values. These results suggest it is infeasible to use this test to detect when Bob is generating short snippets of text (e.g., social media posts) using Alice's model. However, we posit the test may be useful for larger-scale, ecosystem level analyses such as allowing Alice to estimate the total fraction of text on a social media platform that derives from her model. Moreover, the test is able to withstand a substantial amount of finetuning: with enough tokens, we observe decreasing p-values even after 30K steps (i.e., when Bob continues training Alice's 's model for up to 30% of its original pretraining budget).
We report additional results in Appendix B, varying the sampling parameters and token count. We find the p-values we obtain are somewhat sensitive to temperature. Notably, the power of our test diminishes substantially when generating text unconditionally with low temperature, which we attribute to low diversity among the generated texts.
4.3.2 Shuffling the transcript
Test statistic (o). We report p-values using dishuff (Algorithm 2) as we describe in Section 3.3. We use the same model architecture Alice and continue training from intermediate checkpoint. We let be the likelihood of Bob's ' text under each model, either with or without finctuning the model on Bob's text. See Appendix B.2 for additional implementation details.
Sampling text. As before, to obtain Bob's 's text 8 we independently generate short texts then group these texts together. We generate these texts as continuations of prefixes from the TinyStories test set. Each prefix has 20 tokens and each continuation is at most 32 tokens long.
Results. Running dstual requires retraining many models on reshuffled data, which is costly at scale. Thus, we conduct most of our experiments with small (3M) models we train ourselves on 500K documents from the Tinystories dataset. We vary three main factors: 1) the amount of data on which we retrain (ranging from the last 10-50K documents-iac i... the last 2-10% of training); 2) the amount of data (also sampled from the TinyStories dataset) which Bob finetunes; and 3) the amount of text Bob generates. Unlike the tests in the previous section, the output of shulf is not an exact p- p-value but rather z-score; nonetheless, for the sake of comparison to previous results, we report approximate p-vauues we obtain by converting z-scores to p-values under the assumption that the score distribution is normal. We defer the full details of our training and finetuning setup to Appendix B.2. We demonstrate the validity of our approximate p-values by empirically estimating their null distribution in Appendix B.5.
Even with retraining on as little as the last 2% of Alice's pretraining data, we are able to obtain p-values less than 10 from as few 320 tokens in Bob's text (Figure 3, left panel). Moreover,

---

10"
10:10
10
10 14
92
10.19
10
10
14
Icheras = 32 10
Idiana - 52 +0
10
takene 22 100
10
Wians 32 100
tskeno 32 1000
Mioo's - 22 1000
10
0.00
0.25
0.50
0.75
1.00
1.25
1.50
1.75
2.00
armuunt of relrainng (% of protoairing)
amnount of Ereelating( of prefoiming))
We report approximate p-values from applying &dtull to TinyStories models, varying the number of tokens in Bob's text. (Left) We vary the amount of data we use to retrain the test models; we let X be the log-likelihood of Bob's text under each model (no finetuning) and assume Bob uses Alice's ' model without modification. (Right) We fix the amount of retraining at 2% of Alice's pretraining data and vary the amount of data which Bob finetunes Alice's s model; try Pobs with (solid) and without (dashed) finetuning on Bob's text.
Figure 3:
finetuning the retrained test models on Bob's 's text before evaluating their log- ogg-likelhhod substantially improves the robustness of the test to Bob finetuning Alice's model (Figure 3, right panel); however, we require substantially more tokens in order to see a tangible benefit, and even with 32000 tokens both versions of the test lose power once Bob finetunes on as many documents as 1% of Alice's pretraining run. We include additional sweeps and ablations in Appendix B.4; notably, we find retraining the test models more tokens improves the robustness of the test to finetuning.
To extend beyond toy experiments, we leverage existing OLMo checkpoints phase of their pretraining run tnaatriieed on different random shuffles of the same 50B tokens (representing roughly 1% of the pretraining data). In particular, there are three such checkpoints for both -2Mo-2-0425- 1B and OLMo 1B. We use these checkpoints to simulate running shhuff with A: = 2 (and no finetuning on Bob's text) by generating text from one of the checkpoints (using the same setup and prompt distribution as in the previous experiment on TinyStories) and treating the other two checkpoints as test models. Given 640 tokens of Bob's ' text, we obtain median p-values (with interquartile ranges parenthesized) of 6.25 x 10 (6.44 10 1.44 x 10-1) at the IB scale and 1.44 10-3 (1.01 x 10 6 6,00 x 10 2) at the 7B scale; using only two test models makes the p-values more volatile. Additional results are in Appendix B.6.
the cooldown
obs
5 Discussion
We develop exact tests that enable language model providers and developers to determine: 1) whether another language model is trained independently of their model and 2) whether text is generated independently of their model. Our tests are transparent and noninvasive by design, and we evaluate their effectiveness with a number of models across a range of scales and training recipes. We presently remark on some limitations and directions for future work.
Recall that we assume the role of Alice, out our tests. In practice, model developers (including developers of open-weight models) may be reluctant to disclose their training data due concerns around copyright liability competitive advantage (of course, any model developer can still use our tests internally). To enable independent third-panc partt verification of test results under these circumstances, we propose running our tests using subset of the transcript that the model developer is willing to disclose (e.g., the relative ordering of Wikipedia documents in the pretraining data).
have access Alice's training data in order to earry
All our tests are somewhat costly to run, either because they require many tokens from Bob to be effective or retraining collection of language models. Bringing these costs down is an important direction for future work. Particularly in the observational setting, where Alice cannot simply spend more queries to obtain more tokens, designing more lightweight tests than d with similar token complexity is an important open problem.
Finally, beyond our specific problem setting, our experiments yield new insights into memorization in language models. Exploring the implications of these findings on issues of privacy and copyright and for designing more effective models exciting direction for future work.
10

---

Acknowledgments
We would like to thank Jiacheng Liu and Pete Walsh for their help on reconstructing the OLMo training data. This research is supported in part by grants from Google and Open Philanthropy.
References
[1] Merriam-Webster Definition of palimpsest. In Seteriaam-Webste.coom dictionary. URL https:
//www //www.merria wwwrimriaat am- webster com/ ht//dict.a htt//diiiionnrr/ diciionnrrypallinssesss.
[2] Meta. Meta Llama 3 Community License Agreement, 2024. URL https://ww/t 11ama. com/ 11ama3/1 hat/////ccceee/ee/
[3] Jiashu Xu, Fei Wang, Mingyu Ma, Pang Wei Koh, Chaowei Xiao, and Muhao Chen. Instructional Fingerprinting of Large Language Models. In Kevin Duh, Helena Gomez, and Steven Bethard, editors, Proceedings of the 2024 Conference of the North American Chapter of the Association for Computational Linguistics: Human Lan Language Technologies (Volume 1: Long Papers), pages 3277-3306, Mexico City, Mexico, June 2024. Association for Computational Linguistics. doi: 00.18653/v1/2024 120.://////2222.aaaaooooggg 180. URL https: /atp//aaaaaaaooooyyoooooo org/2024. naacl-long. 180/.
[4] Pratyush Maini, Mohammad Yaghini, and Nicolas Papernot. Dataset Inference: Ownership Resolution in Machine Learning. International Conference on Learning Representations, 2021. URL https: bettp::/oppereeviee net/
[5] Stella Biderman, Hailey Schoelkopf, Quentin Gregory Anthony, Herbie Bradley, Kyle O'Brien, Eric Hallahan, Mohammad Aflah Khan, Shivanshu Purohit, USVSN Sai Prashanth, Edward Raff, et al. Pythia: A Suite for Analyzing Large Language Models across Training and Scaling. In International Conference on Machine Learning, pages 2397-2430. PMLR, 2023.
[6] Dirk Groeneveld, Iz Beltagy, Evan Walsh, Akshita Bhagia, Rodney Kinney, Oyvind Tafjord, Ananya Jha, Hamish Ivison, Ian Magnusson, Yizhong Wang, Shane Arora, David Atkinson, Russell Authur, Khyathi Chandu, Arman Cohan, Jennifer Dumas, Yanai Elazar, Yuling Gu, Jack Hessel, Tushar Khot, William Merrill, Jacob Morrison, Niklas Muennighoff, Aakanksha Naik, Crystal Nam, Matthew Peters, Valentina Pyatkin, Abhilasha Ravichander, Dustin Schwenk, Saurabh Shah, William Smith, Emma Strubell, Nishant Subramani, Mitchell Wortsman, Pradeep Dasigi, Nathan Lambert, Kyle Richardson, Luke Zettlemoyer, Jesse Dodge, Kyle Lo, Luca Soldaini, Noah Smith, and Hannanch Hajishirzi. OLMo: Accelerating the science of language models. In Lun-Wei Ku, Andre Martins, and Vivek Srikumar, editors, Proceedings of the 62nd Annual Meeting of the Association for Computational Linguistics (Volume Long Papers), pages 15789-15809, Bangkok, Thailand, August 2024, Association for Computational Linguistics. doi: 10.18653/v1/200 acl-long..ll....... 841. URL https: /tttt:::///caaateee http:///aaaaaahohooeooeo org/2024. acl-long. 841//./...........
[7] Evan Pete Walsh, Luca Soldaini, Dirk Groeneveld, Kyle Lo, Shane Arora, Akshita Bhagia, Yuling Gu, Shengyi Huang, Matt Jordan, Nathan Lambert, Dustin Schwenk, Oyvind Tafjord, Taira Anderson, David Atkinson, Faeze Brahman, Christopher Clark, Pradeep Dasigi, Nouha Dziri, Allyson Ettinger, Michal Guerquin, David Heineman, Hamish Ivison, Pang Wei Koh, Jiacheng Liu, Saumya Malik, William Merrill, Lester James Validad Miranda, Jacob Morrison, Tyler Murray, Crystal Nam, Jake Poznanski, Valentina Pyatkin, Aman Rangapur, Michael Schmitz, Sam Skjonsberg. David Wadden, Christopher Wilhelm, Michael Wilson, Luke Zettlemoyer, Ali Farhadi, Noah A. Smith, and Hannaneh Hajishirzi. 2 OLMo furious (COLM's version). In Second Conference on Language Modeling, 2025. URL http://pppaareeeeeeeeeeeeerrrdaa
[8] Daryna Oliynyk, Rudolf Mayer, and Andreas Rauber. I Know What You Trained Last Summer: Survey on Stealing Machine Learning Models and Defences. ACM Computing Surveys, 55:1
41, 2022. URL https: //api www.stitttttttoleeeeee
249848185.
[9] Junchao Wu, Shu Yang, Runzhe Zhan, Yulin Yuan, Lidia Sam Chao, and Derek Fai Wong. A survey on LLMM-generated text detection: Necessity, methods, and future directions. Computational Linguistics, 51(1):275-338 March 2025. doi: 10.1 162/colii a 00549. URL.
org/ 2025. cl-1.8/.

---

[10] Ganesh Jawahar. Muhammad Abdul-Mageed, Laks Lakshmanan, and Dujian Ding. LLM performance predictors are good initializers for architecture search. In Lun-Wei Ku, Andre Martins, and Vivek Srikumar, editors, Findings of the Association for Computational Linguistics: ACI. 2024, pages 10540-10560, Bangkok, Thailand, August 2024. Association for Computational Linguistics. doi: 118653/v1/2000 http:////....iinnaaall..... findings-acl.627.
URL https:
//aclanthology.oo org/ 2024. findings-acl. ac1.677/..
[11] Soumya Suvra, Ghosal, Souradip Chakraborty, Jonas Geiping, Furong Huang, Dinesh Manocha, and A. S. Bedi. A Survey on the Possibilities & Impossibilities of Al-generated Text Detection. Transactions on Machine Learning Research, 2023. URL https: http:::s///xxxv.ooo org/ ab300 15264.
[12] Jiashu Xu, Fei Wang, Mingyu Ma, Pang Wei Koh, Chaowei Xiao, and Muhao Chen. Instructional fingerprinting of large language models. In Kevin Duh, Helena Gomez, and Steven Bethard, editors, Proceedings of the 2024 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies (Volume Long Papers), pages 3277-3306, Mexico City, Mexico, June 2024. Association for Computational Linguistics. doi: 10.18653 180/.
180. URL https: http:///aaaaaaaaaaaaeee org org 2tg/2024.aacc-11oo... maacl-long.
[13] Jie Zhang. Dongrui Liu, Chen Qian, Linfeng Zhang, Yong Liu, Yu Qiao, and Jing Shao. REEF: Representation Encoding Fingerprints for Large Language Models. In The Thirteenth International Conference on Learning Representations, 2025. URL httpps//oppereevee: net/forum? i id=SnDmPkOJOT.
[14] Xuandong Zhao, Yu-Xiang Wang, and Lei Li. Protecting Language Generation Models via Invisible Watermarking. In Proceedings of the 40th International Conference on Machine Learning, ICML'23. JMLR.org, 2023.
[15] Xuanli He, Qiongkai Xu, Yi Zeng, Lingjuan Lyu, Fangzhao Wu, Jiwei Li, and Ruoxi Jia. CATER: Intellectual Property Protection Text Generation APIs via Conditional Watermarks. In Advances Neural Information Processing Systems 35, 2022.
[16] Xuanli He, Qiongkai Xu, Lingjuan Lyu, Fangzhao Wu, and Chenguang Wang. Protecting intellectual property of language generation APIs with lexical watermark. In Proceedings the Thirty-Sixth AAAI Conference on Artificial Intelligence, 2022.
[17] Ivica Nikolic, Teodora Baluta, and Prateck Saxena. Model Provenance Testing for Large Language Models, 2025. URL http:://t:///aata /tt////rxiivvoogaa////////// 0000000.00.
[18] Heng Jin, Chaoyu Zhang, Shanghao Shi, Wenjing Lou, and Y. Thomas Hou. ProFLingo: A Fingerprinting-basse -based Intellectual Property Protection Scheme for Large Language Models. In 2024 IEEE Conference on Communications and Network Security (CNS), pages 1-9, 2024. doi: 10. 109/CNS62487 2024. 10735575.
[19] Sally Zhu, Ahmed M Ahmed, Rohith Kuditipudi, and Percy Liang. Independence Tests for Language Models. In Forty-secome International Conference on Machine Learning. 2025. URL net/f orum? wiw:idmmmmmmmmmaa
https://openrevias
[20] Boyi Zeng. Lizheng Wang, Yuncong Hu, Yi Xu, Chenghu Zhou, Xinbing Wang, Yu Yu, and Zhouhan Lin. HuRef: HUman-REadabe Fingerprint for Large Language Models. In The Thirty-eighth Annual Conference on Neural Information Processing Systems, 2024. URL https://openrevieer wwtpeerreveeeee/e/ffooeeeeeeeeee feet:fforum FeiiiRilggEeesseee
[21] Eric Mitchell, Yoonho Lee, Alexander Khazatsky, Christopher D Manning, and Chelsea Finn. DetectGPT: Zero-Shot Machine- Generated Text Detection using Probability Curvature. arXiv preprint arXiv:2301.11300 2023.
Machine-Generated
[22] Abhimanyu Hans, Avi Schwarzschild, Valeriia Cherepanova, Hamid Kazemi, Aniruddha Saha, Micah Goldblum, Jonas Geiping, and Tom Goldstein. Spotting LLMs With Binoculars: Zero- Shot Detection of Machine-Geenerate Text, 2024. URL. https://taaeet //tr:i/////v.//// www.wwwwwwwwaa./////o//......aae org/abs/2400
12070.
12

---

[23] Scott Aaronson and Hendrik Kirchner. Watermarking GPT Outputs, 2023.
[24] John Kirchenbauer, Jonas Geiping, Yuxin Wen, Jonathan Katz, Ian Miers, and Tom Goldstein. Watermark for Large Language Models. In Andreas Krause, Emma Brunskill, Kyunghyun Cho, Barbara Engelhardt, Sivan Sabato, and Jonathan Scarlett, editors, Proceedings of the 40th International Conference on Machine Learning, volume 202 of Proceedings of Machine Learning Research, pages 17061-17084. PMIR, 23-29 Jul 2023. URI. https: / /pproceedings.oo mal.eeeedinnmmm..........m.. prwss/eeaaae hataas/v222///5 200022/krrchebauee2255 html.
[25] Miranda Christ, Sam Gunn, and Or Zamir. Undetectable Watermarks for Language Models. In Shipra Agrawal and Aaron Roth, editors, Proceedings of Thhhryy Seventh Conference on Learning Theory, volume 247 of Proceedings of Machine Learning Research, pages 1125-1139. PMLR, 30 Jun-03 Jul 2024. Uttpst:s:: http:///pooeeedeeeeeee boggs.ml.. mlr. pprss//v2ae christt4aitttt.. html.
[26] Rohith Kuditipudi, John Thickstun, Tatsunori Hashimoto, and Percy Liang. Robust Distortion- free Watermarks for Language Models. Transactions on Machine Learning Research, 2024. ISSN 2835-8856. URL https: //wxxvaavvvv////////a///aa//aaaa .rrg abs 2207. 15593.
[27] Chenchen Gu, Xiang Lisa Li, Percy Liang, and Tatsunori Hashimoto. On the Learnabllity of Watermarks for Language Models. The Twelfth International Conference on Learning Representations, 2024. URI. btttps://ppenreiiee net/ffrumm httrrmmmdddkkkoNNw.llll.
[28] Nicholas Carlini, Chang Liu, Úlfar Erlingsson, Jernej Kos, and Dawn Song. The Secret Sharer: Evaluating and Testing Unintended Memorization in Neural Networks. In 28th USENIX Security Symposium (USENIX Security 19), pages 267-284, Santa Clara, CA, August 2019. USENIX Association. ISBN
URL htttps://wwt usenix. org/conference/
[29] Kushal Tirumala, Aram H. Markosyan, Luke Zettlemoyer, and Armen Aghajanyan. Memorization Without Overfitting: Analyzing the Training Dynamics of Large Language Models. In Alice H. Oh, Alekh Agarwal, Danielle Belgrave, and Kyunghyun Cho, editors, Advances in Neural Information Processing Systems, 2022. URL derrm?id=u3vEuRrOO
net/
[30] Matthew Jagielski, Om Thakkar, Florian Tramer, Daphne Ippolito, Katherine Lee, Nicholas Carlini, Eric Wallace, Shuang Song, Abhradeep Guha Thakurta, Nicolas Papernot, and Chiyuan Zhang. Measuring Forgetting of Memorized Training Examples. In The Eleventh International Conference on Learning Representations, 2023. URL id=7bJizxlKrR.
wwwprrvveeeeeeeeforuueeeee net/forum?
[31] Pietro Lesci, Clara Meister, Thomas Hofmann, Andreas Vlachos, and Tiago Pimentel. Causal estimation of memorisation profiles. In Lun-Wei Ku, Andre Martins, and Vivek Srikumar, editors, Proceedings of the 62nd Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers). pages 35616-15635 Bangkok, Thailand, August 2024. Association for Computational Linguistics. doi: https:://alaaaahlllogeee org/ 2024./222...2 acl- aol-long... 834/.
URL
[32] Hoyeon Chang, Jinho Park. Seonghyeon Ye, Sohee Yang, Youngkyung Seo, Du-Seong Chang, and Minjoon Seo. How Do Large Language Models Acquire Factual Knowledge During Pretraining? In The eiirth- Annual Conference on Neural Information Processing Systems, 2024. https://:///
httttss//openrevvieee wwwppperriie..eeeeeeeeeeeeeeeeee hett/forumidd-Teezz....
[33] Jing Huang, Diyi Yang, and Christopher Potts. Demystifying verbatim memorization in large language models. In Yaser Al-Onaizan, Mohit Bansal, and Yun-Nung Chen, editors, Proceedings of the 2024 Conference on Empirical Methods in Natural Language Processing, pages
Miami, Florida, USA, November 2024. Association for Computational Linguisties. doi: 10. 1t..1853//112222.mmm/ 2024. mamlp-main.5000 598/.
URL http:::://aclaathlloooooo org/
[34] Daphne Ippolito. Florian Tramer, Milad Nasr, Chiyuan Zhang, Matthew Jagielski, Katherine Lee, Christopher Choquette Choo, and Nicholas Carlini. Preventing generation of verbatim memorization in language models gives false sense of privacy. In C. Maria Keet, Hung Yi Lee,
13

---

and Sina ZarrieB, editors, Proceedings of the 16th International Natural Language Generation Conference, pages 28 28-53, Prague, Czechia, September 2023. Association for Computational Linguistics. doi: 10. (0.186533v1/2023..mm inl. main.3. URL. aulg-main.3/.
org/2023.
[35] Antonia Karamolegkou, Jiaang Li, Li Zhou, and Anders Søgaard. Copyright violations and large language models. In Houda Bouamor, Juan Pino, and Kalika Bali, editors, Proceedings of the 2023 Conference on Empirical Methods in Natural Language Processing, pages 7403-7412, Singapore, December 2023. Association for Computational Linguistics. doi: 10. 10.118/3/v120000 emnlp-main.4558 URL https: peettps:/acllatthhote acl.atoooottooyoooooo org/2023. emnlp-main. 458/.
[36] Michael Aerni, Javier Rando, Edoardo Debenedetti, Nicholas Carlini, Daphne Ippolito, and Florian Tramèr. Measuring Non-Adversarial Reproduction of Training Data in Large Language Models. In The Thirteenth Internationnal Conference on Learning Representations, 2025, URL bttp::://ppnreviiii http://www.....eoooooooooooooo/.
[37] Aly M. Kassem, Omar Mahmoud, Niloofar Mireshghallah, Hyunwoo Kim, Yulia Tsvetkov, Yejin Choi, Sherif Saad, and Santu Rana. ALPACA AGAINST VICUNA: Using LLMs to uncover memorization of L.LMs. Luis Chiruzzo, Alan Ritter, and Lu Wang, editors, Proceedings of the 2025 Conference of the Nations of the Americas Chapter of the Association for Computational Linguistics: Human Language Technologies (Volume 1: Long Papers), pages 8296-8321 Albuquerque, New Mexico, April 2025, Association for Computational Linguistics. ISBN 979-8-89176-18944 URL https: laat:p://acclanholoog) org/2025 naacl-long 421/.
[38] Ximing Lu, Melanie Sclar, Skyler Hallinan, Niloofar Mireshghallah, Jiacheng Liu, Seungju Han, Allyson Ettinger, Liwei Jiang, Khyathi Chandu, Nouha Dziri, and Yejin Choi. AI as humanity's salieri: Quantifying linguistic creativity of language models via systematic attribution of machine text against web text. In The Thirteenth International Conference on Learning Representations, 2025. URL /ttttp:://pppreeeee wiw..wwww.... net net/f formm? www.oromidiiii 1d=11000qq00..
[39] C. Spearman. The proof and measurement of association between two things. The American Journal of Psychology, 101, 1904. ISSN 00029556. URL http: //www//wwww// jstor org/ stable/1et 1412159.
[40] Ronen Eldan and Yuanzhi Li. TinyStories: How Small Can Language Models Be and Still Speak Coherent English?, 2023. URL https://arxiv http::////rivvrr/g/.e//o htrg/abs/220 hatpa//b////////////// 07755.
[41] Leo Gao, Stella Biderman, Sid Black, Laurence Golding, Travis Hoppe, Charles Foster, Jason Phang. Horace He, Anish Thite, Noa Nabeshima, Shawn Presser, and Connor Leahy. The Pile: An 800GB dataset of diverse text for language modeling, 2020. https:// ://rxiv org/ abs/2101. 200p/217000007. 211V::210100022:
[42] Jiacheng Liu, Sewon Min, Luke Zettlemoyer, Yejin Choi, and Hannaneh Hajishirzi. Infini Infini-gram: gram: Scaling Unbounded n-gram Language Models to a Trillion Tokens. In First Conference on Language Modeling. 2024. URL https://openreriee wttp://wwww..tettnonnneeeeeeaeee http//oruunnddlyyyyeeeellla
[43] Pinzhen Chen, Shaoxiong Ji, Nikolay Bogoychev, Andrey Kutuzov, Barry Haddow, and Kenneth Heafield. Monolingual or multilingual instruction tuning: Which makes better alpaca. In Yvette Graham and Matthew Purver, editors, Findings of the Association for Computational Linguistics: EACL 2024, pages 1347-1356, St. Julian's 's Malta, March 2024. Association for Computational Linguistics. URL https bittpp://accaaae /aaaaaathooogy........ org/2024. 2024. findings-eacl. 90/.
[44] Yizhong Wang, Hamish Ivison, Pradeep Dasigi, Jack Hessel, Tushar Khot, Khyathi Raghavi Chandu, David Wadden, Kelsey MacMillan, Noah A. Smith, Iz Beltagy, and Hannaneh Hajishirzi. How Far Can Camels Go? Exploring the State of Instruction Tuning Open Resources. In Proceedings of the 37th International Conference on Neural Information Processing Systems, NIPS '23, Red Hook, NY, USA, 2023. Curran Associates Inc
14

---

Technical Appendices and Supplementary Material
A
Query Setting Experimental Details and Additional Results
A.1 Query distribution
When sampling sequences from Alice's training dataset construct queries from her transcript vary the following hyperparameters: (I) the length of the sequence L, (2) the training step at which the sequence occurs (the last training step in case a sequence occurs multiple times in training), and (3) the start token position of the sequence in the original training example.
All three hyperparameters affect the computation of likelihood used in Qquery- Specifically, given a sequence of L tokens ... Lt,, we compute the average language modeling loss (i.e., negative log- lellikilihoodd across all tokens
= L L-1
Σ riit
- Log(u((r,,52.,) ....,,))))))
We provide the experiment details below on OLMo-7B. Overall, we observe that for a fixed number of sampled tokens (i.e., the number of sampled sequences x the length of the sequence), sampling from the first few tokens of each training example, i.e., = O, and using sequences that occur closer to the testing checkpoint provide the strongest memorization signals. As discussed in Section these results also provide insights into LLM memorization behaviors, characterizing which sequences are more likely to be memorized by the model.
p-value vs. # #Sampled Tokens
Epoch samples
Epoch samples
# Takens Sequence
1e-12
1e-9
8-
16
16
1e4
1e-1
128
128
5e+02
1e+03
2e+03
5e440
1e+04
2e+04
5e404
1e+05
2e+05
5e+05
1e406
5e+02
Ie+03
2o+03
Se+03
1e+04
2e+04
54+04
1e405
2e+05
5e+05
1e+06
Sampled Sequence
* -Samleed Sequence
Figure 4: We vary the sequence length and number of sequences in the transcript P in computing
Qquery for OLMo-7B. P-values are affected by the total number of tokens I and the training step which the sequence occurs, i.e., Epoch sequences provide more signals than Epoch 0 sequences.
Sequence length. We measure how sequence length affects our test results using o in Figure Given a fixed number of sequence samples, increasing the sequence length L leads to a smaller p-value, i.e., each column, p-value decreases as we move from top bottom. However, given fixed budget of tokens, sequence length has significant effect the p-value. We can see the diagonal trend of low p-values from Epoch samples (and slightly in the Epoch samples). Hence, the total number of tokens in is most relevant in the value of geel of sequences or sequence length individually.
Qquery rather than the number
Training step. Comparing Figure 4 Left and Right, show that samples from Epoch yield smaller p-values given the same sample size, suggesting sequences seen later in training are more likely to be memorized by the model.
Start token position. OLMo-7B uses context window of 2048 tokens, so we can ablate the position of the tokens of sampled text. We sample 64-token sequences starting at position t 0/256/512 202122/0224 0024/1136//// 1992.. Figure 5 shows that sampling sequences position 0 have the strongest memorization signals across all sampling sizes.
A.2 Finetuning results
We provide the raw results for the fine-tuned models shown in Section 4.2, Figure 2b and 2c.
15

---

10-10
10-35
10-60
o-value
10 -0 85
10-130
#Queries
200000
10 10-135
400000
600000
10-160
1000000
500
1000
1500
2000
Sequence Start Token Position
Figure 5: We vary the sample token position and number of queries in computing dest for OLMo-7B models. Sampling from the start of an sequence consistently yields smaller p-value across all sample size.
We evaluate dinerr models from HuggingFace against the pythia and pythia-deduped models different sizes. For each of the listed models, we use transcripts (training orders) of pythia, pyth pythia-dedupal
pythia-deduped Epoch and pythia-deduped Epoch 1. In Table 2,3 we evaluate Pythia model derivatives and in Table 4 we evaluate non-derivatives, such as OLMo family models. Table 5, evaluate OLMo fine-tuned models using transcripts of OLMo, 0LMo-1.7.1 OLMo-2. The transcript a consists of 100k randomly sampled sequences of length 64 for Pythia and IM sequences for OLMo (except for OLMo-1. 7, for which we use 5M sequences since the training order of the second epoch is not available. Instead we must use samples from the first epoch, which have weaker memorization effects).
p-value different training data order Epoch 0
Model Name
pythia
pythia-deduped
Epoch
Base Models Trained pythia
ElentherA//yythaa-1.. -..4b EleutherAL/pythiaa2.80 ElentherAt/pythia-..00 EleutherA///yttii--66b---- EleucherAT/pythha-112
9.6 10-24
1.5 10
9.3 10
3.3 10-1
10-42
8.2 10-1
7.8 x 10-
10-14 142
10-67
16x 10
1.3 10
10
10 184
10
4.1 10
4.8 10-1
Base Models Traiiee -athii--dppppt
9.6 10
5.4 10
9.3 x 10-18
1.8 x 10
3.6 x 10-1
1.0 10-12
2.7 x 10-10
4.4 10
Eleutherdl/pytthiae--Bb-bb
x 10
9.1 x 10
9.1 10
8.1 x 10
7.3 x 10-1
3.3 10-79
7.1 10-54
1.4 10
Elenther///gythia-6...b 9b-dedueed
7.7 x 10
1.8 x 10-17a
1.2 x 10
EleucherAt//yyttaee..bbb 6.b--dduuedd-ddoo
x 10-1
9.3 10-60
1.3 236
5.7 x 10 228
ClentherAt/pythia-122 12b-euupp
9.3 10-1
7.6 10-158
2.5 10 203
4.3 x 10
10-1
2.5 x 10-86
0.0 x 10"
3.7 10-301 901
Table We compute p-values using query for different Pythia base model variants [5].
The p-values of running Queery with the different transcripts are reported in Tables 3, 4, and 5. The blue boxes highlight p-values that less than 1
As we discussed in Section 4.2, pythia-2. siriia-2.Bb-dduupee db-dede has a non-significant p-value with the pythia-deduped training order, but a p-value of 1.2 x 10-60 with the pythia transcript, i.e., the non-deduped data used train pythia-2.. 8b, which suggests there likely mislabeling of the HuggingFace model.
16

---

p-value w.r. different training data order eythia--eduupa Epoch
Model Name
pythia
Epoch
Fine-tuned Models with Base Models Trained on pythia
her/astar/pytthia..cc lamini-
7.4 10
3.2 10
2.5 10
5.9 10
6.4 x 10-1
6.0 x 10
6.1 X 10
5.9 x 10
boabbaryppyhha--..--beeeppp. 1
3.8 x 10
5.6 10
2.9 10
8.4 x 10
1.9 10-9
1.7 10
1.0 10
4.1 10
3.3 10
3.0 10
4.3 10
4.8 x 10
onahhorrppythit--..-bbbelllpl 9b-hh =ft
9.4 10-ai
1.4 10-2
1.2x 10
9.0 x 10
4.9 10 125
2.0 10
4,2% 10
9.0 x
x 10-27
2.5 10
5.4 10
pkarrppas/ppythiis-ultrraaaa
2 10
6
5.9
10 132
6.3 10
x.9
4.2 10
1syepaap/9yytaaa---.88--pp
10
6 10
6.6 10
2.7 10
14-144
8.4 10
44x 10
7.7 10
2.6 10 161
9.0 10
5.0 10
6.0 10
Fine-tuned Models with Base Models Trained pythia-deduped
9.4 10-1
4.9 10-6
2.9 x 10-3
4.7 x 10
8,3 10-1
1.7 10-16
2,2 10-5
1.2 x 10-5
hww.aallbbbppthaaa-----eeeeeeeed syathatic-instzuet
8.3 10
2.7 x 10
9.5 10
1,5 x 10
b:2-1ibb/yythia--bb-dddeppp
-elpppeddddddd---------- tldr-sft
10
3.4 10-15
4.3 10-3
4.0 10
Jabbaalabb/gyythiee4----ddde bbb.aiuee----ddddee_____eeee
10
7.8 10-24
2.5 10-34
4.1 10
Hletberrl/gyhhhe-itteevrvroorr
10
5.1 10-137
4.0 170
6.1 x 10
deduped
ccode/Elleuthhrrl--pythssss
5.6 10
3.0 10-96
2.0 10-
3.3 10
full-=t-ccimmicca-bezz-------
9.4 10
2.3 x 10-3
1.6 X 10-9
6.1 X 10
durustte//oasst--yytaaee
8,8 10-1
2.4 10-8
1.0 x 10-20
1.4 x 10
CarperAt/ppythia-6..880 Sh-de-upeddddddddd------
8.9 x 10-1
9.0 10-56
3.4 10-129
5.8 x 10
Lambdalabsspyyhha--6. -bb-duueed-
5.3
x 10-12
2.1 10-31
1.9 x 10-31
synthetic-instzrer.
Huggiingfacll//LLttttaLLL.L
3,0 10
1.2 10-60
4.2 x 10
3.6 10
6.
tr1--ib/pytthaa-.999-
1.6 x 10-1
1.1 10-83
7.3 10-92
2.6 10-20
offline-dpo
10-1
3.8 x 10-63
1.4 10-90
3.9 x 10-5
lambdalabaa/pythia--2- 12b-hhhha---dduuppe--d
9.6 10
5.9 10-36
3.6 x 10
2.4 x 10 108
pyathetic-inetruue
(penkssistant/pythiin -20-biaa--2--stt---111-
10
7.1 10-64
2.9 10-88
2.0 10-5
2k-steps
Table We compute p-values using Qquery for different Pythia fine-tuned model variants [5, 43, 44]. Fine-tuned models are listed in the same order as in Figure 2c from left to right.
P-Value w.r.t. Different Training Data Order
Model Name
pythia
pythia-
Epoch 0
Epoch
Base Models Trained on Neither of the Two Orders
EleutherAI/pythia. hatpaa////pppiaaa..--eeeeeeeeed 9b-deduped-v0
5.9 10
x 10
8.5 10
x 10
-seed42
allenai/0LMo-7BB
7.9 10-1
5.8 x 10-1
6.4 10-1
x 10-1
>llemai/0UMMo-78-00222
7.6 x 10-1
5.7 10-1
5.9 10-1
x 10-1
2llemaa//0MMo--7-0724--1 allemaa/0UMo-22-1224-7 meta-llaaa/Kett-L1aamms
8.8 x 10-1
6,3 x 10
6.6 10
3.1 10
1.5 x 10-1
2.9 x 10-1
6.1 10
2.3 10-1
2.3 x 10-1
1.3 x 10
7.8 10
5.4 10-1
Table 4: We compute p-values using queryy for models that are not trained on pythia or sechiia-dedupdd. All p-values are above 0.1, indicting the null hypothesis is true, which matches the ground truth.
A.3 Continued pretraining results
We provide additional analysis for continued pretraining in Section 4.2 on Pythia models. Specifically, choose a base checkpoint A of pythia-deduped (c.g., the 1000100000 checkpoint) and finetune
17

---

P-Value w.r.t. Different Training Data Order
Model Name
OLMo
OLMo-1.7
OLMo-2
Fine-tuned Models with Base Models Trained on OLMo
alllenai/OLNo-7B-SFTII 7F-SFT
1.9 x 10 27
3.8 x 10
7.9 x 10-1
allenat//LXoo-7BIInsueee
1.3 10
10
9.6 10
Fine-tumed Models with Base Models Trained on OLMo-1.7
allenaa/00110-7B-0044---777
8.2 x 10-1
7.7 x 10-21
6.5 x 10-1
allenaa//LLLo-----044---Lsttrr
8.4 10
4.0 10-21
7.9 x 10
allemaai/01oo--B-00744--7777
7.6 x 10-1
4.4 x 10-6
9.6 x 10
allenaa/0LAo--7-07744---e--
8.4 10
x 10-5
7.8 10
Monetuned with Base Models Trained on OLMo-2
allena./O1Xoo-211224-8
4.2 x 10
7.0 10
5.4 10 10-15 15
allenai/0UNoo-2-1224-333
6.3 x 10
1.5 x 10
1.9 10 15
allenaa/0Uloo-2112244---D999
4.7 x 10-1
1.2 10
1.8 X 10
allenah/00MMo222124----h--e
4.9 10
1.3 10
3.1 10
2aal::rrcco0lo222-1122--------e
7.1 X 10
5.4 X 10
1.7 10
Weolocrric/0Moo-211122---- Instruct_rrut
4.8 10
2.1 10
2.2 10
Table 5: We compute p-values using query for different OLMo model variants.
checkpoint B after it (e.g., the step 20000 checkpoint), and evaluate B on A's data and ordering (only samples seen before step (0p00)00)) We compute querry 40K samples with tokens each for all valid base and checkpoint model pairs from step70000 to step140000, in Figure 6. We use pythia a as the reference model. Even after the step70000 checkpoint was trained for an additional 70000 steps, the p-value between the step70000 and step 140000 models is very small (2.23e-66). We can also as perhaps expected, that the p-values are much lower for the 6. 9b-parameter models than the 1.4b, 1b, and (0m-parameter models-i.ed the larger model memorizes its training data more.
A.4 Training for multiple epochs on TinyStories
We train a Transformer architectcr model (d_ (d_model 256, d_ffn = 512, num layers
approximately 3M parameters) the TinyStories dataset [40] for 10 epochs of 50k samples. We run deef with the Spearman correlation p-value on the training dataset between the model trained the data ordering of Epoch j for
0, 0,..... (order_ (order_epoch), The results when using N 50K and 20K samples are shown in Figure 7. Although variable, memorization effects are strong in general for least 3 epochs, and the p-value is low in general for the recent 4 or 5 epochs. We use reference model that uses the same architecture and training strategy but different training order seed.
Epoch for 0,...,9 (model, (model_epoch)
A.5 Reference model ablations
For our experiments. we have used reference models very similar to the candidate model being audited-foo example, the pythia-6.99 model as the reference model for testing pythia-6 6.9b-deduped. We ablate the reference model μi used in eeef query and compare the p-values
we get in Table 6. The averaged Llama models are:
meta-1llama/Neet-LLllaa--
meta-1laaaa/1laaa2227bhhhtt and
huggyllama/llama-7b
and the averaged Pythia models are
EleutherAl/pprthaa--..00
EleutherAA//pytthaa-...bb.
EleutherrA//pythha-1144bb
EleutherAAl/pythaa-hb
#leutherA//pythhaa44.00
18

---

Pythis-6 6.9.--6.bbb-oo 9b-deddped checkpoints
Pythia-1 4b-deduped checkpoints
70000
13te
306od
257e43
325ed0 304e-4 2:0444 635-4)
70000
63te11
sibeda
2:3eec 2ile 57txa 315444
36
60000
Mohelw SAke
1330
Lobest 30xe 2.0e
Base checkooint
Base checkpoint
sampon
natess saweoz sowed 500-c4
cample
130000
519377 3ot 30w Li5e
120000
Vae 1htll LONE77
120000
taw4r
199992
Finetune checkpoint
Hietune checkpoint
(a)
on Sythia-6.9b-dediie across
Quurry checkpoints.
on pythia-1. 4b--duuuee across
checkpoints.
Pythia 1b deduped checkpaints
Pythla-410m deduped checkpaints
70002
33le08 11040
10000
Sch-20
Arteic s4be-t 2-seds 3.tw
00000
Base chackgoint
Base checkpaint
230000
LIwOo Laaace
L3be00
220000
3itei 15telt
13t600 723e
110000
130300
Lite15 2x
120000
10000 00000
93000 330000
L10000 120000 130060 340000
70000
00000 93000 300000
210000 120000 130000 140000
Finetune checkpoint
Finetune checkpoint
(c) query
- 1b-dedu across
(d)
Quuary
ref
3ythia-410m-deeie deduped across
checkpoints.
checkpoints.
Figure 6: We Quuery using 40k samples pythia-6. 9b-deduped, pythia-1 pythia--1.A-ddedupel 4b-dedu prchha--b-dedupeer and after significant continued pre-training (70k more steps after the 70k checkpoint), and also that memorization effects are smaller for smaller models.
checkpoints. We find p-values are low even
EleutheerAt/ppthha--1600::
Eleuche:\//yyyhiaa222----ddde
Eleutherrl/pppthie----dddddpppd
Hlothhe:s///ppttaa--440--ddddddd and
We can see that there little signal without using a reference model to account for inherent variations in perplexity of text (the first row). For the later rows, based off the p-values, we conclude that the best reference models are models from the same family, and of similar capability. With 20M tokens, we get p-values significant at le-2 for all reference models (and combinations) tested.
A.6 Estimating logprobs from generated text
Let's consider the case where Bob's API only provides access to the generated text without token probabilities. We show that Alice still approximate logprobs by repeatedly querying Bob's model.
19

---

Ou p-values for (order, apoch, model esoch) pairs; 50000 examples
4 p-values epoch,,
examples
Lattra CTt-r)) 0:00-1
3:30-20 Librit
Litect
11trc) (Mled) 20ww
13004
(AAee Kiird
Slard Viival LX--
Spearmnn p-veluee
LOWA
Xhret KUPC
order Lepoch
ander Epp02
Spearmmen
siwel iee Janea)
model
model Leeoch
(a) Qqueyy between order epoch and model epoch for 50k eval samples.
(b) query between order epoch and model epoch for 20k eval samples.
We train a Transformer model on TinyStories samples for 10 epochs (of 50k sequences), each a random data ordering. We report p-values from the 10 epochs on the 10 data orders, which remain small even after 10 epochs.
Figure 7:
Reference Model
20M Tokens
12M Tokens
4M Tokens
None
2.1 x 10-1
5.9 x 10-1
8.6 x 10-2
EleutherAI/pythia-..c0
8.0 X 10 20
3.1 x 10
2.6 x 10
EleutherAl/pyythia22..8
1.0 x 10-15
7.0 x 10-12
3.7 10-3
EleutherAA//yythaa-1.44
8.1 x 10-12
3.4 x 10-9
1.3 x 10-2
EleutherAT/pyythaa-b
7.3 x 10-10
6.4 x 10-7
5.5 x 10-2
EleutherrAl/pyythia-41000 EleutherAl/ppythia--660
2.5 10-6
2.0 x 10-4
2.3 x 10
1.2 x 10-3
1.2 x 10-2
9.0 10-1
Averaged models models
1.7 x 10-7
1.3 x 10-5
4.2 x 10-4
meta- Maae-1llamaMMtt-LLlaaa 8-88
7.9 x 10-3
9.2 x 10-3
1.6 x 10
Averaged Llama models
3.9 x 10-3 1.2 x 10
7.4 x 10
Table 6: We compute p-values using different reference models μ4 with gret
Qquery with
EleutherAL/pythia-G wttpp//iiaa.bbddduueeeeeeeeeeeee as the candidate model. Reference models similar capability (like EleutherAI/pythie-.. or EleutherA//pythia 2.8b)) yield the lowest p- values.
Our approach follows a simple idea idea- given prefix, we can sample multiple generations to estimate the next token probability. Moreover, Alice does not need to have an accurate probability estimation to compute the test statistics dauery as long as the ranking of the estimated probability is highly correlated with the ranking of the actual probability for each sequence. Specifically, we estimate the next token probability of prefix x by random sampling model output N times:
P(yge(x) RigigN [1 [ypred,5 = ygr]]
where ygt is the ground truth continuation of x in the transcript a and Upred, is the predicted next token for the ith sample. Once we computed the estimated probability, We can apply reference model as in Eq
We conduct experiments on OLMo-7B, using 3M samples from the Epoch I transcript with a prefix length of 16. For each prefix, we repetitively sample up 16 times. We use OLMo-7B-0724 as our reference model.
Results. We first verify that the Spearman correlation between the ranking of actual logprob and the ranking of estimated logprob is high. When the number of repeated samples is N = 1/4/16.
20

---

the correlation is 0. 0..066//..448/////// /0.916 respectively. For our test using queryy the corresponding
p-value is 1.6 x 10 /5.6 x 10-7/6.4 10-6.
B
Observational Setting Experimental Details and Additional Results
B.1 Implementation Details for oobs
We build the index over subset (18.75%) of n-grams of the pythia-deduped training set, up the 100k training checkpoint. (Note that following our problem formulation, this n-gram index can be fully constructed from Alice's 's transcript (or subset of) a.)
For each of Bob's texts _. we count the number of n-gram matches with each of the training documents in our n-gram index, and keep count of how many matches are at each training step. In particular, each 14; the list of n-grams seen training step (e.g. between the i-th and + 1-th checkpoint) and the metric X is the number of matches between x. and f4z Since we index the first 100k checkpoints, we have the number models = 100, 000. We count n-grams up to n 8.
When counting n-grams, we first start with a series of 8 tokens, then decrease down to one token until match is found. We also use time-out feature for practicality, if searching the index for a particular n- g-gram takes more than 0.01 seconds on our machines, due to the high number of n n-grams to index. We believe this does not impact the test effectiveness significantly.
B.2 Implementation Details for @obs
We treat Bob's text x as a collection of shorter documents and let x be the average log-likelihood of these documents under each model, either before or after finetuning. In the case where we do finetune on x we do so for one epoch with a learning rate x 10 with 4 documents per batch.
On Storie, we use the same model architecture (d_model = 256, d_ffn = 512, nwm_nmmmmaaaaaaa sum_layers approximately 3M parameters) as in the multiple epoch experiments. For the observational setting experiments, we train for a single epoch on 500K documents with constant learning rate of 1 x 10-5 and 4 documents per batch. We save checkpoints every 10k documents starting at 450K documents, which we use to resume training on reshuffled data to obtain the models yı...., Pa in our implementation of dishuff
We obtain Bob's model by continuing to train on additional documents from the TinyStories training set (distinet from the 500K documents we use train Alice's model). Unlike when we resume training from an intermediate checkpoint in of shulll we reinitialize the optimizer state for Bob's model.
obs
B.3 Sampling Parameter Ablations for o
We report p-values computed from dote for the pythia-6. wwwdeduped--tepp100000 100000 model and generated texts, which are prefixed with random sequences from The Pile. We vary sampling temperature and top_p (with temperature) in Figure 8.
We find that the results somewhat sensitive to temperature; in particular, test is less effective at lower temperatures, potentially due to the lack text diversity limiting the effective sample size.
B.4 Ttoriss Ablations for shuff obs
We report additional results on TinyStories with the detutt statistic. We vary the amount of retraining and finetuning together (Figure 9), the sampling temperature (Figure 10) and model size (Figure 11). We find that increasing the amount of retraining improves robustness to finetuning (as expected). Our test generally works well across a range of sampling temperatures, and notably works unexpectedly well at temperature 0.9 for enough tokens (recall we report results at temperature 1.0 for all other experiments). Finally, our test is more effective when scaling up the model size.
B.5 Validity of dhulf
We plot the empirical distribution of output of dshuff under the null in Figure 12. We generate samples of text from the null by training an independent copy of Alice's 's model from scratch (i.e., we keep all training hyperaparameters the same but use a different random seed). The output empirically behaves
21

---

-077
10
10"
00
92
0.4
066
0.6
0.70
0.75
0.80
0.85
0.00
0.6
1000
mamplng lamporsturee
sempling top, pararrade
(a) We vary Bob's sampling temperature and number total tokens.
We vary Bob's top_p sampling parameter and fix 6.4M total tokens.
We report median p-values and interquartile ranges over 10 trials of d using n-gram counts for pythia-6. 9b-deduped-step110000 00000000 while varying sampling parameters of Bob's model. Because sampling is expensive, for each trial we resample generations with replacement from an initial sample of 50K generations (i.c., 6.4M tokens), so the reported ranges are likely narrower than true confidence bands.
Figure 8:
10
10-17
100
10
pvoluue
10
10
smount of reiraining 2% smount of retraining retraining
72
10
smount of retraining
amount of retraining
10%
8
10
amount of finstuning (% of pretraining)
Figure 9:
We report approximate p-values from applying dل to TinyStories models, varying the amount of retraining in the test and the amount of finetuning Bob.
like conservative p-value, in the sense that the probability it is less than for [0. 1] is at most (whereas for true p-value the probability would be equal to r). Concretely, this implics that the approximate p-values we report from dhull typically larger than true p-values.
obs
B.6 Additional OLMo Results for shuff
obbs
In Table 7, we evaluate dsbull on OLMo-2-0422 -2-255--22 1B and 1MMo-221124-77 for different sizes of Bob's text, using the same protocol described in Section 4.3.2 but for smaller token counts. Notably, even evaluating single generation (i.e., |x = 32) we observe p-values on the order of 10-3 and 10-5 roughly quarter of the time the 1B and 7B scales respectively.
C Discussion of Related Work [17, 18]
We discuss two related works in the query setting. Both Nikolic et al. [17] and Jin et al. [18] try to determine whether two models are independently trained versus not based on the similarity their outputs. Neither approach yields exact p-values, and morcoever show that they can be unreliable
22

---

10
tokens 32 100
10
10-5
10
p-volue
10
10
10
10-15
0.2
0.6
1.0
1.4
sampling temperature
Figure 10: We report approximate p-values from applying shulf
obs to TinyStories models, varying the
sampling temperature of Bob's model.
3M parameters
24M parametars
10-12
10
10
10
10
10-96
10
10
10-21
10-
10-M
lakens x2 100
tokens 32 1000
50C0
10000
15000
20020
25000
32002
0.00
0.25
0.50
0.75
1.00
1.25
1.50
1.75
2:00
Takera
amount Rinstuningss of protisiring)
Figure 11: We report approximate p-values from applying dc to TinyStories models of different scales. We obtain the 24M parameter model using the exact same procedure for the 3M model, except we increase all intermediate activation sizes (i.e., both for self-attention and MLP layers) and the number of layers each by a factor of two. The total parameter counts are rough estimates. (Left) We vary the number of tokens in Bob's 's text and do not finetune on Bob's text. (Right) We finetune on Bob's text and vary the amount of finetuning by Bob, with results for the smaller model in solid.
in practice for models independently trained on similar data. By using specific prompts to generate responses, both methods also do not work for the observational setting.
Nikolic et al. [17] count token matches in outputs between two models μ and μº (i.e., Alice and Bob's 's's models) over prompts x², and compare similarity score based these counts between μº and و versus between μº and a class of reference models (see Algorithm in [17]). In particular, this does not yield provably exact p-values for independently trained models.
We run an experiment involving Pythia models to show a failure mode of their thet- test indicates that pythia-1. 4b and pythia-1.. 1b....bbddduun de-dedd not independent, independent runs trained on two different transcripts. Specifically, we use the reference models given in their Bench A and compute similarity of Pile sequence prompts (which likely all the models were trained on) with pythia-1 4b, shown in Table
though they were two
We see that the similarity between the independent pythia-1... 4b and pythia-1 aatlaaa..bbbaae 4b-deddeed models (0.7976) higher than the similarity between finetune pair pythia-1. 1ithia--..bb and pythial. 221...B-f4 www.inntuedddoo-lammniti..
(0.7714). Using 7-test with the given reference models from [17] would yield a p-value of 9.622e-06 for the independence between pythia-1 and pythia-1 4b-deduped, when in fact they independent, meaning there is potentially very
23

---

1.0
0.8
Probability thele
0.8
0.4
0.2
0.0
0.0
0.2
0.4
Figure 12: We plot the distribution of the output by @iuuff under the null.
Model Size
x 32
96
160
320
1B
1.7 10-1
9.7 10
1.1 x 10-1
1.3 10-1
10 1.6'x
x1.0 x3 10-1)
x04x10- 1.000.2222. 22 10-1)
(6.5 1.55x1000 2.8 10-1)
7B
3.7 x 10 (2.7 x 10-5.7.5x
1.3 x 10
9.9 x 10
4.5 x 10 3
7.5 x
x6.4 10 1.00..300 10-1) (3.0 x 10 x 10-1) (L.0 10 4.9 10
Table 7: Median p-values (with interquartile ranges) across 10 trials from applying dshuff to NLMo-2-0425-11 and 0LMo-2-1122-7B.
high false positive rate. In contrast, our methods yield provably exact p-values (see Theorem 1) where control for Type-I error.
Jin al. [18] craft prompts for models and compare the similarity of outputs. Like Nikolic al. [17], they use metric "target response rate" (TRR) that measures similarity of responses, and they identify two models as fine-tunes if their TRR higher compared to the values for non fine-tunes. However, they find that independently trained models from the same model developer may have similar responses, and they classify such models as "related" (e.g. may share training data, for example). As such, in Table III of their paper, they find the TRR between Mistral-7B- 1 and Mistral-7B-v0
0.70 and higher than the TRR with any Mistral-7B-vO. 1 fine-tunes, although the first two are independent.
D Compute and Cost
D.1 Audits and Cost Analysis
Auditing and cost. We give cost estimates of rumning the query-setting test for current language model APIs. For our test under the query setting, we observe that logprob over 64M token samples is typically enough to achieve a p-value below 10-3, Thus, we estimate the cost using a sample of 8M sequences, each with 8 tokens.
As of May 2025, the cost of querying 1M inference tokens for OpenAI GPT-4.1 mini is $0.40 per IM input tokens and $1.60 per IM output tokens". Computing an average over window size of 7 8/2 32 tokens per sequence) with one output token per window would have cost of $0.40 x 8 x (1 + 2 ...... such the Pythia family.
7) + $1.60 x 8 x 7 $$19..0.. Some models may require fewer tokens,
Note that some APIs only give top-k logprobs or text output. In this case, one can estimate logprobs using the method described in Appendix A.6.
phttps://plaae htt.:///w...pppaaaaco.eoo/////pe www.////c///cciiiaa.cc.m

---

model
fine-tuned?
1a with pythia-1.
0.5451
neta-lllanna/iaamm-3-----------
X
0.5408
0.6137
microsoft/phi-2
X
0.6453
google/gemma-22
0,6322
50ogle/genma-2-25
X
0.6148
Qven/Qwen2-1.5B
0.6126
(pen//Wwen22-511.----nstttutttt
0.4875
X
0,5473
Timyylaam//ThyyLlaaa-..... 1h
0.6050
BleutherrAl/pyttia--14bb
1.0000
abbeu//pythia-1111-ffffttttt
0.8205
0.7714
EleucherrAl/pyttiii------- deduped
0.7976
Table 8: We compute model output similarities f following Nikolic et al. [17] between pythia-1. and other models on HuggingFace. The similarity with the independent pythia- 1eriiia-..b--eueeee -bb-deduped is higher than the similarity with a pythia-1. 4b fine-tune.
The observational setting does require querying an API, Alice passively observes text generated by Bob's model.
D.2
Compute Resources
We rum our experiments on internal cluster using NVIDIA A100 and A6000 GPUs. The primary compute use for our methods is getting the perplexity of samples for different models. For 6.9b- parameter language model (the majority of our experiments), computing the perplexity on 1M sequences of length 64 tokens per sequence takes around 75 mins on single A A100 GPU.
For the observational setting, we build the n-gram to training steps index for Pythia's 's pre-training data using the infini-gram package [42]. We index 18. 18775% of the training data, which takes 258G disk space. To run the test o once all the documents are processed, which involves getting the counts of n-gram matches each train step, takes around minute for 100000 texts (12.8M tokens),
and scales approximately linearly with number of tokens (most processing time disk 10).
Training the TinyStories models described in Appendix A.4 for 50000 sequences [40] takes around 10 minutes on a A6000 machine.
25

---

NeurIPS Paper Checklist
Claims
Question: Do the main claims made the abstract and introduction accurately reflect the paper's contributions and scope?
Answer: [Yes]
Justification: We clearly state our claims and assumptions in our problem formulation (Section 3.1). We prove the validity of our tests in Theorem I, and present emprical evidence (Section 4) supporting the main claims that our methods can determine provenance for fine-tuned models and generated text. We discuss our limitations in Section 5. Guidelines:
The answer NA means that the abstract and introduction do not include the claims made in the paper.
The abstract and/or introduction should clearly state the claims made, including the contributions made in the paper and important assumptions and limitations. A No or NA answer to this question will not be perceived well by the reviewers.
The claims made should match theoretical and experimental results, and reflect how much the results can be expected generalize to other settings.
It is fine include aspirational goals motivation long it is clear that these goals not attained by the paper.
2. Limitations
Question: Does the paper discuss the limitations of the work performed by the authors?
Answer: [Yes]
Justification: Yes, we discuss limitations in discussion section (see Section 5).
Guidelines:
The answer NA means that the paper has no limitation while the answer No means that the paper has limitations, but those are not discussed in the paper.
The authors are encouraged to create a separate "Limitations" section in their paper.
The paper should point out any strong assumptions and how robust the results are to violations of these assumptions (e.g., independence assumptions, noiseless settings, model well-specification, asymptotic approximations only holding locally). The authors should reflect on how these assumptions might be violated in practice and what the implications would be.
The authors should reflect on the scope of the claims made, e.g., the approach was only tested few datasets with few runs. In general, empirical results often depend on implicit assumptions, which should be articulated.
The authors should reflect on the factors that influence the performance of the approach. For example, facial recognition algorithm may perform poorly when image resolution is low or images are taken in low lighting. Or speech-to-tex system might not be used reliably to provide closed captions for online lectures because it fails handle technical jargon.
The authors should discuss the computational efficiency of the proposed algorithms and how they scale with dataset size.
If applicable, the authors should discuss possible limitations of their approach address problems of privacy and fairness.
While the authors might fear that complete honesty about limitations might be used by reviewers as grounds for rejection, worse outcome might be that reviewers discover limitations that aren't It acknowledged in the paper. The authors should use their best judgment and recognize that individual actions in favor of transparency play important role in developing norms that preserve the integrity of the community. Reviewers will be specifically instructed to not penalize honesty concerning limitations.
Theory assumptions and proofs
Question: For each theoretical result, does the paper provide the full set of assumptions and a complete (and correct) proof?
Answer: [Yes]
Justification: Yes, we have one theoretical result (Theorem 1), and we provide its proof in the main text. Assumptions are clearly stated in the theorem statement.
26

---

Guidelines:
The answer NA means that the paper does not include theoretical results.
All the theorems, formulas, and proof' in the paper should be numbered and cross-referenced.
All assumptions should be clearly stated or referenced in the statement of any theorems.
The proofs can either appear in the main paper or the supplemental material, but if they appear in the supplemental material, the authors encouraged provide short proof sketch to provide intuition.
Inversely, any informal proof provided in the core of the paper should be complemented by formal proofs provided in appendix or supplemental material.
Theorems and Lemmas that the proof relies upon should be properly referenced.
4. Experimental result reproducibility
Question: Does the paper fully disclose all the information needed to reproduce the main experimental results of the paper to the extent that it affects the main claims and/or conclusions of the paper (regardless of whether the code and data are provided or not)?
Answer: [Yes]
Justification: We provide sufficient details about our experiments to reproduce them (see the section on Experimental Setup, Section 4.1). We also plan release code prior to publication.
Guidelines:
The answer NA means that the paper does not include experiments.
If the paper includes experiments, a No answer to this question will not be perceived well by the reviewers: Making the paper reproducible is important, regardless of whether the code and data are provided or not.
If the contribution a dataset and/or model, the authors should describe the steps taken to make their results reproducible or verifiable.
Depending on the contribution, reproducibility can be accomplished in various ways. For example, the contribution is novel architecture, describing the architecture fully might suffice, or if the contribution is specific model and empirical evaluation, it may be necessary to either make it possible for others to replicate the model with the same dataset, or provide access to the model. In general. releasing code and data is often one good way to accomplish this, but reproducibility can also be provided via detailed instructions for how to replicate the results, access to hosted model (e.g., in the case of a large language model), releasing of a model checkpoint, or other means that are appropriate to the research performed.
While NeurlPS does not require releasing code, the conference does require submissions to provide
reasonable avenue for reproducibility, which may depend on the nature of the contribution. For example
(a) If the contribution is primarily new algorithm, the paper should make it clear how to reproduce that algorithm.
(b) If the contribution is primarily a new model architecture, the paper should describe the architecture clearly and fully.
(c) If the contribution is a new model (e.g., large language model), then there should either be way access this model for reproducing the results way reproduce the model (c.g., with an open-source dataset or instructions for how to construct the dataset).
(d) We recognize that reproducibility may be tricky in some cases, in which case authors welcome to describe the particular way they provide for reproducibility. In the case of closed-source models, it may be that access to the model is limited in some way (e.g.. to registered users), but it should be possible for other researchers have some path to reproducing or verifying the results.
5. Open access to data and code
Question: Does the paper provide open access to the data and code, with sufficient instructions to faithfully reproduce the main experimental results, as described in supplemental material?
Answer: [No]
Justification: We do not yet provide code or experimental data, but plan to release all relevant artifacts before publication.
Guidelines:
The answer NA means that paper does not include experiments requiring code.
27

---

Please see the NeurIPS code and data submission guidelines (attps:/hhipss.ccpbbll) guides/CodeSuumis ssinnooicyy for more details.
While we encourage the release of code and data, understand that this might not be possible, so "No" is an acceptable answer. Papers cannot be rejected simply for not including code, unless this is central to the contribution (e.g., for new open-source benchmark).
The instructions should contain the exact command and environment needed to run to reproduce the results. See the NeurIPS code and data submission guidelines (https: htabbl/c/ggides///oo// The authors should provide instructions on data access and preparation, including how to access the raw data, preprocessed data, intermediate data, and generated data, etc.
/
wwwpeuiwisssssoooooooooooooooooo ssionPolicy) for more details.
The authors should provide scripts to reproduce all experimental results for the new proposed method and baselines. If only subset of experiments reproducible, they should state which ones are omitted from the script and why.
At submission time, to preserve anonymity, the authors should release anonymized versions (if applicable).
Providing much information possible in supplemental material (appended to the paper) is recommended, but including URLs to data and code is permitted.
6. Experimental setting/details
Question: Does the paper specify all the training and test details (e.g., data splits, hyperparameters, how they were chosen, type of optimizer, etc.) necessary to understand the results?
Answer: [Yes]
Justification: Yes we describe details for our experiments (Section 4.1), such as which models we test from HuggingFace and the datasets used. We have more details for experiments discussed in the Appendix.
Guidelines:
The answer NA means that the paper does not include experiments.
The experimental setting should be presented in the core of the paper level of detail that is necessary to appreciate the results and make sense of them.
The full details can be provided either with the code, in appendix, or as supplemental material.
7. Experiment statistical significance
Does the paper report еπтоr bars suitably and correctly defined or other appropriate information about the statistical significance of the experiments?
Question:
Answer: [Yes]
Justification: Yes; our main results all yield exact p-values (see Theorem 1). These p-values allow for false positive control, and we report results for fine-tuned and non-fine-tuned model pairs. We also describe classification task and our accuracy.
Guidelines:
The answer NA means that the paper does not include experiments.
The authors should answer "Yes" if the results are accompanied by error bars, confidence intervals, statistical significance tests, at least for the experiments that support the main
claims of the paper.
The factors of variability that the error bars are capturing should be clearly stated (for example, train/test split, initialization, random drawing of some parameter, or overall run with given experimental conditions).
The method for calculating the error bars should be explained (closed form formula, call to library function, bootstrap, etc.)
The assumptions made should be given (e.g., Normally distributed errors).
It should be clear whether the error bar is the standard deviation the standard error of the mean.
It is OK to report I-sigma error bars, but one should state it. The authors should preferably report a 2-sigma error bar than state that they have 96% CI, if the hypothesis of Normality
errors is not verified.
For asymmetric distributions, the authors should careful not to show in tables figures symmetric error bars that would yield results that are out of range (c.g. negative error rates).
28

---

If eтror bars are reported in tables plots, The authors should explain in the text how they were calculated and reference the corresponding figures or tables the text.
8. Experiments compute resources
Question: For each experiment, does the paper provide sufficient information on the computer resources (type of compute workers, memory, time of execution) needed reproduce the experiments?
Answer: [Yes]
Justification:
Yes, in Appendix D we discuss compute resources used for our experiments.
Guidelines:
The answer NA means that the paper does not include experiments.
The paper should indicate the type of compute workers CPU or GPU, internal cluster, or cloud provider, including relevant memory and storage.
The paper should provide the amount of compute required for each of the individual experimental runs as well as estimate the total compute.
The paper should disclose whether the full research project required more compute than the experiments reported in the paper (e.g., preliminary or failed experiments that didn't ' make it into the paper).
9. Code of ethics
Question: Does the research conducted in the paper conform, in every respect, with the NeurlPS Code of Ethics https: /tte::://eeurippp cc/ppbbltttBhhhiccttiellll.eeeee
Answer: [Yes]
Justification: We use public models and datasets on HuggingFace, do not involve human subjects, and don't 't believe our work has harmful consequences or impacts.
Guidelines:
The answer NA means that the authors have not reviewed the NeurlPS Code of Ethics.
If the authors answer No, they should explain the special circumstances that require deviation from the Code of Ethics.
The authors should make sure to preserve anonymity (e.g., if there is special consideration a
due to laws or regulations in their jurisdiction).
10. Broader impacts
Question: Does the paper discuss both potential positive societal impacts and negative societal impacts of the work performed?
Answer: [Yes]
Justification: We diseuss the broader impact of our work in the introduction, and how it is important for IP protection for model developers. We do not believe our work has negative societal impacts.
Guidelines:
The answer NA means that there is no societal impact the work performed.
If the authors answer NA or No, they should explain why their work has no societal impact why the paper does not address societal impact.
Examples of negative societal impacts include potential malicious or unintended uses (e.g., disinformation, generating fake profiles, surveillance), fairness considerations (e.g., deployment of technologies that could make decisions that unfairly impact specific groups), privacy considerations, and security considerations.
The conference expects that many papers will be foundational research and not tied to particular applications, let alone deployments. However, if there is direct path to any negative applications, the authors should point it out. For example, it is legitimate to point out that an improvement in the quality of generative models could be used to generate deepfakes for disinformation. On the other hand, it is not needed to point out that generic algorithm for optimizing neural networks could enable people to train models that generate Deepfakes faster.
The authors should consider possible harms that could arise when the technology is being used as intended and functioning correctly, harms that could arise when the technology is being used as intended but gives incorrect results, and harms following from (intentional or unintentional) misuse of the technology.
29

---

If there are negative societal impacts, the authors could also discuss possible mitigation strategies (e.g., gated release of models, providing defenses in addition to attacks, mechanisms for monitoring misuse, mechanisms to monitor how a system learns from feedback over time, improving the efficiency and accessibility of MI.).
11. Safeguards
Does the paper describe safeguards that have been put in place for responsible release of data or models that have high risk for misuse (e.g.. pretrained language models, image generators, scraped datasets)?
Question:
Answer: [NA]
Justification: NA
Guidelines:
The answer NA means that the paper poses no such risks.
Released models that have a high risk for misuse or dual-use should be released with necessary safeguards to allow for controlled use of the model, for example by requiring that users adhere to usage guidelines restrictions to access the model or implementing safety filters.
Datasets that have been scraped from the Internet could pose safety risks. The authors should describe how they avoided releasing unsafe images.
We recognize that providing effective safeguards is challenging, and many papers do not require this, but we encourage authors to take this into account and make a best faith effort.
12. Licenses for existing assets
Question: Are the creators or original owners assets (e.g., code, data, models), used in the paper, properly credited and are the license and terms of use explicitly mentioned and properly respected?
Answer: [Yes]
Justification: We cite existing code packages (and the associated papers where applicable) that we use for our methods. The language models we test are public from HuggingFace (Apache 2.0 licenses) and cite relevant original papers if they are online.
Guidelines:
The answer NA means that the paper does not use existing assets.
The authors should cite the original paper that produced the code package or dataset.
The authors should state which version of the asset is used and, if possible, include a URL.
The name of the license (e.g., CC-BY 4.0) should be included for each asset. For scraped data from particular source (e.g., website), the copyright and terms of service of that source should be provided.
If assets released, the license, copyright information, and terms of use in the package should be provided. For popular datasets, paperswi thcode. paperswithcode. com/ datastts has curated
licenses for some datasets. Their licensing guide help determine the license of dataset.
For existing datasets that re-packaged, both the original license and the license of the derived asset (if it has changed) should provided.
If this information is not available online, the authors are encouraged to reach out to the asset's creators.
13. New assets
Question: Are new assets introduced in the paper well documented and is the documentation provided alongside the assets?
Answer: [NA]
Justification: NA
Guidelines:
The answer NA means that the paper does not release new assets.
Researchers should communicate the details of the dataset/code/model as part of their submissions via structured templates. This includes details about training, license, limitations, etc.
The paper should discuss whether and how consent was obtained from people whose asset is used.
30

---

At submission time, remember to anonymize your assets (if applicable). You can either create an anonymized URL include an anonymized zip file.
14. Crowdsourcing and research with human subjects
Question: For crowdsourcing experiments and research with human subjects, does the paper include the full text of instructions given participants and screenshots, if applicable, well as details about compensation (if any)?
Answer: [NA]
Justification: NA
Guidelines:
The answer NA means that the paper does not involve crowdsoureing nor research with human subjects.
Including this information in the supplemental material is fine, but if the main contribution of the paper involves human subjects, then as much detail possible should be included in the main paper.
According to the NeurIPS Code of Ethics, workers involved in data collection, curation, or other labor should be paid least the minimum wage in the country of the data collector.
15. Institutional review board (IRB) approvals or equivalent for research with human subjects Question: Does the paper describe potential risks incurred by study participants, whether such risks were disclosed to the subjects, and whether Institutional Review Board (IRB) approvals (or an equivalent approval/review based on the requirements of your country or institution) were obtained?
Answer: [NA]
Justification: NA
Guidelines:
The answer NA means that the paper does not involve crowdsourcing nor research with
human subjects.
Depending on the country in which research is conducted, IRB approval (or equivalent) may be required for any human subjects research. If you obtained IRB approval, you should clearly state this in the paper.
We recognize that the procedures for this may vary significantly between institutions and locations, and we expect authors to adhere to the NeurIPS Code of Ethics and the guidelines for their institution.
For initial submissions, do not include any information that would break anonymity (if applicable), such as the institution conducting the review.
16. Declaration of LLM usage
Question: Does the paper describe the usage of LLMs if it is an important, original, or non- standard component of the core methods this research? Note that the LLM used only for writing, editing, or formatting purposes and does not impact the core methodology, scientific rigorousness, or originality of the research, declaration required.
Answer: [NA]
Justification: NA
Guidelines:
The answer NA means that the core method development in this research does not involve LLMs as any important, original, or non-standard components.
Please refer to our LLM policy http:p//eaarrpp.ccoooooorreeee/a wwt.c///oooeeeeeee////OOOO what should or should not be described.
31


Reviews:
[{"review_id": "QhxRIUfq0K", "reviewer": "Reviewer_zdZQ", "summary": "This paper introduces a novel black-box model provenance method by analyzing the memory properties of a language model on the order of its training data. The method proposes tests in two settings: query setting and sample setting, and empirically verifies its effectiveness even when the model has undergone common post-training practices. This work provides a new framework for provenance verification of open source language models, which can help improve model accountability and protection.", "questions": "In experiments 4.3.1, why samples the training data from the first epoch to construct the transcript?", "limitations": "The threat model should be further discussed. What information Alice has to disclose? How about prior work? What information should be disclosed in their settings? This is important because the proposed method looks like a new framework with new threat model.", "rating": 4, "confidence": 3, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": 2, "clarity": 3, "significance": 3, "originality": 3, "strengths_and_weaknesses": "Strengths:\n1. This paper proposes a unique method based on \"palindromic memory\" to solve the model traceability problem by using the language model's memory property of the training data sequence. This is different from previous methods that rely on private triggers or test sets and is innovative.\n2. Wide applicability and robustness: The method is robust to common post-training practices (such as supervised fine-tuning, preference optimization, model fusion, etc.) in the query setting, and can reliably attribute text with only thousands of tokens in the sample setting, which greatly broadens its potential for real-world applications.\n3. Proposed testing framework is able to provide provable false positive error control (via precise p-values), a significant advantage over existing work that relies on heuristics, providing statistically rigorous guarantees on model provenance.\n\nWeakness:\n1. The main limitation of this method is that it requires access to Alice's training data to perform testing. Although the authors mention that it is possible to use only a subset of the training data, in practice model developers may be reluctant to disclose their training data. While prior work like [1] do not need to public the training data, they only public the hash keys for verification. In the experiments, Alice expose 1M training samples to construct the transcript. Which is a large privacy concern, especially considering prior work does not need to leak any training data.\n2. If Bob has the transcript, it can shuffle the training dataset and fine-tune the model on such adversaial desined trainscript. So the memory of the LLMs will be shuffled. In this case, can the proposed test can identify such fine-tuned LLMs? If so, why? Since the insight of this paper is that the later memories will overwrite earlier memories, so if the attacker also uses this to fine-tune, wouldn't it be possible to eliminate the previous order?\n3. For smaller models (e.g., Pythia-1.4b), the test showed false negatives in some cases, which may be due to \"catastrophic forgetting\" causing the model's performance on the pre-training data to drop significantly after fine-tuning. This means that the method may not be effective in identifying fine-tuned models derived from the Alice model in some cases, especially when the model quality is severely affected.\n\n[1] A Watermark for Large Language Models.", "was_revised": true, "final_justification": "The reviewer's rebuttal addressed most of my concerns, but I still have concerns about the practicality.", "rebuttal": "We thank the reviewer for their time and effort. We address their questions and concerns below.\n\n**Weakness 1** (Disclosure of Alice's training data)\n\nWe understand the reviewer's concerns on disclosing information about the training data. We would like to further clarify when and what Alice needs to disclose about her training data.\n\nPublishing information of training data is only needed **for third parties to verify the test result**. The main use case of this test is for the model developer (Alice), who clearly has her full training data, to check whether someone else has used their model. In this case, Alice can keep her training data private. \n\nIf she wishes to prove to someone else that others are using her model (for public verifiability purposes), she can publish the order of a dataset without privacy concerns like Wikitext. Specifically, for public verification,\n\n- under the query setting, Alice needs to disclose usually less than 1M sequences of her training data. As we discuss on L323-324, Alice can choose which subset of the training data to make public. For example, she may choose to release Wikitext samples. Wikitext is used in training nearly all language models and is under the Creative Commons license, which would not be a privacy concern.\n\n- **under the sample setting, Alice does not need to disclose any exact copy of her training data**. She only needs to release aggregated n-gram stats of the data for some small n. Given the nature of information that needs to be disclosed, we believe the privacy risk can be mitigated (especially if the subset is something like Common Crawl where releasing it would not have privacy concerns).\n\nPrior methods like [1] have not considered this public verifiability perspective, which is why they do not need to publish test implementation details. Also, we would like to emphasize that inference-time watermarking methods like [1] are not applicable to the sample setting, as Bob can directly sample text himself without the watermark (L112-L114). \n\n**Weakness 2** (Retraining on the shuffled data)\n\nThis is indeed an interesting question! We have studied this adversarial setting in Section 4.3.3. In our experiments in TinyStories models (Appendix A.4), we find retraining on the same texts with a different shuffle still preserves the orders of the recent iterations. Specifically, we tried retraining for 10 epochs on TinyStories, and the orderings from the last 4 epochs **all** yielded significant correlation with the final model (Figure 6 of the appendix). We agree that with enough epoching over the same texts, the original order could be lost, however, this means to fully mask Alice’s original training order, Bob would have to do fine-tuning at the scale of the original pretraining budget. In fact, on Pythia and OLMo models, we find that even after retraining 50% of the original pretraining texts (L289-L291 and Figure 5), our tests still yield extremely low p-values. Moreover, fine-tuning on a small subset of the transcript, such as one million sequences, is not sufficient to evade the test. In that case, Alice can use another random subset of her training data, which very likely Bob has not retrained on.\n\nRegarding memorization, we would like to clarify that the most interesting finding is that, in the multi-epoch setting, earlier pretraining orders are inscribed into the model along with later ones (L292-L293), instead of being completely overwritten. This is exactly why we call it **“palimpsestic” memorization**.\n\n**Weakness 3** (Catastrophic forgetting in smaller models)\n\nAs the reviewer noted, we show that all failure cases have much higher loss on general language modeling, which means the fine-tuned model’s quality is severely affected. Doing such fine-tuning defeats the purpose of modeling stealing, as these fine-tuned models would not be very usable in practice.\n\nMoreover, larger models are the most likely target of model stealing and misuse, and our test yields low p-values on all 7B and 12B models tested. This is in line with existing studies that show smaller models memorize less [2, 3].\n\n[2] Nicholas Carlini, Daphne Ippolito, Matthew Jagielski, Katherine Lee, Florian Tramer, and Chiyuan Zhang. 2023. Quantifying memorization across neural language models. In The Eleventh International Conference on Learning Representations.\n\n[3] Pietro Lesci, Clara Meister, Thomas Hofmann, Andreas Vlachos, and Tiago Pimentel. 2024. Causal estimation of memorisation profiles.\n\n**Question** (Sampling training data from the first epoch in experiments)\n\nWe sample from the first epoch for consistency across model families (for example, `pythia` models are trained for one epoch, whereas `pythia-deduped` models are trained for more than one epoch). The fact that our tests yield such results even using only the earliest epoch ordering shows how effective it is against continued pre-training or fine-tuning. \n\n**Limitation** (Prior work and disclosure of Alice's training data)\n\nTo run this test, Alice would have to disclose a subset of her training data (discussed in weakness 1). However, this subset could be something like Wikitext that would not be proprietary. The most-relevant prior work is [4] which requires using a held-out validation set that would also have to be disclosed to run the test. In particular, this validation set has to be i.i.d., i.e., must be representative of the entire training data, and would in fact be more proprietary. \n\n[4] Pratyush Maini, Mohammad Yaghini, and Nicolas Papernot. Dataset inference: Ownership resolution in machine learning. In International Conference on Learning Representations, 2021.\n\nWe appreciate the reviewer's comments and are happy to discuss further!"}, {"review_id": "Rfs1AqOr1f", "reviewer": "Reviewer_Gt7T", "summary": "This paper studies the following question: Suppose one party, Alice, trains an open-weight language model, and a second party, Bob, produces text using a derivative of that model. Can Alive prove that Bob is using her model? This question may be important for a company like Meta, whose terms of service requires that those using their models within a product would need to acknowledge that the product is \"built with Meta Llama 3.\"\n\nThe main insight of the authors' approach is the observation that LMs are typically trained on randomly shuffled data, and **data seen later in training is more likely to be memorized**. Therefore, we can test whether Bob's model derives from Alice by correlating its output with the ordering of examples in Alice's training run. The test works in two settings:\n1. **The query setting:** Given the ability to obtain log-likelihoods from Bob's model for a given piece of text, test whether the log-likelihoods assigned to pretraining examples correlates with the order of those examples in pretraining.\n2. **The sample setting:** Given only the ability to sample text from Bob's model, test whether n-gram overlap between the generations and pretraining examples correlates with the order of those examples in pretraining.", "questions": "1. Why is transparency important for a good test? The authors argue this is a downside of tests based on holding out a secret test set or embedding a secret fingerprint, but it is not clear why model developers would prefer completely transparent tests.\n2. Can you comment more on the training data for each of the models you studied? E.g., Is it known to be true that `pythia-deduped` reshuffles the (deduplicated) training data of `pythia`, or is it possible that `pythia-deduped` \"pulls out\" the duplicated examples from the current ordering?\n3. In the caption of Figure 1c, what does \"*The order of losses matches the training order*\" mean? Do you mean loss diffs at the end of training for both models? In general, the interpretation of the results in this caption versus §4.2 can be cleaned up. It is not always clear how the plots should be interpreted to support all of the claims.\n4. In the sample setting, does the test work at detecting post-trains of Alice's model? If I understand correctly, it's using the same `pythia-deduped` model as both Alice's and Bob's model (unlike the query setting).\n5. Are `OLMo-0424` and `OLMo-0724` both used?\n\n**Other suggestions:**\n- It may be a good idea to define more of the notation used in §3.1, e.g., $\\Delta$, $\\perp$ (since I assume here it does not indicate difference or perpendicular to)\n- L. 259: \"*earses*\" $\\rightarrow$ \"*erases*\"\n- Since the TinyStories results are only shown in the Appendix, I would recommend removing their mentions in the main paper. At the end of the paper, I was confused about not seeing any results on TinyStories despite it being mentioned multiple times earlier in the paper.", "limitations": "Yes.", "rating": 5, "confidence": 3, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": 4, "clarity": 4, "significance": 4, "originality": 4, "strengths_and_weaknesses": "### Strengths:\n1. The paper describes the problem setup very well, and clearly distinguishes multiple lines of related work and their different assumptions and realms of applicability. Beyond addressing this particular problem, the paper is rife with insights that are interesting to the broader memorization community.\n2. The paper presents an extremely effective and clever solution, based on the insight that models assign higher likelihood to text seen later in pretraining. In the query setting, it achieves essentially perfect precision and recall at detecting model derivatives.\n3. The test works without requiring Alice to hold out *any* information about the test, unlike methods which use a secret held-out test set or embed secret strings into the pretraining data.\n\n### Weaknesses:\n1. Experiments in the sample setting are much more limited than those in the query setting. For instance, if I understand correctly, the sample setting doesn't actually consider detectin model *derivatives*. While it is expected for the test to be less effective in the sample setting (due to Alice having less information to work with), it would provide valuable insights if the experimental settings were more comparable.\n2. Ultimately, if Bob is hosting (a derivative of) Alice's model through an API, the query setting (where users can access log probabilities) is pretty unrealistic.", "was_revised": false, "final_justification": "", "rebuttal": "We thank the reviewer for their time and effort. We address their questions and concerns below.\n\n**Question 1** (Transparency)\n\nThis is a good question. Transparency is important so that the test can be used by third-parties and auditors, for example. It is also important if a model developer (Alice) wants to prove that Bob’s model is a derivative —- she must disclose the secret test set or fingerprint so another party can independently verify the result. \n\n**Question 2** (Pythia training data)\n\nIn general, Pythia and OLMo construct the training data by first filtering (include deduplication) and then apply a global shuffle. As a result, the training data of `pythia-deduped` is shuffled independently of `pythia`. The implementation details can be found in the pythia and OLMo git repositories.\n\nWe use the training scripts provided by EleutherAI, in which they provide the seeds. Pythia and OLMo generate the training data by filtering / deduplicating then shuffling, so the `pythia-deduped` and `pythia` datasets are shuffled independently. \n\n**Question 3** (Figure 1c clarification)\n\nYes, we mean the differences in losses (e.g. subtracting the loss of a reference model), although the order can also be seen in 1(a), which is just the loss. These plots are mostly to demonstrate that the model in fact preserves the training order, which is more visible when using a reference model. We will add more explanation in the revision. \n\n**Question 4** (Fine-tuned models for the sample setting)\n\nYes, our current experiments in the paper are for the same model for Alice and Bob; we provide additional experiments for the fine-tuned setting in the discussion for Weakness 1. \n\n**Question 5** (Are OLMo-0424 and OLMo-0724 both used?)\n\nYes, in Figure 2 (b) in the main text, Table 3, and Table 4 in the appendix, we show results for fine-tunes of both `OLMo-0424` and `OLMo-0724`.\n\n**Weakness 1** (Additional experiments in the sample setting for fine-tuned models)\n\nWe provide some additional experiments for fine-tuned models, specifically for later checkpoints of the `pythia-6.9b-deduped` model. These are using slightly a different method from the paper, which we found it’s more effective; here, we count the number of n-gram matches from each training step, and take the Spearman correlation between the number of matches and the training step. (In Algorithm 2, this would be taking a frequency count (like `collections.Counter`) over “matches”, and taking the Spearman rank with `arange(100000)`). Using the Spearman correlation this way is valid for the same reason it is valid for the query setting. We use the base model `pythia-6.9b-deduped` at checkpoint 100000 (Alice) and generate $N$ texts (Bob’s texts) from checkpoints 105000, 110000, 115000, 120000, and 130000. \n\nWe find that we get low p-values even after fine-tuning for 30% the number of pretraining tokens, if we use enough generated texts. \n\n| checkpoint | N=5000 | 10000 | 25000 | 50000 | 100000 | 200000 |\n| -------------| --------|---------|-----------|-----------|------------|-----------|\n| step100000 | 0.0196 | 0.0128 | 0.00136 | 0.00063 | 4.95e-05 | 1.51e-05 | \n| step105000 | 0.128 | 0.0106 | 0.0400 | 0.0196 | 0.0013 | 0.00451 | \n| step110000 | 0.312 | 0.206 | 0.238 | 0.0906 | 0.0575 | 0.0190 | \n| step115000 | 0.575 | 0.0455 | 0.0117 | 0.0179 | 0.0506 | 0.0863 | \n| step120000 | 0.819 | 0.940 | 0.478 | 0.450 | 0.185 | 0.0305 | \n| step130000 | 0.325 | 0.380 | 0.214 | 0.0638 | 0.0852 | 0.0406 |\n\n**Weakness 2** (API logprobs)\n\nThis is true, but logprobs can be estimated from an API with repeated sampling (we show this is feasible under a reasonable budget in Appendix A.6 and C.2), or one could use the sample setting. \n\nWe also thank the reviewer for their additional suggestions and will implement them in the revision. We appreciate the reviewer's engagement and are happy to discuss further!"}, {"review_id": "CMxoOzCTcS", "reviewer": "Reviewer_AR86", "summary": "The goal of the paper is to give a statistical test that allows one to prove that some model B was derived from their model A. (Derivation here may be unintentional, and can take many forms, such as training on the model’s outputs or fine-tuning the original model.) Although this task is not new, the paper proposes a novel method to obtain control over the null distribution: the observation that training runs use random permutations to decide their data ordering. This allows one to perform this test without making any modifications to the data or training of the model. Indeed, the test can be applied to models that were trained in the past, so long as a record the data ordering used is available.", "questions": "1. It is common for the best models to be trained for multiple epochs on the same data. Do you think the method will continue to work in that setting?", "limitations": "See weakness 1.", "rating": 5, "confidence": 4, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": 3, "clarity": 3, "significance": 3, "originality": 4, "strengths_and_weaknesses": "## Strengths \n\n1. The approach leveraging the data ordering is very neat and perfectly suitable for the task at hand. It makes provenance testing usable in many more situations than previous methods (e.g. dataset inference requires an IID hold out set which are not available for many popular datasets.)\n2. The paper reports results for both Pythia and OLMo models, the two most prominent models where the training data order is known. \n\n## Weaknesses\n\n1.  The main weakness is that the very idea of “model provenance” is a bit overly broad and it’s not clear how useful it will be in practice. A similar idea is articulated in [1], but essentially the idea is that any model B which is trained on data written or generated after the release of model A is very unlikely to be truly independent of it. For example humans may read the output of A, influencing their style of writing, and thus a model B trained on text written by them would not be independent of A. To be clear, I think the paper is interesting and impactful regardless but I do think some discussion of this issue would be nice to have. \n\n[1] Position: Membership Inference Attacks Cannot Prove that a Model Was Trained on Your Data.” Jie Zhang, Debeshee Das, Gautam Kamath, Florian Tramèr. IEEE SaTML 2025 / arXiv:2409.19798", "was_revised": true, "final_justification": "I think the paper presents a solid approach to an important task (dataset inference) which has which leverages a nearly universal source of null hypothesis (randomized of data order) and obtains good results. The authors responded well to my questions as well.", "rebuttal": "We thank the reviewer for their time and effort. We address their questions and concerns below.\n\n**Weakness 1** (Discussion of model provenance)\n\nThis is an excellent point — over time, it is possible that model A could influence model B (or human text that it is trained on) indirectly, and we agree that this is relevant discussion for \"model provenance\". The main issue [1] raises is a lack of a clear null hypothesis with respect to training data composition. Crucially, our methods are valid regardless of dataset composition, and we have a clear null hypothesis (independence with A’s training order). \n\nWe are careful not to claim that our methods can show Bob has necessarily fine-tuned Alice’s model. It is true that if our method labels a model and set of texts as independent, it’s still possible they share some other provenance (not “truly independent”). In the future, one can imagine model providers may run our tests on large samples of human text in order to re-establish a (approximate) null baseline. We will include this discussion in the revision. \n\n**Question 1** (Multiple epochs of training)\n\nUnfortunately we could not test this for large models trained for multiple epochs as none are fully open-source (OLMo and Pythia models are trained for up to 1.5 epochs). We show that at least after an additional 0.5 epochs, we can still use the first-epoch order to get significant p-values (Table 2 in the appendix), and there are strong correlations with the last four epochs’ orders for TinyStories models (Appendix A.4). In practice, one could use the training order of the final epoch for the best signal and we believe our method would still work in that setting.\n\nWe appreciate the reviewer's comments and are happy to discuss further!"}, {"review_id": "Vn7eBz8hCW", "reviewer": "Reviewer_UXNh", "summary": "This paper proposes tests for verifying the provenance of open-weight language models, based on the hypothesis that data seen later in training is more likely to be memorized. It includes tests for both query and sampling settings and demonstrates the effectiveness of these tests on several language models.", "questions": "1. When using pre-trained models as Alice's models, how to obtain the training transcript (i.e., the order of training samples)?\n\n2. In the sampling settings, how does the temperature parameter in Bob's model affect the results?", "limitations": "Yes", "rating": 4, "confidence": 4, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": 2, "clarity": 2, "significance": 3, "originality": 3, "strengths_and_weaknesses": "**Strengths:**\n\n1. The proposed method does not require modifying the model or inserting special triggers.\n\n2. The tests are evaluated on several families of language models.\n\n**Weaknesses:**\n\n1. The clarity of the paper needs improvement. For example, Definition 2 is confusing.\n\n2. Lack of membership inference context: Although “membership inference” is used in the title, the paper provides no discussion of its relationship to prior membership inference attacks. If the term is meant differently, this should be clarified; otherwise, either background should be added or the term removed from the title.\n\n3. Algorithm 2 is problematic (single_kgrams is not defined).\n\n4. The method assumes that the model developer has access to the exact training order and can expose (at least a part of) the training data. This may be impractical in real-world settings, especially for large-scale training runs where streaming, multi-phase data loading, or privacy constraints prevent full transcript reconstruction.\n\n5.  The paper does not explore how Bob’s decoding strategy (e.g., sampling temperature) affects the test performance. \n---", "was_revised": true, "final_justification": "The rebuttal addressed some of my concerns. However, my concern about its practicality remains, particularly due to the requirement of access to training data. Additionally, the clarity of the paper should be improved. Thus, I’ll raise my rating to a 4 but not higher.", "rebuttal": "We thank the reviewer for their time and effort. We address their questions and concerns below.\n\n**Weakness 1** (Definition 2)\n\nDefinition 2 aims to establish our primary assumption that the training algorithm $A$ shuffles the data order, which we use for the following algorithms. We will clarify this definition in the revision. \n\n**Weakness 2** (Membership inference context)\n\nThanks for the suggestion! We acknowledge that in the literature, “membership inference” refers to predicting whether an individual example is in the training dataset. Here, we are extending the definition following [1, 2]: instead of focusing on the membership of an individual example, or the membership of a set of examples (e.g., dataset inference), we consider the “palimpsestic membership” as whether \"a sequence of examples\" is a subsequence of the training examples. We agree that this is not the standard implication of membership inference. We will add more explanation for this in the revision.\n\nOur work is also related to membership inference attacks. The features we use are commonly used in membership inference attacks, e.g., logprobs and logprobs with a reference. We believe better features discovered by the membership inference community could also help improve our test.\n\n\n**Weakness 3** (Typo) \n\nWe apologize for the typo; `single_kgrams` should be `matches`. \n\n**Weakness 4** (Exact training order in real-world settings)\n\nWe agree that reconstructing and releasing the full transcript might not be feasible in practice. However, we would like to clarify that our method only requires knowing the relative order of a small sample (<0.0001%) of the training data, and Alice does not need to release the full copy of her training data publicly. Specifically, for technical constraints such as data streaming and multi-phase loading, as long as Alice can track the relative order of a small subset of data, for example, logging a few thousands training examples once in a while, she can run this test. For privacy constraints, disclosing a full copy of training data is only required for public verification under the query setting, in which case Alice can choose to release a subset from a commonly used, CC licensed corpus, such as Wikitext (also see our response to Reviewer zdZQ, weakness 1).\n\n**Weakness 5** (Sampling strategies for the sample setting)\n\nPlease refer to our response to question 2.\n\n**Question 1** (Obtaining the training transcript for pretrained models)\n\nFor Pythia and OLMo, the training transcript is reconstructed using a global index, i.e., a mapping from training example order to the order in the original memory-mapped dataset. This global index is either directly provided by the model developers, or can be deterministically regenerated from the random seed and the total number of examples through a shuffle, ensuring the exact training order is reproducible. \n\n**Question 2** (Varying Bob's sampling strategies (temperature and top_p) for the sample setting)\n\nWe present additional experiments varying decoding strategies (temperature and nucleus sampling) and the prompt distribution in the sample setting and present the p-values. These are using a different method from the paper; here, we count the number of n-gram matches from each training step, and take the Spearman correlation between the number of matches and the training step. (In Algorithm 2, this would be taking a frequency count (like `collections.Counter`) over `matches`, and taking the Spearman rank with `arange(100000)`). Using the Spearman correlation this way is valid for the same reason it is valid for the query setting. \n\nWe vary the temperature for Pile completions for the `pythia-6.9b-deduped` model, compared with itself, and keep `top_p`=1.0. With lower temperatures, we find the p-values are larger, but relatively low. At 50000 texts (each 128 tokens), the results are relatively stable across temperatures. We then vary `top_p` and find similar results. Sometimes the p-values can be noisy, so in the revision we will report distributions of p-values for different samples of $N$ texts. \n\np-values, varying temperature: \n| temp | N=5000 | 10000 | 20000 | 50000 | \n| --- | -- | -- | -- | -- |\n| 0.01 | 0.711 | 0.685 | 0.033 | 0.072 |\n| 0.1 | 0.831 | 0.888 | 0.021 | 0.062 |\n| 0.2 | 0.886 | 0.159 | 0.534 | 0.072 | \n| 0.3 | 0.015 | 0.103 | 0.004 | 0.0007 | \n| 0.4 | 0.858 | 0.144 | 0.149 | 0.032 | \n| 0.5 | 0.038 | 0.002 | 0.054 | 0.002 | \n| 0.6 | 0.023 | 0.422 | 0.007 | 0.0005 | \n| 0.7 | 0.002 | 0.537 | 0.013 | 0.0002 | \n| 0.8 | 0.124 | 0.315 | 0.010 | 0.011 | \n| 0.9 | 0.007 | 0.003 | 0.074 | 0.001 | \n| 1.1 | 0.113 | 0.028 | 0.0007 | 0.0009 | \n| 1.2 | 0.030 | 0.218 | 0.041 | 0.0001 | \n| 1.3 | 0.008 | 0.003 | 0.004 | 7.55e-06 |\n\np-values, varying `top_p`: \n| temp | top_p | N=5000 | 10000 | 20000 | 50000 | \n| --- | --- | --- | --- | --- | --- |\n| 0.7 | 1.0 | 0.379 | 0.532 | 0.235 | 0.001 | \n| 0.7 | 0.95 | 0.647 | 0.309 | 0.154 | 0.001 | \n| 0.7 | 0.9 | 0.701 | 0.194 | 0.120 | 0.0006 | \n| 0.7 | 0.8 | 0.281 | 0.032 | 0.010 | 0.018 | \n| 0.7 | 0.7 | 0.073 | 0.054 | 0.039 | 0.011 | \n| 0.9 | 1.0 | 0.509 | 0.152 | 0.046 | 0.0005 | \n| 0.9 | 0.95 | 0.416 | 0.460 | 0.005 | 0.005 | \n| 0.9 | 0.9 | 0.021 | 0.028 | 0.003 | 0.001 | \n| 0.9 | 0.8 | 0.754 | 0.364 | 0.067 | 2.29e-05 | \n| 0.9 | 0.7 | 0.613 | 0.734 | 0.563 | 0.029 |\n\nWe also try varying the prompt distribution by sampling prompts from Wikitext (also for the `pythia-6.9b-deduped` model). We report the p-values below and find they are also significant at 50000 texts. \n| temp | N=5000 | 10000 | 25000 | 50000 | \n| --- | --- | --- | --- | --- | \n| 0.8 | 0.401 | 0.158 | 0.0267 | 0.0032 | 0.0011 | \n\n(For a sanity check, these are the p-values for using completions from the `pythia-6.9b` model compared with the `pythia-6.9b-deduped` training order, which we see are not significant.)\n| temp | N=5000 | 10000 | 25000 | 50000 | \n| --- | --- | --- | --- | --- | \n| 0.8 | 0.600 | 0.310 | 0.659 | 0.544 | 0.664 | \n\nWe appreciate the reviewer's comments and are happy to discuss further!\n\n[1] Pratyush Maini, Mohammad Yaghini, and Nicolas Papernot. Dataset inference: Ownership resolution in machine learning. In International Conference on Learning Representations, 2021.\n\n[2] Pratyush Maini, Hengrui Jia, Nicolas Papernot, Adam Dziedzic. LLM Dataset Inference Did you train on my dataset? In The Thirty-eighth Annual Conference on Neural Information Processing Systems, 2024."}]

Author rebuttal:

Meta-review: The paper aims to prove whether a model B is derived (e.g., finetuned) from another model A, and a statistical proof based approach is proposed. The insight behind the paper is that the order of the training samples matters and the model may slightly memorize more the data seen later ("palimpsestic memorization"). Thus, if the original training samples order is saved during the training of model A, they can be used to statistically test whether model B is derived from model A, since further alteration of model A cannot fully erase the "palimpsestic" effect of training sample ordering. Experiments demonstrated the validity and robustness of this approach.

The idea proposed in this paper is novel and is also supported by sufficient experimental results. The contribution of this paper to the field of LLM provenance is significant. Reviewers generally believe that the paper’s quality is high, and additional questions were addressed during the rebuttal. Thus, I recommend the acceptance of this paper. Since the idea in this paper looks quite interesting and may inspire researchers to address other problems in this field, I recommend a spotlight presentation for this paper.
Decision: Accept (spotlight)