SeedPrints: Fingerprints Can Even Tell Which Seed Your Large Language Model Was Trained From
Authors: Yao Tong
Haonan Wang
Siquan Li
Kenji Kawaguchi
Tianyang Hu
Venue: ICLR 2026 Poster (2026)
Keywords: fingerprint
LLM
Primary area: alignment, fairness, safety, privacy, and societal considerations

Abstract: Fingerprinting Large Language Models (LLMs) is essential for provenance verification and model attribution. Existing methods typically extract post-hoc signatures based on training dynamics, data exposure, or hyperparameters—properties that only emerge after substantial training. As a result, prior evaluations largely focus on lineage verification after fine-tuning, where detection is considerably easier, potentially giving a false sense of safety.
In contrast, we propose a stronger and more intrinsic notion of LLM fingerprinting: **SeedPrints**, a method that leverages random initialization biases as persistent, seed-dependent identifiers present even before training begins. We show that untrained models exhibit reproducible prediction biases induced by their initialization seed. Although weak in magnitude, these biases remain statistically detectable throughout training, enabling high-confidence lineage verification.
Unlike prior techniques that are unreliable before convergence or vulnerable to distribution shifts, **SeedPrints** remains effective across all training stages and robust under domain shifts and parameter modifications. Experiments on LLaMA-style and Qwen-style models demonstrate seed-level distinguishability and enable birth-to-lifecycle identity verification akin to a biometric fingerprint. Evaluations on large-scale pretrained models and fingerprinting benchmarks further confirm its effectiveness under prolonged training and realistic deployment scenarios.
Together, these results suggest that initialization itself imprints a unique and persistent identity on LLMs, forming a true ``Galtonian'' fingerprint.

Published as a conference paper at ICLR 2026
SEEDPRINTS: FINGERPRINTS CAN EVEN TELL WHICH SEED YOUR LARGE LANGUAGE MODEL WAS TRAINED FROM
Kenji Kawaguchi* Tianyang Hu National University of Singapore The Chinese University of Hong Kong, Shenzhen Tyyooong; haonan wang) (u nus edu kenjifcomp nus edu. sg
Yao Tong Haonan Wang**
Siquan Li
2 ilisiquan, hutianyang)
ABSTRACT
Fingerprinting Large Language Models (LLMs) is essential for provenance verifica- tion and model attribution. Existing methods typically extract post-hoc signatures based on training dynamies, data exposure, or hyperparameters- proppertisss that only emerge after substantial training. As result, prior evaluations largely focus on lineage verification after fine-tuning, where detection is considerably easier, potentially giving false sense of safety. In contrast, we propose stronger and more intrinsic notion of LLM fingerprinting: SeedPrints, a method that leverages random initialization biases as persistent, seed-dependent identifiers present even before training begins. We show that untrained models exhibit reproducible predic- tion biases induced by their initialization seed. Although weak in magnitude, these biases remain statistically detectable throughout training, enabling high-com confidenee lineage verification. Unlike prior techniques that are unreliable before convergence or vulnerable to distribution shifts, SeedPrints remains effective across all training stages and robust under domain shifts and parameter modifications. Experiments on LLaMA-style and Qwen-style models demonstrate seed-level distinguishability and enable birth-to-lifeeycle identity verification akin to a biometric fingerprint. Evalu- ations on large-scale pretrained models and fingerprinting benchmarks further con- firm its effectiveness under prolonged training and realistic deployment scenarios. Together, these results suggest that initialization itself imprints unique and persis- tent identity on LLMs, forming a true "Galtonian" " fingerprint. Code available at https: / thub. com /ne/ 10/11//prr// http//rro__a00 2025 fi ingerprints
INTRODUCTION
LLM fingerprints have recently been proposed as tool to identify, attribute, and trace L.MSs by examining their observable behaviors (Pasquini et al., 2024; Xu et al., 2024; Yoon et al., 2025; Zhang et al., 2024; Zeng et al., 2024). Such methods aim to provide model owners with a verifiable link between suspicious model and its putative original, enabling detection of model theft or unauthorized reuse (Yoon al., 2025; Zhang, 2025).
Much of this literature explicitly borrows the metaphor of biological fingerprints from Francis Galton's Finger Prints (1892) (Galton, 1892):
"A fingepprint is the pattern formed by friction-ridge skin on the fingertips: this ridge configuration is individually unique and essentially permanent across an individual's ''s lifetime."
The analogy suggests that an effective fingerprint should be both unique and persistent, present from the very moment of model's s "birth" initialization. Yet most existing so-called fingerprinting approaches fall short of this standard (Pasquini et al., 2024; Xu et al., 2024; Yoon et al., 2025; Zhang al., 2024; Zeng et al., 2024; Zhang, 2025: Luan al., 2025; Tsai al., 2025; Alhazbi et al., 2025).
"Equal contribution.
1 "Coreeponneecce Tianyang Hu.

---

Published as a conference paper at ICLR 2026
They are defined only after models have been fully trained and converged (finish pretaining), e.g.. extracting patterns from parameters or generated text, and thus capture traits that emerge as the model "grows up" through exposure to data and optimization. In other words, the separability these methods can achieve is less a birthmark of the model itself than an imprint left by its surface training components: data signatures, optimization dynamics, or hyperparameters. Such methods, therefore, function more as post hoc identifiers than as Galtonian fingerprints- thooee innate, immutable marks that accompany a model from its very beginning.
In this work, we propose a stricter notion of an LLM fingerprint: an intrinsic property present at model initialization and detectable at any time of the subsequent training. We observe that untrained models already exhibit seed-dependent preferences (or biases) in token selection. The same prompt induces subtle but reproducible biases in next-token probabilities, and that these biases remain measurable. Building on this observation, we introduce the evaluation method SeedPrints, designed to isolate initialization-bon duration, optimization noise, objective choice. Importantly, our study reveals a limitation in existing evaluation practices: most prior fingerprinting methods assess lineage primarily after fine- tuning, where separability is substantially easier to achieve. In contrast, early pretraining stages constitute a significantly more challenging regime for lineage verification. Under this lens, many existing "fingerprint" methods collapse (see Section 5.1): their discriminative signal weakens or disappears when models are examined from the early stage of training, or when descendants undergo severe distribution shifts in their training data, indicating that they capture signal related to the subtle training attribute, such as data distribution and hyperparameters, rather than intrinsic marks.
fingerprints from confounds arising from data distribution, training
Extensive experiments in Section 5.1 show that our method can even distinguish between models that differ only in their initialization seed, despite an identical training pipeline and data order. Our method reliably detects the lineage of early-stage models with high confidence (e.g.. p-values ranging from 10-3 to 10 10-1)), whereas all prevailing baselines fail to separate them. Moreover, our fingerprint remains persistent under continual training on diverse datasets, while prior baselines merely track the training data distribution and are easily misled by suspicious models that have been continuously trained on very different corpora. Finally, in Section 5.3, we evaluate our method both under standard pretrained foundation model settings and in practical deployment scenarios using the comprehensive LeafBench benchmark (Shao et al., 2025), which includes 696 model pairs spanning 6 model transformations, e.g. instruction tuning, fine-tuning, PEFT, quantization, model merging, and distillation. Our method matches the strongest baseline (nearly perfect) while substantially surpassing all remaining baselines.
RELATED WORK
LLM protection broadly falls into two families: (i) watermarking active fingerprinting methods that insert an identifiable signature into model or its outputs (Xu et al., 2024; Nasery et al.; Tsai et al., 2025; Sander et al., 2024), and (ii) passive fingerprinting methods that extract a signature from a model's pre-existing behaviors without modifying it (Yoon et al., 2025; Zhang al., 2024; Alhazbi et al., 2025; Pasquini et al., 2024; Suzuki et al., 2025; Zhang. 2025).
Watermarking and Active Fingerprinting. Active approaches deliberately implant verifiable identifiers for later ownership checks. Classic techniques include backdoor attacks (Adi et al., 2018; Liet al., 2019; Zhang et al., 2018), digital signatures and hash functions (Guo & Potkonjak, 2018; Zhu et al., 2020). For language models, two common forms are: text watermarks, which bias generation or insert predefined patterns to encode hidden information (Kirchenbauer et al., 2023; Xu et al., 2024; Nasery et al.); and model weight watermarks, which embed identifiers into parameters or link them to secret triggers through fine-tuning (Luan et al., 2025; Li al., 2023). Although backdoor-style fingerprints are relatively straightforward and may persist after moderate fine-tuning (e.g., Dasgupta et al. 2024), these invasíve schemes require control of the training process, making them unsuitable for retroactively marking third-party models.
Passive LLM Fingerprinting, In contrast, passive fingerprinting identifies models by analyzing their intrinsic properties without any modification. Passive fingerprinting techniques vary by model access. With white-box access, signatures are extracted from model weights, leveraging intrinsic properties like the distribution of attention matrices (Yoon et al., 2025), the kernel alignment of internal representations (Zhang et al., 2024), or the stable direction of parameter vectors. In the black-box setting, fingerprinting relies on analyzing input-output behavior. These methods use crafted
2

---

Published as a conference paper at ICLR 2026
Bias Prediction Frequency over Rank
Alignment in Token Preference
0.10
1413
Sams
Min-logit Uniform Random
1413
intralizaiion
Percentage
Correlation
0.05
0.00
5000
10000
30000
Percentage (%)
0.400
Min-hidden
Checkpoint Step
Uniform Random
Top-3
Probable Tokens (Min-Logiit)
0.200
Searl-123:
Taken:
Count:
Taken: ervation,
Count:6
Token: adopted,
0.000
Seed=1000:
Token: ni,
C 9
0
200
400
600
Takan:
Count:
Rank (high + low blas)
Token: David,
Count:
Under uniform random inputs, randomly initialized LLaMA-2-style model exhibits systematic bias in its outputs: certain dimensions are consistently disfavored by the model (i.e., frequently receiving the minimum value across random trials). Such extreme bias occurs in both the logits (top, red) and the final hidden representations (bottom, blue). The dashed line denotes the expected frequency under a uniform output distribution. The arrows in the top panel indicate broken x-axis that omits the low-frequency tail ranks. Upper Right: Continuously trained models exhibit weakly correlated bias profiles across training checkpoints, consistently shifted upward relative to the uncorrelated baseline. Lower Right: The preferred (biased) dimensions depend on the initialization seed; here we illustrate this using argmin-logit tokens as an example.
Figure I: Initialization-bora output bias persists through training. Left:
queries (Pasquini et al., 2024), unique prompt-response pairs (Tsai et al., 2025), stylometry (Alhazbi al., 2025) or iterative nompping-responnee games (lourovitski et al., 2024) to identify a model, though they can be less robust to fine-tuning. However, these methods define their signatures post hoc. Specifically, they identify emergent properties from a completed training process, rather than the innate. "Galtonian" fingerprints present from random initialization that our work seeks to discover.
3 BIASES ORIGINATING FROM INITIALIZATION PERSIST AFTER TRAINING
In this section, we present our key observation that language models exhibit strong prediction biases originating from the random initialization seed, and such biases remain detectable even after training.
Initialization induces seed-specific argmin bias profiles. We evaluate LLaMA-2-style model initialized with seed 123 on 10,000 random input sequences of length 1,024, where tokens are sampled uniformly from the vocabulary. In Figure (Left), we plot the frequency with which each token (upper) and each hidden dimension at the final layer (lower) attains the minimum value across the 10,000 random trials. The pronounced non- uniform shape in both plots indicates the presence of extreme output preference bias patterns. Repeating the experiment across different initialization seeds shows that while the overall magnitude of bias remains stable, the specific argmin dimensions depend on the seed (Figure 1, Lower Right). We refer to Li et al. (2026) for a principled theoretical and empirical analysis this phenomenon. Briefly, the bias originates from inter-sequence representation contraction primarily driven by asymmetric nonlinear activations (together with LayerNorm) in MLP blocks, and is further amplified by self-attention. Different initialization seeds induce different contraction directions, resulting in distinct identity dimensions.
Training preserves cross-sample bias structure. Although training substantially influences output magnitudes, we empirically observe that trained models retain a weak cross-sample bias pattern inherited from their initialized ancestors. Intuitively, we may view the response of a trained model f' at output dimension j as fz(x), (z()) + (z)), where bz(z) captures the irtializaaton-induce bias and (z) reflects training-inde indduedd variation. For the initialized model f. f()); x b, (x). If bias across random inputs x Drand, then the residual bias component could induce weak positive alignment in cross- sample responses, i.e., Corr (f(z), f'(x),) O.
training does not intentionally negate reverse the initialization-induss
We next focus on the most suppressed m dimensions selected from the initialized model, i.c., M == arg min (1 =m Σje, J,. where f == 1/4 E. f(;;) with v; Drand- Starting
3

---

Published as a conference paper at ICLR 2026
Algorithm 1 Distribution Correlation Test on Identity Dimensions
Require: hase model f. suspicious model f"; random inputs X { 1: fingerprint m.;
significance level
Step I: Localize biased dimensions
Compute average outputs f' X
M(f), MM(')) bottom-m dimensions of
8 + n.(f)
(identity dimensions)
Step Form correlation distribution
each 8y E S
5:
4 KendallTaul ((f(s)),,) (ffaalTTTT(((((a))) (a(a)))
6: end for
7: T + [[T]],=1
Step 3: Hypothesis test against null
Construct Trall by applying the same pipetine two Gaussian matrices y0u, ( ~ NO0 NyNahe
Test Ho T =Tull vs. H1 T > Taull
10: Return SameLineage +- 1(p-value a))
from randomly initialized model (seed 42), train for one epoch Open Wex (Gokaslan et al., 2019) and evaluate intermediate checkpoints. For each checkpoint, we compute the correlation between its responses and those of the initialized model on dimensions in M. As uncorrelated baseline, we compute the same statistic between independently initialized models (seed 42 vs. seed 123). We report the mean and standard deviation over the bottom-m dimensions (m 50). As shown in Figure 1 (Upper Right), although the absolute correlation values are small, they are consistently shifted above the uncorrelated baseline and stabilize rather than decaying to zero. This indicates that initialization leaves a weak but statistically tractable fingerprint in the trained model's 's outputs.
4 ALGORITHM
Figure observes that bias profiles between models from the same lineage are weakly correlated, and their correlation scores are distributed above the uncorrelated baselines. However, three challenges remain: (1) the existing bias direction relies on access to the initialization model and its bottom-n ranked dimensions, which are often unavailable in realistic scenarios; (2) although the same- lineaee correlations are distributionally higher, the two distributions overlap, making statistics based solely on mean standard deviation less powerful; and (3) empirically constructing reliable uncorrelated baseline involves a non-trivial trade-off between computational cost and estimation accuracy. We next show how our proposed method addresses these three issues.
Resolve (1): Extract identity dimensions between any two models. The bottom-n ranked dimensions from the initialized model are unavailable when only two trained models are provided (e.g., early checkpoint and late checkpoint). To address this, consider the coset of the two models. If both models inherit bias from the same initialization, their suppressed dimensions expected to exhibit non-trivial overlap and align with shared bias direction. Conversely, if the two models show little or nearly no overlap, they are likely unrelated. We therefore extract dimensions that are jointly suppressed in both models and use this intersection proxy for the latent imitialization-induee identity dimensions.
Formally, let X {r.) where each x; E exd denotes random sequence of length obtained by independently sampling & random vectors from a d-dimensional isotropic Gaussian distribution. For any model g. define the mean output vector ğ == 1 output dimensionality. We identify its suppressed dimensions as
n Σi= (xx)) Rdda, where dout denotes the
My
arg min
mig
Σ gz.
Jl......d []]m m jEJ
Let S My n. denote the intersection of the two sets. We refer to S as the identity dimensions.
In practice, when one of the models is an early checkpoint close to initialization, directly using its bottom-m dimensions as the identity dimensions often yields stronger discrimination than the coset construction. This is intuitive, as early checkpoints more faithfully preserve the muitalizationnindune bias direction. Anchoring such model therefore provides a cleaner estimate of the latent bias.
Resolve (2): Hypothesis test on distribution of correlation statistics. As in Section 3, we obtain [S] empirical Kendall-Tau correlations forming correlation distribution. We then perform hypothesis
4

---

Published as a conference paper at ICLR 2026
Table 1: Comparison of fingerprint behaviors between models initialized with different seeds.
Table 2: Trained models share the same fingerprint behaviors as their initialization (p-value < 0.01).
Logits Output
Hidden State
Logits Output
Hidden State
Seed Pair
Model Pair
t-test
U-test
t-test
U-test
t-test
U-test
t-test
U-test
942 VS. $2000
0.404
0.456
0.357
0.532
42
vs. buse
642 3.33e-3 1.02e-3 2.20e-8 6.28e-8
$123 VS. $22
0.214
0.295
0.678
0,565
$22
gintt
vs. $133
base
7.33e-3 7.09e-6 1.37e-5
$1000 VS. $123
0.219
0.246
0.363
0.335
sinel vs. $1000
$1000
stuse
2.44e-3
5,58e-4 2.81e-3
$2000 vs. $1000
0.282
0.291
0.434
0.481
ginsl
vs, stase
5.63e-3 4.00e-10 1.27e-9
test (e.g., one-sided t-test the distribution is significantly higher than a null distribution of no association, constructed from an uncorrelated baseline. The distributional test avoids relying on fixed summary statistic (e.g., mean or standard deviation), which can be unstable when the two distributions exhibit substantial overlap, and thus provides stronger discrimination in such cases. If the null hypothesis is rejected, we deem f to be derived from f. We declare significance at < 0.01; further details are provided in Algorithm 1.
U-test in Section 5) to evaluate whether this
Resolve (3): Constructing practical null distribution. A final challenge is how to obtain a reliable null baseline without relying on costly empirical comparisons between thousands of independently initialized models. Empirically, applying the same dimension-selection and correlation computation pipeline to independent Gaussian matrices of the same size as the model outputs closely approximates the correlation distribution observed between independently initialized models (see Section B for details). We therefore construct the null distribution in this way as a lightweight surrogate for unrelated models while preserving the dependency structure introduced by selecting identity dimensions.
5 EXPERIMENTS
This section has parts. In Section 5.1, demonstrate that method is genuine fingerprint: (i) it enables birth verification at the seed level, whereas prior methods primarily track training-date distributions and lack this capability; and (ii) it remains verifiable across the entire training lifecycle, while prior approaches are reliable only after convergence-onee the parameters have encoded stable, data-dependent signals. We report results using both logits and hidden stafe outputs, and we test with both the one-sided f-test (f-test) and Mann-Whitee Whitney U test (U test). The consistency across tests indicates that our method robust the choice of hypothesis test. In Section 5.3, we evaluate all methods under practical infringement scenarios using LeafBench (Shao et al., 2025). Across 58 distinct model instances spanning representative post-development techniques, our method matches the best baselines in post-training settings and remains robust diverse deployment transformations.
Baselines We mainly consider four passive fingerprinting baselines (weight- or repressentttionn-baeed)) Intrinsic fingerprint (Yoon et al., 2025) (or PDF in some papers) compares models via the similarity of the layerwise standard-deviation profiles of attention parameters. REEF (Zhang et al., 2024) computes centered-kende -eeeel-lllgnnneeeee (CKA) similarity between feature representations from the same samples across two models. PCS and ICS (Zeng et al., 2024) (or collectively as HuRef in some papers) are weight-similarity methods: PCS flattens all parameters and measures cosine similarity; ICS forms invariant terms from the weights and measures cosine similarity on those invariants. In Section 5.3, we additionally include (weaker) gradient-based fingerprint Gradient (Shao et al., 2025). Following Zhang et al. (2024), we use a 0.8 similarity threshold for binary decisions.
Note, in all experiment tables, cell colors indicate lineage: with green denotes models from the same source, and red denotes different sources. For example, iin't vs. Juse compares a model initialized with seed 42 and its continued compares a seed-2000 initialization with a model trained from a seed-42 initialization, hence red. Additionally, v denotes correct detection, while x denotes an error.
counterpart, hence green. By contrast, snss vs. baae
5

---

Published as a conference paper at ICLR 2026
Table Fingerprint persistence under continual training on diverse datasets (base model: seed 1000, corpus openwebtext). U-test refers to the Nann-Whitngr U test.
Setting
Ours (logits)
Ours (hidden)
Baselines
Continual corpus (seed)
t-test
U-test
t-test
U-test
Intrinsic
REEF
PCS
ICS
rinyStorlesv__lcanee (1000)
0
0
0
7.77e-89
1.000'
0.759 0.999* 0.996'
TinyStoriesV2_clean cleaaned (123)
1.000
1.00
0.943
0.902"
0.950
0.658 0.332 0.012
the_stack (1000)
1.73c-287
0r
3.09c-69*
0.489
0.557 0.585 x 0.123*
the stack (123)
0.616
0.479
0.732
0.831
0.445%
0.580 0.301 0.026
Table 4: The same dataset and training order do not shape fingerprint behaviors to be identical across different initializations.
10
Hidder State Teeet
Logit Mar H
90
Hidder Stale Vann-Whatrey Loghts Tokuat
p-0.01
Model Pair
Logits Output
Hidden State
10
t-test
U-test
t-test U-test
10 10
ArLit $12 vs. $100
0.484
0.500
0.385
0.486
8 Anst
1000 vs. $2000
0.946
0.956
0.035
0.096
Mnst 42
vs. $123
bicee
0.598
0.589
0.426
0.337
Checkpoint Index
snit 2000 VS.
buse
0.756
0.781
0.388
0.287
Figure 2: Fingerprint verifies lineage at every checkpoint (p-values < 0.01).
42
5.1 BIRTH-TO-LIFECYLLEE "BIOMETRIC" FINGERPRINTING
We train 12-layer, 12-head LLaMA-style models (Touvron et al., 2023) with RoPE (Su al., 2021) and Qwen-style models (Team, 2024) from scratch. Because our random baseline is stochastic, we report p-values averaged over 10 independent trials and adopt significance level of a 0.01. Importantly, the absolute magnitude of extremely small p-values is not meaningful: once p falls below numerical and sampling noise (e.g.. < 10 10-2"), values like 10 (0-260, should not be interpreted as stronger evidence than 10 100200000 both decisively reject the null. In the main paper, we present results for LLaMA-style models and defer those for Qwen-style models to Section D.3. The overall conclusions are consistent across the two.
Different initialization seeds produce distinct fingerprints Table reports p-values from our correlation tests between pairs of models initialized with different random seeds (42, 123, 1000, and 2000) under the t-test and Utesst. All p-values are consistently > 0.01, indicating that our method reliably distinguishes models trained from different seeds. This shows that distinct seeds yield distinct fingerprint behaviors, allowing models to be separated "at birth."
Training preserves the initialization fingerprint. Table 2 compares each initialization model snmtt with its descendant g buse trained on the WebTet dataset (Gokaslan et al., 2019) (+= 10B tokens). Across all seed-model pairs, p-values are consistently < 0.01, indicating that their bias profiles remain strongly correlated and thus share a common lineage. In short, the trained model inherits the same fingerprint as its initialization. We also evaluate baseline methods (results Appendix 8); without exception, they fail to distinguish across seeds, which in turn suggests their separability stems from training-induus induced artifacts rather than initialization.
Identical data and order do not make fingerprints converge In Table 4, all four "suspicious" models s bause for E {42, 123, 100, 2000) are trained on exactly the same corpus WebTex)) and in the same data order (we fix the training seed to lock the data order); the only difference lies in their initialization seeds i. Across all cross-seed pairs, p- remain consistently > 0.01, in sharp contrast to the near-zero values in Table 2. That is, fingerprints remain seed-specific under identical data and curriculum.
Continual training diverse datasets does not confound the fingerprint The purpose of our earlier experiments is solely to demonstrate the strengths of our SeedPrints: it can act as a biometric fingerprint. From copyright perspective, the inability of prior works to distinguish models with

---

Published as a conference paper at ICLR 2026
Table 5: Fingerprinting results vs. LLaMA-2-7B, Each row compares a target model against LLaMA- 2-7B. U-test reports the p-value from our hidden-state correlation test (< 0.01 indicates a strong signal). Intrinsic, REEF, PCS, and ICS report similarity scores (higher better).
Model
# Tokens
U-test p (< 0.01)
Intrinsic
REEF (1) PCS (t)
ICS (1)
Llama-2-financ-78 (Heenan, 20233)
5M
1.34 10 10-41v 41V
1.0000
0.9950
0.9979
0.9952
Vicuna- 1.5-7B (Chiang al., 2023)
370M
1.49 x 10 10-96r 96V
1.0000
0.9985
0.9985
0.9949
Wizurdmath-777 (Luo et al., 2023)
1.8B
4.09 x 10 100V
1.0000
0.9979
1.0000
0.9994
Medituon- (Chen al., 2023)
48B
5.212 x 10 10-1/
0.9990
0.9978
1.0000
0.9817
CodeLLama-778 (Meta AI, 2024)
500B
2.008 x 10 10-3/ 3V
0.9480
0.9947
0.6863
0.3369*
LB (Azerbayev al., 2023)
700B
6.95 x 10 10-9/
0.9470
0.9984
0.6682
0.2905*
different seeds is not a weakness, since initialization seeds have no clear copyright status. The real fragiliiy is that their attribution can be easily misled by data distribution, faillin to recognize lineage when the training distribution shifts substantially during continued training.
In Table 3. we continue training a base model (seed 1000, pretrained on OpenWebText Gokaslan et al. (2019)) on two very different datasets: TinyStories Eldan & Li (2023) (synthetic children's stories) and The Stack Kocetkov et al. (2022) (permissively licensed GitHub code). We compare (i) true descendants trained from the base, versus (ii) distractors derived from different base model (initialized with seed 123 and trained with a different data order on OpenWebTexty then continued training on the same corpus. The question is whether attribution methods can identify which descendant truly shares lineage with the base.
We find that prior baselines all fail under the code setting (The Stack), misclassifying true descendants as distractors. This indicates that they largely track domain similarity rather than lineage identity: TinyStories is closer in distribution to the pretraining corpus (Open WebText), while The Stack diverges sharply; such a large distribution shift can easily bypass detection. In contrast, our method correctly attributes lineage across both corpora. Hence, our fingerprint is not a proxy for data distribution: it survives substantial domain shift and persists beyond the initial pretraining stage.
5.2 ALL-STAGE VERIFIABLE FINGERPRINTS
Controlled pretraining trajectory. We first conduct a controlled pretraining experiment on Open- WebText, where we know the initialization seed. Each intermediate checkpoint is treated as the base, and we test whether our method can reliably identify its offspring along the same training trajectory. All variants consistently recognize the suspect model belonging to the same lineage, with p-values remaining below the 0.01 threshold (Figure 2). This confirms the stability of SeedPrints under continuous optimization.
Large-scale pretraining. To evaluate lineage under realistic foundation model training, we further analyze ten checkpoints from the OLMo-2-7B (OLMo et al., 2024) Stage-1 run (5B - 3.9T tokens), where we treat the final checkpoint as the target model and all previous checkpoints as tested ancestors. Unlike the controlled OpenWeeTTe setting, this stage spans heterogeneous corpora, longer optimization, and large-scale representation drift. We plot 1 - p for consistent directionality with basil baselines (higher indicates stronger evidence).
In Figure 3, we find that all existing baselines fail to verify lineage during early pretraining (c.g.. ICS 0, PCSK 0.3 and REEF< 0.6 within the first trillion tokens), whereas SeedPrints remains consistently verifiable and strengthens throughout training. These results highlight a critical gap in existing evaluations: most prior work focuses on verifying lineage only after fine-tuning, where lineage detection is substantially easier. However, our findings show that large-scale pretraining itself can dramatically reshape representations and strengthen lineage signals, potentially giving a false sense of safety. We argue that lincage verification must prioritize the carly stages of pretraining when misuse is most difficult to detect, rather than relying solely on late-stage checkpoints.
Large-scale finetuning. We further compare our method with existing baselines under standard evaluations on finetuning stage. In particular, we test suspect models fine-tuned from Llama-2-77 2- (base model) with data volumes ranging from 5 million to 700 billion tokens. The suspects include diverse downstream variants such as 1aima-2-finace-777 7b (Heenan, 2023), Vicuna-1.5-78 (Chiang et al., 2023), WizhrdMath-7b (Luo et al., 2023), Chinese-LLaaMA-2-7b (Chen et al., 2023), Code-
7

---

Published as a conference paper at ICLR 2026
SeedPrints vs Baselines: Across Training Checkpoints
1.0
0.8
0.6
0.4
SeodPrints (Gurs)
0.2
Intrinsic
PCS
0.0
REEF
5B
8698
1.3T
1.7T
2.2T
2.6T
3.0T
3.5T
3.9T
Training Tokens
Figure Lineage verification performance across OLMo-7B Stage-l checkpoints for all methods.
Llama-7b (Meta AI, 2024), and Llemma-7b (Azerbayev et al., 2023). Their fine-tuning data volumes are 5M, 370M, 1.8B, 13B, 500B and 700B tokens, respectively. As shown in Table 5, our method consistently maintains p < 0.01 across all settings.
5.3 ROBUSTNESS UNDER REALISTIC DEPLOYMENTS
Real-world deployments routinely apply parameter-altering adaptations to foundation models (e.g.. fine-tuning, PEFT, quantization), which can weaken distort fingerprint signals. Our method is designed to remain effective this regime. To evaluate its reliability under such realistic conditions, adopt LeafBench (Shao et al., 2025), benchmark for language-modee copyright auditing.
Across Source Models. We consider two types of source models for auditing: Pre-Trained (PT) and Instruction-Tiune (IT). For each source type, the task is distinguish derivative models (obtained by post-training adaptations) from independent models (trained without using the source weights). This mirrors common auditing scenarios where one must test lineage claims for either PT base its IT variant. For extact models, Our evaluation covers 58 models in total; the full list of model names and HuggingFace repositories provided in Appendix E.
Metrics. We evaluate model detection performance using two primary metrics: (1) the Area Under the ROC Curve (AUC) and (2) the Kolmogorov-S -miimov (KS) statistic (Berger & Zhou, 2014), which measures the maximum distance between the score distributions of derivative and source models. Our method produces per-model p-values from statistical tests, which differ fundamentally from the similarity scores used by baseline methods. While similarity scores are linear measures where diffcrences have proportional meaning, p-values are tail probabilities indicating the rarity of an observation under the null hypothesis- comparison with baseline methods that compute AUC from similarity scores, we deliberately evaluate method under an unfair condition by converting our p-values into scores using = 1 p. Despite this unfavorable conversion, our approach still achieves comparable or superior performance. For evaluation, (i) compute AUC for all methods by sweeping thresholds over the scores (for our method, the optimal threshold obtained by sweeping also lies at the confidence level 1 - a = 0.99), and (ii) report the KS statistic as a threshold-frec measure of distributional separability. This protocol ensures compatibility with baseline pipelines while preserving the statistical interpretability of our method. We do not compute Manhattan distance, as the non-linear nature of p-values makes absolute-valus distances between them mathematically meaningless.
cannot be interpreted on a linear scale. To enable
Note, in most practical scenarios, calibrating an optimal threshold is infeasible, preventing the application of similarity-based methods. While we convert our y-values into similarity scores and report AUC for comparability, our method fundamentally differs: provides direct statistical test. Specifically, given any pair of models, our approach outputs p-value that allows definitive claim about whether the two models originate from the same lineage without relying on ambiguous thresholds and thus ensuring reliable verification.
Parameter-AAlerrng Techniques. To assess robustness under weight modifications, we evaluate models produced via (1) Instruction tuning (Instruct), (2) General- General-purpose fine-tuning (Finetune), (3) Parameter-efficiee tuning (PEFT), (4) Quantization, (5) Model merging (Merge), and (6) Distillation.
8

---

Published as a conference paper at ICLR 2026
Table 6: Performance comparison of LLM fingerprinting methods across different source models. "PT" and "IT" refer to using the pre-trained models and instruction-ue -utnee models as source models, respectively. Note that the "Overall" scores are computed over all test combinations, rather than by averaging per-family results.
Qwen2.5.7B
Qwen2.5.14B
Llama3.1-88
Mistral-7B
GemmaZ-2B
LJama2-7B Overall
Method
Metric
PT
IT
IT
PT
PT
IT
AUC
0.796
0.862
0,947
0.913
1.000
0.999
0.997
0.997
0.963
0.968
0.750
0.706
0.914
REEF
KS Statistie f
0.492
0.633
0.875
0.735
0.987
0.987
0.975
0.980
0.950
0.837
0.750
0.694
0.739
AUC f 0.657
0.709
0.763
0.756
0.897
0.885
0.872
0.641
1.000
1.000
0.650
0.722
0.778
Gradient
KS Statistic 1 0.354
0.385
0.500
0,550
0.646
0.662
0.725
0.425
1.000
1,000
0.275
0.475
0.4828
AUC f
1,000
1.000
0.997
0.997
1,000
1.000
1.000
1.000
1,000
1,000
1.000
1,000
0.994
ICS
KS Statistic 1 1,000
1.000
0.975
0.979
1,000
1.000
1.000
1.000
1,000
1,000
1.000
1.000
0,943
AUC f
1,000
1.000
1.000
1.000
1,000
1,000
1.000
1.000
1,000
1,000
1.000
1,000
0.995
Intrinsic
KS Statistic 1 1,000
1.000
1.000
1.000
1,000
1.000
1.000
1.000
1.000
1,000
1.000
1.000
0.989
AUC
0.999
1.000
1.000
1.000
1.000
0,929
1.000
1.000
1.000
1.000
1.000
1.000
0.992
SoodPrints
KS Statistic t 0.985
1.000
1.000
1.000
1.000
0,923
1.000
1.000
1.000
1.000
1.000
1.000
0.992
Table Performance of LLM fingerprinting methods across different parameter-altered techniques.
Method
Metric
Instruct
Finetune
PEFT
Quantization
Merge
Distillation
Overall Avg
AUC
0.996
0.873
0.994
0.871
0.954
0.858
0.924
REEF
KS Statistic f
0.949
0.637
0.940
0.812
0.851
0.615
0.801
AUC f
0.864
0.745
0.724
0.697
0.827
0.840
0.783
Gradient
KS Statistic f
0.733
0.469
0.463
0.537
0.550
0.650
0.567
AUC1
1,000
0.987
1.000
1.000
0.981
1.000
0.995
ICS
KS Statistic f
1,000
0.925
1,000
1.000
0.941
1.000
0.978
AUC
1,000
0.981
1.000
1.000
1.000
1.000
0.997
Intrinsic
KS Statistic t
1.000
0.971
1,000
1.000
1.000
1.000
0.995
AUC KS Statistic t
1.000
0.970
1.000
1.000
1.000
1.000
0.995
SeedPrints
1.000
1.000
0.969
1.000
1.000
1.000
0.995
5.3.1 WASELY EFFECTIVENESS
There are six model familics in total. For each family, we treat both its pretrained base model (PT) and its instruction-tune
model (IT) separate source models and compare them with all other models in the pool of 58. The results are reported in Table 6. Note that each column corresponds to one family (AUC is then computed over the corresponding tests, using tumily-calibrater threshold), whereas the overall score is computed across all test pairs with a global threshold. Thus, the overall score is not equal to the average of the per-family scores. All three our SeedPrints-an are essentially saturated across all alterations (overall AUC of 0.995, 0.994, and 0.992, respectively: KS of 0.989. 0.943, and 0.992). For these methods. the family-wise scores are. in most cases, the full score. By contrast, the remaining two baselines, REEF and Gradient, perform substantially worse. These results indicate that, while our method is primarily designed to capture intrinsic seed-level fingerprints, it effectively functions biometric-like fingerprint that provides reliable and persistent identity tracking across diverse model families.
ICS, and
5.3.2 ALTERATION-WEL ROBUSTNESS
Across alteration types, Instruct is consistently the easiest case (even the weakest baseline, Gradient, can achieve AUC 0. 0.864), whereas the most disruptive are PEFT, Finetune and Quantization: REEF AUC drops by about 0.12 compared to Instruct (0.996 + 0.873 0.873/0..71) 00.871). and Gradient degrades sharply under PEFT/Finetune (0.864 - 0.724/0.745). The effects of Merge and Distillation are more mixed: REEF (0.954) and Gradient (0.827) are only mildly affected by Merge, while ICS shows a small drop (0.981 vs. mostly 1.000). Distillation degrades REEF (AUC 0.858) but leaves other methods largely intact. We attribute these trends to method design: REEF and Gradient depend on fine-ge -heggraindd forward/backwward dynamics that are easily perturbed by parameter updates or quantization, whereas Intrinsic. ICS, and our SeedPrints leverage static, initialization-tier signals. Notably, SeedPrints remain near- serfeeereect across all six transformations, showing robustness even under aggressive parameter changes and shifts introduced by Finetune, PEFT, and Merge.

---

Published as a conference paper at ICLR 2026
5.4 ABLATION STUDY ON HYPERPARAMETERS
Our method has three hyperparameters: the number of random input sequences n, the length of each random sequence I, and the fingerprint size m. The first two control the overall size of random inputs, while m specifies how many low-mean coordinates are used for testing. Throughout the main experiments in Sections 5.2 and 5.3, we set n=2000, 1=1024, and mdn.. where dout denotes the hidden state dimensionality, making the method adjustable for different open-source models. Here, we vary each hyperparameter while fixing the remaining ones their default values (i.e., n 2000, m 400, = 768), and report empirical accuracy and false positive rates under fixed significance level a = 0.01 Table 4.
Figure 4: Ablation study under signifi- cance level a 0.01.
We observe that increasing n and I consistently improves identification performance. This trend is explained by our theoretical analysis in Section C.2, which shows that accurately recover the underlying population-level bias, the
Ablation on n
200
500
2000
required number of queries must satisfy n = = log deu 3²
ACC
0.7368
0.7778
0.9375
Empirical FPR
0
0
where y is the bias margin separating biased coordinates from the remaining dimensions. This means a target out- put dimension requires a minimal number of samples for accurate detection, explaining why smaller leads to sub- optimal performance. On the other hand, increases with due to the accumulation effects, meaning longer sequences strengthen separation. Therefore, under fixed (insuffi- cient) number of sequences, increasing | reduces the quired query count and improves detection.
Ablation on /
512
768
1024
ACC
0.9048
0.9375
1.0000
Empirical FPR
0.04
0
0
Ablation on m
200
400
800
ACC Empirical FPR
0.9375
0.9375
0.875
0.04
O
0.1463
The effect of m is non-monot. When m is too small, the selected coordinates unstable due to insufficient empirical observations to form a reliable distribution. When m is too large, the inclusion of many irrelevant coordinates introduces noise that shrinks y, which would require quadratically more samples to compensate. Empirically, choosing m in the range 200-400 works generally well across model sizes, matching our heuristic of 0.1oout given typical dout e [3000, 5000).
CONCLUSION
In this work, we introduced SeedPrints, a stronger, intrinsic notion LLM fingerprinting that traces model's lineage back its random initialization. We showed that initialization itself leaves persistent "Galtonian" fingerprint on neural language models. Therefore, untrained models already exhibit stable, seed-dependent coken-preference patterns, and these biases persis--statistien and mecaurabbyy-trooughoo
training. Building on this observation, we proposed a distribution- correlation test over identity dimensions that detects shared lineage with calibrated significance. Across LLaMA- and Qwen-style families and broad suite of realistic deployments (e.g.. continued pretraining on divergent corpora, instruction tuning, PEFT, quantization, merging, and distillation), SeedPrints enables -iirfc-to-lifecclle" verification and remains robust under domain shift, offering practical tool for provenance and copyright auditing.
Limitations and Future Directions Although this work uncovers seed-born,
biases that can serve as fingerprint signals, method remains within the white-box paradigm and may not apply when only restricted API access is available. However, we stress that the pursuit of true fingerprints for LLMs is still an early stage: most existing approaches rely on training-induced or alignment-driven artifacts and therefore cannot robustly provide protection at the pretrained stage. Therefore, reliable all-stage fingerprints even more urgently needed. See full discussions in Section A.2.
10

---

Published as a conference paper at ICLR 2026
REFERENCES
Yossi Adi, Carsten Baum, Moustapha Cisse, Benny Pinkas, and Joseph Keshet. Turning your weakness into strength: Watermarking deep neural networks by backdooring. In 27th USENIX security symposium (USENIX Security 18), pp. 1615-1631, 2018.
Saeif Alhazbi, Ahmed Hussain, Gabriele Oligeri, and Panos Papadimitratos. Lims have rhythm: Fingerprinting large language models using inter-token times and network traffic analysis. IEEE Open Journal of the Communications Society, 2025.
Zhangir Azerbayev, Hailey Schoelkopf, Keiran Paster, Marco Dos Santos, Stephen McAleer, Albert Q Jiang, Jia Deng, Stella Biderman, and Sean Welleck. Llemma: An open language model for mathematics. arXiv preprini arXiv:2310. 10631, 2023.
Vance W Berger and YanYan Zhou. Kolmogorov-smirnov test: Overview. Wiley statsref: Statistics reference online, 2014.
Nicholas Carlini, Steve Chien, Milad Nasr, Shuang Song, Andreas Terzis, and Florian Tramer. Membership inference attacks from first principles. 2022 IEEE symposium on security and privacy (SP), pp. 1897-1914. IEEE, 2022.
Zeming Chen, Alejandro Hernández-Cano, Angelika Romanou, Antoine Bonnet, Kyle Matoba, Francesco Salvi. Matteo Pagliardini, Simin Fan, Andreas Köpf, Amirkeivan Mohtashami, Alexan- dre Sallinen, Alireza Sakhaeirad, Vinitra Swamy, Igor Krawczuk, Deniz Bayazit, Axel Marmet, Syrielle Montariol, Mary-Anne Hartley, Martin Jaggi, and Antoine Bosselut. Meditron-70b: Scaling medical pretraining for large language models, 2023.
Wei-Lin Chiang, Zhuohan Li, Zi Lin, Ying Sheng, Zhanghao Wu, Hao Zhang, Lianmin Zheng, Siyuan Zhuang, Yonghao Zhuang. Joseph E. Gonzalez. lon Stoica, and Eric P. Xing. Vicuna: An open-source chathot impressing gpt-4 with 90%* * chatgpt quality, March 2023. URL https:
//mssys.org//////o.....oo..oo... bl og/
ootogg//0022 2023-002200000 03- 30 vi00-viie cuna/.
Agnibh Dasgupta, Abdullah Tanvir, and Xin Zhong. Watermarking language models through language models. arXiv preprint arXiv:2411.0509, 2024.
Ronen Eldan and Yuanzhi Li. Tinystories: How small can language models be and still speak coherent english? arXiv preprint arXiv:2305.07759, 2023.
Francis Galton. Finger prints. Number 57490-57492. Cosimo Classics, 1892.
Aaron Gokaslan, Vanya Cohen, Ellie Pavlick, and Stefanie Tellex. Openwebtext corpus, http: //Skylion007 git io/ /ttp///eeeeetCorppu.u. 2019.
Sylvain Gugger, Lysandre Debut, Thomas Wolf, Philipp Schmid, Zachary Mueller, Sourab Man- grulkar, Marc Sun, and Benjamin Bossan. Accelerate: Training and inference at scale made simple, efficient and adaptable. https: / / hutthu..... com cww./om/////eeeeeeee hugg i n.g face acccleraate 2022.
Jia Guo and Miodrag Potkonjak. Watermarking deep neural networks for embedded systems. In 2018 IEEE/ACM international conference on computer-aice aided design (ICCAD), pp. 1-8. IEEE, 2018.
Collin Heenan. Llama2-7b- 7b-iianneeeeeeeeee (hugging face model). https: http::////hhhggggggggegcc co/ cxllin/Llame 7-- iance,, 2023. Fine-tuned from NousResaarhh/LLmm-227bbiitt MIT
License; accessed 2025-09-02.
Dmitri lourovitski, Sanat Sharma, and Rakshak Talwar. Hide and seek: Fingerprinting large language models with evolutionary learning. arXiv preprint arXiv:2408.099 02877,, 2024.
John Kirchenbauer, Jonas Geiping, Yuxin Wen, Jonathan Katz, Ian Miers, and Tom Goldstein. A watermark for large language models. In International Conference Machine Learning, pp. 17061-17084. PMLR, 2023.
Denis Kocetkov, Raymond Li, Loubna Ben Allal, Jia Li, Chenghao Mou, Carlos Muñoz Ferrandis, Yacine Jemnite, Margaret Mitchell, Sean Hughes, Thomas Wolf. Dzmitry Bahdanau. Leandro von Werra, and Harm de Vries. The stack: 3 tb of permissively licensed source code. Transactions on Machine Learning Research (TMLR), Preprint, 2022.

---

Published as a conference paper at ICLR 2026
Jachoon Lee, Lechao Xiao, Samuel Schoenholz, Yasaman Bahri, Roman Novak, Jascha Sohl- Dickstein, and Jeffrey Pennington. Wide neural networks of any depth evolve as linear models under gradient descent. Advances in neural information processing systems, 32, 2019.
Peixuan Li, Pengzhou Cheng, Fangqi Li, Wei Du, Haodong Zhao, and Gongshen Liu, Plmmark: secure and robust black-box watermarking framework for pre- pre-trained language models. In Proceedings of the AAAI Conference on Artificial Intelligence, volume 37, pp. 14991-14999, 2023.
Siquan Li, Yao Tong, Haonan Wang, and Tianyang Hu. Transformers are born biased: Struc- tural inductive biases at random initialization and their practical consequences. arXiv preprint arXiv:2602.05927, 2026.
Zheng Li, Chengyu Hu, Yang Zhang, and Shanqing Guo. How to prove your model belongs to you: A blttr-wa based framework to protect intellectual property of dnn. In Proceedings of the 35th anmual computer security applications conference, pp. 126-137,0019.
Xiaokun Luan, Zeming Wei, Yihao Zhang, and Meng Sun. Robust and efficient watermarking of large language models using error correction codes. Proceedings on Privacy Enhancing Technologies, 2025.
Haipeng Luo. Qingfeng Sun, Can Xu, Pu Zhao, Jianguang Lou, Chongyang Tao, Xiubo Geng, Qingwei Lin, Shifeng Chen, and Dongmei Zhang. Wizardmath: Empowering mathematical reasoning for large language models via reinforced evol-instruet. arXiv preprint arXiv:2308.09583, 2023.
Meta AI. Codellama-7b-htf Code Ilama base 7b model (hugging face). https:/
h http///huggggggacgeeeeee ca/ /odeo amaa / Codel. ama- 7b-hf, 2024. Base billion-parameter Code Llama model for code synthesis and understanding; trained between January and July 2023; licensed under Meta Llama 2 license; accessed 2025-09-02.
Anshul Nasery, Jonathan Hayase, Creston Brooks, Peiyao Sheng, Himanshu Tyagi, Pramod Viswanath, and Sewoong Oh. Scalable fingerprinting of large language models. In ICLR 2025 Workshop on Building Trust Language Models and Applications.
Team OLMo, Pete Walsh, Luca Soldaini, Dirk Groeneveld, Kyle Lo, Shane Arora, Akshita Bhagia, Yuling Gu, Shengyi Huang, Matt Jordan, Nathan Lambert, Dustin Schwenk, Oyvind Tafford, Taira Anderson, David Atkinson, Faeze Brahman, Christopher Clark, Pradeep Dasigi, Nouha Dziri, Michal Guerquin, Hamish Ivison, Pang Wei Koh, Jiacheng Liu, Saumya Malik, William Merrill, Lester James V. Miranda, Jacob Morrison, Tyler Murray, Crystal Nam. Valentina Pyatkin, Aman Rangapur, Michael Schmitz, Sam Skjonsberg, David Wadden, Christopher Wilhelm, Michael Wilson, Luke Zettlemoyer, Ali Farhadi, Noah A. Smith, and Hannanch Hajishirzi. 2 olmo furious, 2024. URL https: / arxivv org aats///bb/aa http//aaaa//.....e /2000.5000000 00656.
Dario Pasquini, Evgenios M Kornaropoulos, and Giuseppe Ateniese. Llmmap: Fingerprinting for large language models. arXiv preprint arXiv:2407.15847, 2024.
Tom Sander, Pierre Fernandez. Alain Durmus, Matthijs Douze, and Teddy Furon. Watermarking makes language models radioactive. Advances in Neural Information Processing Systems, 37: 21079-21113, 2024.
Shuo Shao, Yiming Li. Yu He, Hongwei Yao, Wenyuan Yang. Dacheng Tao, and Zhan Qin. Sok: Large language model copyright auditing via fingerprinting. arXiv preprint a5Xiv:2508. 19843, 2025.
Jianlin Su, Yu Lu, Shengfeng Pan, Bo Wen, and Yunfeng Liu. Roformer: Enhanced transformer with rotary position embedding. In Proceedings of the 59th Annual Meeting of the Association for Computational Linguistics (ACL), pp. 124. Association for Computational Linguistics, 2021.
Teppei Suzuki, Ryokan Ri, and Sho Takase. Natural fingerprints of large language models. arXiv preprint arXiv:2504.14877 2025.
Qwen Team. Qwen2 technical report. arXiv preprint 10Xiv:2407.10671, 2024.
12

---

Published as a conference paper at ICLR 2026
Hugo Touvron, Thibaut Lavril, Gautier Izacard, Xavier Martinet, Marie-Anne Lachaux, Timothée Lacroix, Baptiste Rozière, Naman Goyal, Eric Hambro, Faisal Azhar, et al. Llama: Open and efficient foundation language models. arXiv preprint arXiv:2302.13971, 20233
Yun-Yun Tsai, Chuan Guo, Junfeng Yang, and Laurens van der Maaten. Rofl: Robust fingerprinting of language models. arXiv preprint arXiv:2505. 1tti::555..2688. /2682, 2025.
Thomas Wolf, Lysandre Debut, Victor Sanh, Julien Chaumond, Clement Delangue, Anthony Moi, Pierric Cistac, Tim Rault, Rémi Louf, Morgan Funtowicz, Joe Davison, Sam Shleifer, Patrick von Platen, Clara Ma, Yacine Jernite, Julien Plu, Canwen Xu, Teven Le Scao, Sylvain Gugger, Mariama Drame, Quentin Lhoest, and Alexander M. Rush. Transformers: State-of-the-ar natural language processing. In Proceedings of the 2020 Conference Empirical Methods in Natural Language Pro- cessing: System Demonstrations, pp. 38 38-45, Online, October 2020. Association for Computational Linguistics. URL www.t::pp/...m......m www. . web. org/ hnt.//ntooolllooooooooo hoy 2020 emwnl-ddemee 6.
Jiashu Xu, Fei Wang, Mingyu Derek Ma, Pang Wei Koh, Chaowei Xiao, and Muhao Chen. Instruc- tional fingerprinting of large language models, arXiv preprint arXiv:2401.12255, 2024.
Do-hyeon Yoon, Minsoo Chun, Thomas Allen, Hans Müller, Min Wang, and Rajesh Sharma. Intrinsic fingerprint of Ilms: Continue training is not all you need to steal model! arXiv preprint arXiv:2507.03014, 2025.
Boyi Zeng, Lizheng Wang, Yuncong Hu, Yi Xu, Chenghu Zhou, Xinbing Wang, Yu Yu, and Zhouhan Lin. Huref: Human-readable fingerprint for large language models. Advances in Neural Information Processing Systems, 37:126332-126366 2024.
Jialong Zhang, Zhongshu Gu, Jiyong Jang, Hui Wu, Mare Ph Stoecklin, Heqing Huang, and Ian Molloy. Protecting intellectual property of deep neural networks with watermarking. In Proceedings of the 2018 Asia conference on computer and communications security, pp. 159 172, 2018.
Jie Zhang, Dongrui Liu, Chen Qian, Linfeng Zhang, Yong Liu, Yu Qiao, and Jing Shao. Reef: Representation encoding fingerprints for large language models. arXiv preprint arXiv:2410.14273. 2024.
Ruichong Zhang. Matrix-driven instant review: Confident detection and reconstruction of Ilm plagiarism on pc. arXiv preprint urXiv:2508.06309 2025.
Renjie Zhu, Xinpeng Zhang, Mengte Shi, and Zhenjum Tang. Secure neural network watermarking protocol against forging attack. EURASIP Journal on Image and Video Processing, 2020(1):37, 2020.
13

---

Published as a conference paper at ICLR 2026
A LLM USAGE AND LIMITATION DISCUSSIONS
A.1 LLM USAGE
We use Al assistants, i.e. ChatGPT and Gemini, for writing and formatting support. Their use covers grammar and style checks, improving clarity of figure and table captions, and other surface-level edits. For programming- assistants, e.g., for code auto-completion and debugging hints.
tasks, we occasionally use GitHub Copilot and Claude as coding
A.2 LIMITATIONS AND FUTURE DIRECTIONS
Although this work uncovers seed-born, waining-persiisen biases that can serve as fingerprint signals, our method remains within the white- box paradigm. may not apply when only restricted API access is available; for instance, when responses deterministic or safety-filtered. However, emphasize that the pursuit of true fingerprints for LLM is still at a very early stage, even under the white-box setting. Most existing fingerprinting approaches rely training-induced signatures alignment behaviors, and thus cannot robustly protect pretrained or early-stage models. In analogy human fingerprints, which remain stable regardless of age environment, believe that true LLM fingerprint should persist across continued training and deployment contexts rather than reflect incidental training artifacts. Achieving this particullrry under black-box access, where there remains a substantial performance gap compared to white-box direction and requires substantial future work. We hope this work motivates further research toward establishing principled, robust fingerprinting frameworks for large models.
remmanss a challenging open
B VALIDATION OF THE GAUSSIAN NULL DISTRIBUTION
Our hypothesis test evaluates whether two models share lineage by comparing the distribution of their correlation statistics over random inputs against null distribution. The null distribution aims to model the correlation distribution between two independent models when evaluated on random inputs.
B. EMPIRICAL NULL DISTRIBUTION FROM INITIALIZED MODELS
The cleanest way instantiate complete independence is compare two independently initialized models (with different random seeds and no training). We estimate this empirical null by evaluating 2500 such pairs and computing Kendall-Tau correlations 10,000 random inputs (as in Section 3). Figure shows that the empirical null distribution closely matches a Gaussian distribution.
Initial Model Correlation Distribution
1200
Int Modal Comelation
Gauesian
800
Density
200
-0.0010
-0.005
0.0000
0.0005
0.0010
Kendall Tau Correlation
Figure 5: Empirical correlation distribution over ~ 2500 pairs of independently initialized models, evaluated on random inputs, closely matches Gaussian distribution centered at zero.
14

---

Published as a conference paper at ICLR 2026
Constructing the Null Distribution in SeedPrints. Approximating the output of neural networks as Gaussian distributions is a widely adopted and effective practice in the auditing literature (Carlini et al., 2022). This practice is supported by both empirical evidence and theoretical applications, which show that model output (e.g. logits) often exhibit Gaussian-like behavior across diverse model architectures, data types, and tasks (Lee et al., 2019). Therefore, in work, and based the previous observation of Gaussian-like Null distirbution (Figure 5), we naturally consider Gaussian as a proxy of model output.
Since two random-initializee models have independent parameter matrices, this makes their outputs independent conditional on the same inputs. Therefore, sampling two independent Gaussian matrices is natural surrogate for two independently initialized models under the null. This provides straightforward way to construct the null:
1. Sample two Gaussian matrices of shape Nxdd,,,,
2. Extract the most biased dimensions by selecting the bottom-m ranked average dimensions,
3. Take the intersection set S and compute correlations on it.
Intuitively, selecting S introduces dependencies among dimensions. However, applying the same selection procedure Gaussian samples preserves this dependency structure under the raw null.
C THEORETICAL PROPERTIES OF OUR METHODS
Our goal is to identify the output dimensions that the initialized model is least likely to predict under random inputs. These dimensions correspond to directions that the model inherently disfavors due to initialization-nnduueed bias (not dependent on the input distribution), forming the basis of stable fingerprint signal.
Setup. Consider model g with fixed parameters and random inputs x. (uniform random tokens or random embeddings as in the main paper). For each output coordinate (i.e., index) j E [domt] 9j (1) thus random variable and its population mean Zj == [=]) The population-level directions we aim to capture correspond to the m output dimensions with the smallest expected values, formally defined as
M arg
min J
95 JEJ
Let (#(1)... ..... Iid.a) denote the sorted means in non-decreasing order, and write
r(m) (m+1) (mm)
for the population "gap" between the m-th and (m+1) 11)th smallest means. This gap measures how well-separated the target set M is from the remaining coordinates. Importantly, for a fixed m and input distribution, I'(m) depends solely on the model's inherent initialization bias. However, increasing the length of random input sequences amplifies this bias via concentration, making the separation more pronounced at the population level.
There are two natural empirical strategies to extract such population-level least-likely dimensions:
1. Sample-mean-basee extraction (ours). Estimate the expectation of each output coordinate by the empirical mean over random inputs X = { and select the m coordinates with the smallest value. Following the notation in Section 4, where g denotes the model (either the base model f or the suspect model f'):
- Σ g(x))
1
M
min
Σ 3
JC dont
=m
JEJ
2. Per-sample argmin voting. For each random input x; E e select the smallest-value coordi- nate and then take the m most frequent dimensions:
j (x:) = arg
min jEfI....do....
(2,),
Mo
arg
max ((r()))) E
JC Wc......ooom JJJm=m
15

---

Published as a conference paper at ICLR 2026
Let Pj Pr(j*(x) = j) denote the population probability that index attains the minimum, and let
Myote
arg
Σ Pj
(1......oo ) = JEJ
be the population top-ra set under voting.
Both approaches are intuitively reasonable. However, as we show below, the sample-mean based strategy is provably stable, whereas per-sample argmin voting is discontinuous and can flip under arbitrarily small perturbations, making it inherently noise-sensitive.
C.1 STABILITY ANALYSIS OF MEAN-BASED IDENTITY EXTRACTION VS. PER-SAMPLE ARGMIN VOTING
C.I.1 STABILITY OF SAMPLE-MEAN MEAD--DMEAD BASED EXTRACTION
We first show that the sample- cemple-mean--eee based operator, which selects the smallest coordinates in the empirical mean vector, is stable under small perturbations to model outputs. The perturbations represents realistic variations in outputs from the model, e.g., numerical noise, evaluation under different precision modes.
Formally, model perturbed outputs as g((x) = g(x) + nlz), where 7)x) R represents stochastic deviations. We assume that for each sample x; and coordinate j. the random variables ny (are) independent across samples (not necessarily across coordinates) and o-sub-Gaussian with Ein (x:)]= 0.!
Lemma C.1 (Stability of sample-mean-base extraction). Let and O' be the empirical means over X = {r: i.e, 9 E g(x;) and g' Ei= 9 (21) denote the empirical gap at rank m. Then,
Let (mm)
nI (m)²
Pr Mmean (9') # Mueea (9)) s 2dout exp
S
Interpretation.
The lemma provides quantitative upper bound the probability that the sample- mean-based extractor changes its selected set under perturbations. This instability probability decreases exponentially with the number of averaged samples due to variance reduction from o² per sample to o² //n in the empirical mean.
Proof. We condition on the sample X {=:) throughout the proof, that ÿ and ((m) are fixed, and all probabilities are taken over the randomness of {1((;)) only.
For any coordinate j. by definition of g'. Tj - 9j = Σ (z,)). Each 7 (z;) is mean-zero and a-sub-Gaussian, so their average is mean-zero and (a/vn)-sin (ir)-suu-Gaussian the variance proxy scales as 1/n under averaging. Hence, for any E >> 0, by the standard sub-Guusuee GabbSansni tail bound,
nE²
Pr (10)
9 X) exp
(1)
20
Now consider the event that any coordinate fluctuates by than e:
10, 91 8).
Taking a union bound over all d,ut. coordinates yields
dout
nE²
Pr max 10 3, x)
<
Pr Pr - 3,1 2 < 2dout exp(
(2)
20
1=1
'This sub- Gaussian assumption is mild: bounded random variables are sub a-suu--Guussaas Gaussian (Hoeffding's lemma), and common initialization schemes together with bounded activations yield light-tailed hehavior in both initialized and trained networks. The Gaussian case (xx) N/(0, o²) special of this assumption.
16

---

Published as a conference paper at ICLR 2026
We now show that
Muccung (g') # M (0) max (3) -Ø51 2 {
Equivalently, if max, 10 921 (m))/2, then the bottom-m identity set remains unchanged.
Let -()):---=((daa))) be a permutation of {I,..., dout} such that Fiay S Ij(a: S... S Dildoo)) Recall that the empirical gap at rank m is [(m) == Bstmi1 - Iimm > 0. Denote M (g) == ((m)))).,,,())) and Mimean (g) = 1im ((dml)) For any Mimenn(g) and any 8
M.mean (g), De- we have
91 2 9j 9i:m) = [mm). With 10 Til < f(m)/2 and 10% - Tel < fm
[(m]) (0+
Flym)) 2 f(m) - [(m) =
ge - x z (
2
Thus no coordinate in Ammenn(4 (g) can become smaller than any coordinate Mmman (g) under this deviation bound, and therefore
M
M (g') Mm (g).
Since ('m) is fixed under conditioning on X, plugging f(m) /2 into (2) yields
&rr(maxx max D7 -zi [(mm)21 S 2dout exp( nm(m)
Pr Mmean (o)) + Mmnan(g)) (0) s Pr
8
C.1.2 INSTABILITY OF PER-SAMPLE ARGMIN VOTING
In contrast, per-sample voting applies discontinuous winner-take-all operator on each input. Thus, arbitrarily small perturbations to outputs can change decisions with non-vanishing probability. We still consider the same stochastic perturbation model g (z) g(x) + n(z).
Lemma C.2 (Instability of per-sample argmin voting). For a fixed input let
(x: g) == arg min (z)
A,( == min {g)(x) 9) (x)} > 0
(tj((ag)
denote the argmin and its margin. Then, for any fixed x,
(r(j"(ng)) # (4; x) 2 o An(2)
V2o
for i...d. inputs T1, Znn
Pr 1 + ((((1))) 2 1- (1 (1-a)"
where a: :=::E,, Ex ao( (z) 22
]
Interpretation. Unlike mean extraction, voting does not benefit from variance reduction: each input incurs a direct hard decision based on noisy outputs. Increasing n only increases the number of opportunities for index-flips.
Proof. We first bound the per-input change probability Pr(j"(x: g') # (x; k). Fix and write
j := j'; g) for brevity. By definition of the margin, A,(x) min (g)(z) gy (z)). Let
j(2) be an index attaining this minimum, i.e., 9,00 (x) 9j (z) = (r())
Under the perturbation gx(x) = g(x) + ((x), we have
Le (x) - o1- (1:) [g (x) gi (x) (x) - n (x) = A,(x) + [nye (x) ny (z) Since 77 (z) and ni- (Ir) are independent A (0, 0²) variables, their difference is Gaussian with my (x) - ワ5 (xr NO,020²)
17

---

Published as a conference paper at ICLR 2026
The event that the argmin jhanges.,"""x;; # 1 (2: certainly occurs whenever g/ (x) 0/j Hence,
(1(")) .)) 2 =
Using the expression above. Pr(g)er()) (x) I oz- (x) (x) = Pr (x) (2) (x) s 00 =
S 50 2).
Let Z ~ (O, 1). Since my (z) 175- (z) d v 22 Z, we obtain
(r(A)(1) + byye (z) - 115 (1) <0 .)) = Pr
2 (r
A,(2))
= 26 A Ag(x)
V20
v 220
Therefore, Pr(ji [ g) + j (z; g) x))
(.)
<20
Now consider n i...d inputs 1 Zn. Conditional on the inputs, the events
E = {j (z g') j (z
E = (0 (2 = j (x::g)}
are independent across i, because the perturbations 17(x;) are independent across i. Moreover,
Pr(E, x4) = Pr(j" (x=i)))))))))))) + 5 (x1:g) 2 A0(1i)
V20
22
Define g(x;) == Pr(j"(ms,, # ( 9) x)) and a == Ex
(.(
Taking expectations
<20
over the randomness of z,, we obtain E Prr(EE æ;)]] = a. Therefore, by independence across i,
Pr(Ff F. = E II ((x:))
< E (1 कه ((-Ag(x))( < (1 a)",
where the last inequality uses i.i.d. inputs and Jensen's inequality applied to the convex function - log(1 u) on u E 1). Therefore,
Pr 3i < n: : (=ng)) # (((ng))= = - - E.) 2 1 111-111- (1-a)).
Finally, note that the empirical voting set M vote is determined by the empirical counts of the If the empirical gap between the m-th and (m-+1)-th most frequent dimensions is of order ((1/n) (i.e., only a constant number of votes), then change in j" (xx) for single input x; is sufficient to swap the ordering across this boundary and thus alter the top-m set. Combining this observation with the lower bound 1 - (I (1-oo (1)a)" shows that, in such regimes, the probability that Myote * MM (g) is bounded away from zero uniformly in i.e., per-sample argmin voting does not enjoy variance reduction with more samples.
dimensions g)); ))
C.2 INFLUENCE OF HYPERPARAMETERS
We now analyze how many random inputs are required for the empirical mean estimator recover the true bias. We reuse notation from Setup: each output coordinate (z) is random variable under random inputs
D, with population mean Øj == E[g, (#)). (gr1)... ((d..) represents the sorted means in non-decreasing order, and y ;= I(m+1) [(m) > 0 is the separation margin. We estimate using n i.i.d. random inputs:
0
Σ (x:),
g
G .... D
The target dimensions set is M with size and our empirical estimation gives M. Lemma C.3 (Sufficient condition for recovering the true set). If flğ ølloo S and 2s < 7, then the empirical bottom-m set equals the true set: M M..
18

---

Published as a conference paper at ICLR 2026
Proof. For any E M and j' $ M, we have Oj (m) and Ø,' 2 Im+ The &oo bound gives
9j (m) E, 95' 2 1(m+1)
Thus,
9 "y S (7 22) 0,
so no outside index can enter nor inside index leave the bottom-m set.
We interpret this recovery condition in terms of hyperparameters. We again adopt the same sub-Gaussian assumption as Section C.I. This assumption is mild: by Hoeffding's 's inequality, any bounded random variable is o-sub-Gaussian with proportional its range. In our setting, each gj (z) corresponds to logit or final hidden activation, which are typically bounded or light-tailed in both initialized networks (due to controlled parameter initialization) and trained networks.
Corollary C.4 (Sample complexity under sub-Gaussian coordinates). Suppose each gj g)(x) o-sub- Gaussian. Then with probability at least 1 8,
2 (2dg2///// (8)
1 Øllo ov
Consequently,
2 722
n 2
log 2
then M = -M with probability at least 1
Proof. For fixed uub-Gaussian concentration implies
Pr( 9,1 =) S eexp(-ne" /202)
union bound over all dout coordinates yields
Pr( - $ 2dout exp (-
nE²
Pr 1 øl >
20
Set the right-hand side to and solve for e to obtain, with probability at least 1 - 8,
2 12gg2/oo/// /
| Ølloo S Vo
Now choose
2 (oog(2do)0))
: o
7
n > So² 7- log (2dm)) (i.e., 2s < 7), then Lemma C.3 applies and implies M -M with probability
least 1 d.
meaning the number random queries grows logarithmically with the output dimension, but scales inversely with the squared separation margin T The margin y itself has two contributing factors:
Therefore, accurate recovery requires n = S2 log
log deu 72
Intrinsic bias strength. Longer random input sequences amplify initializationninuueee bias via concentration, increasing the population separation between coordinates.
Choice of m. Different choices of m correspond to different parts of the ranked spectrum D(dm ) thus yielding different effective margins in practice.
The former can be controlled by the input length 1, while the latter is selected empirically based on the observed gaps in the mean spectrum.
19

---

Published as a conference paper at ICLR 2026
D MORE EXPERIMENT DETAILS
D.1 IMPLEMENTATION DETAILS AND LICENSES
We train all models with the Hugging Face Transformers Trainer (Wolf et al., 2020), using Ac- celerate (Gugger et al.. 2022) for distributed runs. All open-source models are loaded from their official Hugging Face releases and used under their original licenses: Llama models under the Meta Llama Community License, and other models under Apache-2.0. All datasets are downloaded via the Hugging Face Datasets library (the library is Apache-2.0); dataset content follows each dataset's stated license.
D.2 COMPLEMENTARY RESULTS FOR FINGERPRINTS INITIALIZATION AND AFTER TRAINING
Table 8: Full results of methods evaluating fingerprints at initialization and after subsequent training.
Logits Output
Hidden State
Baselines
Model Pair
t-test
u-test
t-test
u-test
Intrinsic
REEF
PCS
ICS
siust
42
vs.
base
3.33e-3V
1.02e-3<
-.
6.28e-8
-0.021*
0.375
0.580 0.196*
$ Anst
vs.
123
2.06e-3
7.33e-3
7.09c-6'
1.37e-5
0.149
0.369
0.581 x 0.188
1000
vs. $1000
2.44e-3
4.14e-3
5.
2.81e-3
-0.252x
0.381
0.581
0.188%
2000
VS. $2000
base
5.63e-3V
6.76e-3<
4.00e-10
1.27e-9
-0.337*
0.331
0.581
0.188%
D.3 MORE QWEN RESULTS
D.3.1 CROSS-SIZE FINGERPRINT WITHINN THE QWEN-2.5 SERIES
To further examine whether SeedPrints "collapse" within a model family when we vary the parameter size, we report cross-size similarity between Qwen-2.5-1493 (target) and smaller models from the same Qwen-2.5 family. Table 9 shows the results for SeedPrint (+) and two baselines, REEF (L) and HUREF (4).
To probe whether initialization- fingerprints "collapse" a family-level signature when scaling model size, conduct cross-size study within the Qwen-2.5 series. We treat Qwen-2.5-14P3 the protected target model and consider Qwen-2.5-(0.5B, 1.5B, 3B, 7B) as suspect models that share the same architectural family but are not trained as descendants of Qwen-2.5-14B, As discussed in the main paper, SeedPrints outputs a p-value for the hypothesis test "the suspect shares initialization-bon baselines that directly
fingerprints with the target", while REEF and HUREF are similarity- output continuous scores and require the user to choose a threshold.
Several observations are noteworthy. First, all SeedPrints p-values comfortably above a typical significance level (c.g., a00001.. Under our ppothesis-testingg view, this means that none of the smaller Qwen-2.5 models is flagged as sharing the same lineage as Qwen-2.5-14B. This behavior is desirable in our setting: even though these models belong to the same family and reuse a similar architecture design, the fingerprinting test does not collapse to a single family-level signature that would spuriously conflate distinct pre- mr--trainin runs.
The cross-sizz size p-values occupy different regions of the [0, 1] range rather than concentrating around a continuous "similarity -their role is to
single value. We do not interpret these magnitudes support a calibrated yes/no decision at a fixed significance level. Nonetheless, the spread indicates that our test statistic remains sensitive to the concrete combination of architecture and initialization seed, instead of degenerating into a generic pattern shared by all Qwen-2.5 variants.
Finally, the similarity- baselines behave differently. REEF produces scores in relatively narrow band (0. 00.8820--088 0,8833) across all sizes, while HUREF outputs values that are numerically very close to zero (on the order of 10 50-5). Since these methods do not provide a built-in decision rule, operators must manually choose thresholds, and it is unclear how to robustly separate cross-size variants from genuinely lineage-related models using single cutoff. In contrast, SeedPrints directly yields
20

---

Published as a conference paper at ICLR 2026
Table 9: Cross-size similarity between Qwen-2.5-14B (target) and other Qwen-2.5 models.
Qwen-2.5-0.5B
Qwen-2.5-1155
Qwen-2.5-3B
Qwen-2.5-7B
SeedPrint (1)
0.7184
0.9707
0.3898
0.5429
REEF (L)
0.8250
0.8399
0.8833
0.8461
HUREF (4)
x.377x00
-2.84x10-3
-8700x10
-6.08
statistical decision at a prescribed significance level, providing a clearer and more operational answer the question "do these models plausibly share training lineage?" within a model family.
D.3.2 DIFFERENT INITIALIZATION SEEDS PRODUCE DISTINCT FINGERPRINTS
Table 10 presents the p-values from correlation tests Qwen-style models initialized with different random seeds (42, 123, 1000, 2000), evaluated under both the t-test and U-test. In all cases, the p-values remain above 0.01, confirming that our approach can consistently tell apart models trained from different seeds.
Table 10: Comparison of fingerprint behaviors between models initialized with different seeds for Qwen-style models
Logits Output
Hidden State
Seed Pair
t-test
U-test
t-test
U-test
$123 vs. $1000
0.741
0.727
0.094
0.074
$1000 $123
0.954
0.971
0.125
0.094
942 vs. $2000
0.273
0.360
0,451
0.529
$2000 $42
0.215
0.206
0.130
0.095
D.3.3 TRAINING PRESERVES THE INITIALIZATION FINGERPRINT
We also compare each initialization model s iinit with its corresponding trained version ghase on the OpenWehTexx dataset (Gokaslan al., 2019) 1=: tokens) for Qwen, as shown Table 11. Consistent with the L.LaM-style results, all seed-model pairs yield p-values below 0.01. This indicates that training does not erase the initialization fingerprint; instead, the signature is preserved in the descendant model.
Table 11: Trained models share the same fingerprint behaviors as their initialization models (p-value < 0.01).
Logits Output
Hidden State
Model Pair
l-test
U-test
1-test
U-test
trit 8
vs. 8
base
123
2.38e-03
1.68e-03
7.36e-15
3.38e-13
Vs. $1000
1.35e-04
4.84e-05
4.41e-13
2.01e-11
42
$22
base
1.39e-03
1.46e-03
1.06e-24
2.05e-19
2000
VS. 22000
base
1.80c-03
1.28c-03
4.87e-24
1.92c-20
D.3.4 IDENTICAL DATA AND ORDER DO NOT MAKE FINGERPRINTS CONVERGE
Following the setting in Table 4, also test whether fingerprint behavior would be erased or confounded by identical data and order for Qwen-style models. Table 12 shows that fingerprints remain seed-specific under identical data and curriculum.
21

---

Published as a conference paper at ICLR 2026
Table 12: The same dataset and training order do not shape fingerprint behaviors be identical across different initializations.
Logits Output
Hidden State
Model Pair
I-test
U-test
t-test
U-test
irit 8
vs. 8 bocse
0.598
0.638
0.286
0.254
1000
$123
0.804
0.805
0.036
0.040
$2
Jnst
22000
basse
0.589
0,608
0.026
0.043
2000 Vs.
anst
42
base
0.523
0.482
0.112
0.123
D.3.5 CONTINUAL TRAINING ON DIVERSE DATASETS DOES NOT CONFOUND THE FINGERPRINT
Table 13 reports the same test results as Table for Qwen. Our method remains stable and reliable across all cases (even under severe training distribution shifts), whereas many baselines become confused and produce errors.
Table 13:
Fingerprint persistence under continual training on diverse datasets (base model: seed 1000, corpus openwebtext).. U-test refers to the Mann-Wiin Kunn-WWitney U test.
Setting
Ours (logits)
Ours (hidden)
Baselines
Continual corpus (seed)
t-test
U-test
t-test
U-test
Intrinsic
REEF
PCS
ICS
TinyStories (1000)
5.36e
09
1.92e 07%
8.49e 214
5.09e 71
1.000
0.957' 0.999 0.996
TinyStories (123)
0.434
0.433
0.256
0.065
0.913
0.199
0.328
0.039
the_stack (1000)
0r
4.16c
237'
1.16e
211
2.30e
76
0.999
0.313
0.995 0.976
the_stack (123)
0.999
0.993
0.610
0.491
0.916
0.255
0.328 0.038
D.3.6 ALL-STAGE VERIFIABLE FINGERPRINTS
Our fingerprinting method on Qwen also demonstrates verifiability at all training stages (Figure 6).
Across all variants, the suspect model consistently recognized as belonging to the same lineage, with p-values remaining below the 0.01 threshold.
10
10 89
10 105
P-value
141
Hidden State T-test
Logits Mann-Whitney a=0.01
Hidden State Mann-Whitney U
10
177
Logits T-test
10 221
249
10
4K
8K
12K
10k
20K
24K
28K
32K
Checkpoint Index
Figure 6: Fingerprint verifies lineage at every checkpoint (p-values < 0.01) for Qwen structure model.
22

---

Published as a conference paper at ICLR 2026
E MODEL CATALOG AND DECODING SETTINGS
Our evaluation covers 58 models in total. Below we document the decoding configuration used throughout all runs and enumerate the model catalog grouped by family. For each derived model we also indicate the post-training transformation (finetuee, adapter, merge, quantization, distillation.. "PT" and "IT" refer to pre-trained and instruction-tunee source models, respectively.
Parameter
Value
max_new max_new_tokens
512
temperature
0.7
top_p
0.9
top_
50
do do_sample
true
max_input_langte
512
E.1 MODEL LIST BY FAMILY
E.1.1 QWRN-2.5-7B
Variant
HuggingFace repo
Type
PT
Twen/Qwen2 5-72
IT
Derived
Bhen//uenn.5-Methh-ae
finetune
Derived
finetune
Derived
finetune
Derived
finetune
Derived
merge
Derived
Dumnyyoree/evvaccc11----ttc
merge
Derived
Eaggaaccan/saaeiD0/eeeeeee222122 Instruut
adupter
adapter
Derived
Qwen/Qwen2.5 7B Instruct GPTQ Int4
quantization
Derived
Qwen/Qwen2.5 7B Instruct GPTQ Int
-
quantization
Derived
distillation
E.1.2
QWEN2.5-14B
Variant
HuggingFace repo
Type
PT
S-14B
IT
Derived
#wenn/Owen2.S-ooee-1111
finetune
Derived
oxyap3/oxy-1-aaa=.
finetune
Derived
5000000/Cwen2.5 14B Gutenberg Instruct Slerpeno
merge
Derived
Toastyyiggenn/geennne story test glora
adapter
Derived
quantization
Derived
deepseek-al seepaaattsll/aagssee------------ 1t1----en--111111140--111111 -Qwen-14R
distillation
23

---

Published as a conference paper at ICLR 2026
E.1.3 LLAMA-3.1-8B
Variant HuggingFace repo
Type
PT metta-1laama/22aaa-33------
IT mate:1aamm//222222----------1111
Derived vallanaccaab22222m33.33114411eee
finetune
Derived Rediaacii//laama331-------.22
finctune
Derived TF025cccooos2222222me3--------.-
finetune
Derived
finetune
Derived Beccraasweeeiiaaaaa1aaaaaaaaaaaa
merge
Derived [kagg_asss2201112233333222211113
merge
Derived uaa.pppess...nnnno.....aatcceeee
adapter
Derived oocar//jieeaatteeenna.aaseeererr
adapter
Derived oagesannssstteeiiaaaaamaaeaeeees
quantization
quantization
Derived aatp:///eeees.1eeea..a....eeeea.
distillation
E.1.4 MISTRAL-7B--O.33
Variant
HuggingFace repo
Type
PT
mistrallai/Mistrri--11--
Derived
KuzmaAI/AQUA-73
finetune
Derived
finetune
Derived
merge
Derived
charmmeemerrrrrrioosm/msttttaa Brain Mocie Macian_ncciA__ccrrriiiee
adapter
Derived
Redhatat//Mistral 7B instruct vo.3 GPTQ 4bit
quantization distillation
Derived
aggawwo/mlstral7bbb llll wew.stttldd-dd-ddded---- deepseeeeeeee-e-. qwen322
E.1.5 GEMMA-2-2B
Variant
HuggingFace repo
Type
PT
googl hee//emmaa2222
IT
googlee/enma-22220-11
Derived
cioma/gemma--2-buuu-r
finetune
Derived
finetune
Derived vonjjckk/gemm2222--eerrg
merge
Derived pooglee-loudd-pttteeesaaa. gemma-2-222 1--bbitt 2awiib-lloeeeee cer--iitoorrssee lora-sql
adapter
Derived
quamtization
Derived
Syed Hasan 8503/Genma 2 2b it distilled
distillation
E. LLAMA-2-7B
Variant HuggingFace repo
Type
meta- meta--llamm/tlama222
meta-
Derived allenai/calu-2-12
finetune
Derived QLATINCC/EVE-LLlams
finetune merge
Derived bercussaar/ccoa-7---01.1
Derived Amma--L/lllama-Betteeerr-
adapter
Derived Tro.f:.oeeeiiaamee---11-------00
quantization
Derived http::/hhaaccttttttttteaaallllld
distillation
Notes. "Type" denotes the post-training transformation relative to the PT/IT source model: finetune includes supervised/prefeeeeee optimization variants; adapter includes LoRA/QLORA-styl modules; merge includes model soups and TIES-style merges; quantization includes GPTQ/INTx variants; distillation includes student models distilled from reasoning-augmeented teachers.
24

---

Published as a conference paper at ICLR 2026
E.2 SAMPLE RESULTS
We present partial results from one LeafBench evaluation run in Table 14, Reported metrics are 1 - used similarity scores. We adopt a significance level of 0.01 for p-values; empirically, LeafBench identifies an optimal decision threshold of 0. 0.9920, further supporting the reliability of our method.
Crucially, while individual p-values should not be interpreted as linear similarity measures, their distribution across model pairs is informative: same-lineage models yield much lower p-values, while unrelated models remain near uniform.
25

---

Published as a conference paper at ICLR 2026
Model
Qwen-2.5-7B
Swen-2.5-7R-Innsttee
Qwen2.5-14B
Qwen2.5-144h Insrrrtttttttt
Qwen-2.5.7B
1.000
1,000
0.079
0.000
Qwen-2.5-B-IInttraet
1.000
1,000
0.813
0.536
Qwen2.5-7B-44 Mathhath
1,000
0.994
0.047
0.987
Qwen2.5.77 Cod-Corrr
0.996
0.999
0.177
0.002
Qwen2.5-78-Instree 6-77--nnstret-ccciiin
1.000
1,000
0,576
0.369
1,000
1,000
0.276
0.886
Qwen2.S.7R.Shee
1.000
1,000
0.939
0.225
QevaCoT-7B
1,000
1,000
0.199
0.119
Qwen2.5-7P8 3--78---ntttuut-Tss---
1,000
1.000
0.672
0.090
Qwen2.5 100---7-t-ttt
1,000
1,000
0.155
0.987
(wen2.5-78-Instruell
1.000
1
0.827
0.961
Qwen2.5 7B. Instruct
1,000
1.000
0.914
Qwen2.5.7B.000 177-Open-RR RI-Distill
1,000
1,000
0.302
0.955
Qwen2.5-144
0.324
0.487
1,000
1,000
Qwen2.5 14B Instruct
0.403
0.702
1.000)
1.000
Qwen2.5 Coder-14
0.492
0.660
1.000
1,000
oxy-I-smal
0.492
1,000
1
Qwen2.5 Gutenberg Instruct- Sterpeno
0.931
1,000
Qwen stor--ttrr..ue lest-etteee glara
0.112
0.898
1.000
1.000
Qwen2.5. 14B 11t..---1----sstc----------- Instrnnc---PTQ-Ini
0.583
0.092
1. 1,000
1.000
DeepSeck R1 R1-Disuill-e Distill-ween Qwen-1418
0.204
0.000
1,000
1.000
LJama 3.1-8B
0.638
0.714
0.156
0.000
1.ma--3.-8R.
0.118
- -BFFreepaaa
0.532
0.065
LJama 3.B TTDD.
0.925
0.811
0.331
0.014
Llama 3.1.a-3.-8B..CC. SBCCalall
0.001
0.605
0.003
0.946
Lamma-3.1-83 -ww--a-stuut-Abliiiiiee
0.061
0.005
0.121
0.422
Llama 1.1ma-3.1-85 3.1-8B Instruct-
Saatmuc-lallAAAbieerrree TIES
0.356
0.509
0.796
0.000
1.ma-3.--8B-ExraMM
0.640
0.160
0.721
0.875
Destruct-vv-jb--deseript
0.242
0.004
3.1-a-3.1-B 111-88--Isstut--PPPe Payaruct
0.682
0.057
0.063
0.191
0.798
0.214
0.154
0.385
Llema-3.18B--nsttuee -bb--
0.513
0.023
0.005
0.933
3.1---.Insttttt Openn Ristill
0.800
0.101
0.772
0.024
Mistral-7B-v0.3
0.224
0.352
0.506
0.044
Mistral-7B-00334mmttuu
0.000
0.984
0.104
AQUA-7B
0,014
0.613
0.703
0.569
Mistal-7B--0033-- ---pplllkk
0.005
0.481
0.872
0.105
0.904
0.774
0.058
0.522
(istral-7B-v0.3--10
0.340
0.983
0.481
0.016
0.002
0.281
0.089
0.784
0.009
0.001
0.511
0.015
Gemma-2-22
0.747
0.869
0.689
0.000
Gem-n-2-2b-it
0.944
0.039
0.626
0.060
Gemma-2-bukn-2b
0.616
0.231
0.176
0.464
Gemma- 2-2g-nnogennssiiiin
0.002
0.536
0.079
0.980
Gemma- 2-2b-merge
0.012
0.651
0.075
Semma--2-2--i-looo--sgl
0.236
0.408
0.000
0.000
Gemma 2-w.--2----i----PPP-PP
0.945
0.159
0.000
0.008
Gemma-2-2h-- 2-i-in-2-bbittdisiilleddd
0.482
0.573
0.271
0.276
Llama-2-7b
0.443
0.470
0.148
0.276
Ilama 1.7ma-2-7h.chat
0.528
0.868
0.424
0.766
tulu-2-7h
0.272
0.1
0.858
EYE-Llama_gg
0.657
0.244
0.674
0.968
coma. v0.1
0.953
0.004
0.573
0.731
1lama2-Rette-
0.136
0.699
0.469
0.860
Llama-2-TB-Chat-GR-
0.594
0.603
0.582
0.424
lamaa 2. lo watermark disiill kgw gamma0..2 delta2
0.349
0.224
0.366
0.976
Table 14: Sample results. Pairwise similarity between target models (rows) and base models (columns). Cells with 1 p 0.99 are highlighted in green as same-lineage pairs.
26


Reviews:
[{"review_id": "H78m52lkJm", "reviewer": "Reviewer_2xKH", "summary": "SeedPrints fingerprints LLMs using their random initialization rather than training artifacts. Untrained networks already show reproducible token selection biases set by the seed, and these signatures persist through training. A simple statistical test recovers model lineage with high confidence and stays robust across training stages, domain shifts, and moderate parameter edits, with strong results on LLaMA and Qwen families and public benchmarks.", "questions": "Please see the weakness part. I will consider raising the score if the authors address the weaknesses/problems clearly", "limitations": "", "rating": 4, "confidence": 4, "soundness": 3, "presentation": 3, "contribution": 3, "strengths": "1. Finding of the biometric fingerprint is interesting, where initialization creates stable token selection biases that are seed-specific and persist throughout training, giving each model a seed-level identity.\n2. The authors find an effective algorithm by constructing identity indices from the most down-weighted output dimensions on random inputs, then run a rank-based correlation test with calibrated p-values to decide shared lineage.\n3. The experiment setting is comprehensive, which verifies lineage at every checkpoint and stays accurate under domain shift, finetuning, PEFT, quantization, merging, and distillation, outperforming similarity-based baselines.", "weaknesses": "1. Within the same model family, how similar are fingerprints across sizes. For example, what is the similarity between Qwen2.5 7B and 14B, and does similarity grow with parameter sharing or architectural reuse, and will they be distinguishable?\n\n2. For models like deepseek distill llama or deepseek distill qwen, which fingerprint do they most resemble? The teacher, the student base, or a new hybrid, and does SeedPrints remain reliable in these distillation settings\n\n3. The SeedPrints relies on logits or hidden states, which many closed-source APIs do not expose. Are there practical black box surrogates to test for seed-level fingerprints, for example, using only top-k outputs or sampled continuations with controlled prompts\n\n4. Rank-based tests may be sensitive to large vocabularies and temperature scaling. Could authors add more analysis of sensitivity to temperature, alternative normalizations before ranking, and report how these choices affect power and false positive rates\n\n5. In the experiment, the comparisons emphasize p-values. Maybe add more effect sizes, confidence intervals for Kendall tau, could improve the interpretability and calibration", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "EYVmdZAAKe", "reviewer": "Reviewer_4uC2", "summary": "The paper identifies an interesting phenomenon in model training: randomly initialized language models exhibit some bias in their token predictions and this bias persists even after training. This signature can be leveraged to form the basis of a fingerprinting test, which can test whether a given model has statistically significant correlation with the bias detected in a given randomly initialized model. The key innovation is the introduction of _identity indices_, continuous embeddings which have more persistent signatures throughout training.", "questions": "1. The paper describes an identity index with a low mean $\\bar{g}_j$ to be \"unwilling\", but in the case of the final embedding, where the dimensions are signed and there is no distinction between \"true\" and \"non-true\" classes, it's not clear why choosing with the argmin makes sense here.\n2. How many parameters did the LLaMA/Qwen-style models have? \n3. There is an opportunity here to show that this technique persists across scales by leveraging the checkpoints published during the training of the OLMo/OLMo 2 family of models. This is a very interesting experiment, since it shows that the signature can persists during pretraining at the scale of trillions of training tokens without having to train such models from scratch for this work.\n4. In Table 2, the negative p-values are on the order of 1e-10 or larger while in Figure 2 they are on the order of 1e-50 or smaller. Why are the p-values so different in these two settings?\n5. The results in Figure 1 suggest that token level biases also persist through training, although the rank correlation is quite small. Do you think this could be converted into an effective test by increasing the number of samples until the p-value becomes small enough? If it can be made to work, the token-level setting is desirable because it can be implemented without access to the continuous embeddings of the model.", "limitations": "", "rating": 4, "confidence": 4, "soundness": 2, "presentation": 3, "contribution": 3, "strengths": "1. The paper makes use of a scientifically and technically interesting signature that is present in LLMs. To my knowledge, this is the first time this particular signature has been documented.\n2. The introduction of the _identity indices_ is a clever innovation which allows the test to focus on a statistical signature which is more stable (across training).", "weaknesses": "1. The test itself is a bit complicated. This, in itself, is not a significant weakness, but it does makes it difficult to determine if the test is correct or not. The paper itself does not justify why the null distribution used is correct. In particular, due to the selection used to build the intersection $T$, the random variables in $\\mathcal{T}$ are not independent. Thus, it seems odd to me that the null distribution treats each simulated index as being independent. I'm not saying the test is incorrect, but I do think some justification here is necessary, given the complexity of the method.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "fU6Eu39TTf", "reviewer": "Reviewer_Jec7", "summary": "This paper proposes SeedPrints, a stronger and more intrinsic notion of LLM fingerprinting. The SeedPrints methodology is built on a key empirical observation: randomly initialized LLMs exhibit subtle but reproducible token selection biases that are unique to their initialization seed. These biases persist throughout training, creating a detectable signature. Experiments show that SeedPrints remains effective across all training stages and robust under domain shifts or parameter modifications.", "questions": "- In Figure 1 (top-right subplot), why is the baseline at -0.0003?\n- In lines 178-179, why is the mean output vector calculated? Will the mean output vector differ for different values of n, thereby affecting identity indices?\n- In practical applications, how is the size of m determined? Do different m values affect the stability and discriminative power of the fingerprint? Should the value of m vary for different models based on their size and vocabulary?\n- In Table 5, how does the t-test result change? Based on the U-test results, it seems that the correlation (if any) weakens as the number of parameters increases.\n- In Algorithm 1, is it possible for no intersection of identity indices to exist?", "limitations": "", "rating": 6, "confidence": 3, "soundness": 3, "presentation": 2, "contribution": 3, "strengths": "- The idea of seed-based intrinsic fingerprinting is interesting.\n- The proposed method is validated through extensive experiments.\n- It demonstrates a certain degree of practical application value.", "weaknesses": "- Some descriptions in the paper are unclear. For example, the terms \"min/max\" and \"top/bottom\" appearing in Figure 1 are not explained.\n- There is a lack of explanation as to why models with different seeds possess different \"identity indices,\" which is somewhat counter-intuitive. Is it theoretically possible for two models, initialized with different seeds, to have statistically indistinguishable SeedPrints due to some coincidence?\n- The paper does not discuss potential evasion strategies. For instance, if a malicious actor, understanding how SeedPrints work, might attempt to develop methods to \"erase\" the model's original fingerprint or \"forge\" a fingerprint to make it appear to originate from a different seed.\n- Although the paper mentions its inapplicability to black-box models, it must be acknowledged that this limits its application to some extent.\n- There is a typo: in line 136, \"10001st\" should be \"1025th\".", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "6vEH1Y3MYG", "reviewer": "Reviewer_cFVL", "summary": "This paper investigates whether large language models (LLMs) inherit statistically detectable traits from their random initialization seeds, a phenomenon termed “Galtonian fingerprints.” The authors propose SeedPrints, a training-free statistical framework that traces model lineage at the seed level by measuring correlations across output distributions. Systematic experiments on 58 models spanning six major LLM families (LLaMA, Qwen, Mistral, Gemma, Phi, and OLMo) show that token preference biases established at initialization remain stable, even after diverse training and deployment interventions such as Instruct tuning, finetuning, PEFT, quantization, model merging, and distillation. These findings indicate that a model’s identity is partially determined at birth (initialization) rather than solely during training, providing a practical tool for model provenance analysis.", "questions": "1. Could the authors provide a theoretical or intuitive explanation for why seed-induced bias persists after long training? Is it due to early optimization path dependency or weight-space geometry?  \n2. How many samples are required to detect a fingerprint reliably (e.g., 95% confidence)? Please quantify this relation to support practical use.    \n3. Please conduct experiments on Llemma-7b[c1] in Table 5, because the robustness of the proposed method against large-scale data fine-tuning needs to be verified.\n4. The motivation for using the Kolmogorov-Smirnov (KS) statistic needs to be clarified. In such a setting, REEF underperforms ICS, which is contrary to common sense.\n\n\n\n[c1] Zhangir Azerbayev, Hailey Schoelkopf, Keiran Paster, Marco Dos Santos, Stephen McAleer, Albert Q Jiang, Jia Deng, Stella Biderman, and Sean Welleck. Llemma: An open language model for mathematics. arXiv preprint arXiv:2310.10631, 2023.", "limitations": "", "rating": 4, "confidence": 5, "soundness": 3, "presentation": 3, "contribution": 2, "strengths": "The paper proposes a statistical method for detecting and characterizing persistent fingerprints left by random initialization seeds in LLMs.\nSpecifically,  the authors introduce the novel idea that random seeds can leave stable, measurable fingerprints that persist through training. The findings reveal that seed-level fingerprints remain robust despite continued training across diverse datasets and objectives.", "weaknesses": "1. The paper empirically demonstrates the effect but does not fully explain why seed-level bias persists after optimization.   \n2. Reliance on Kendall-Tau significance may be unstable for small sample sets; sensitivity analysis (sample size vs. detection power) should be more explicit.\n3. The motivation for using the Kolmogorov-Smirnov (KS) statistic is unclear.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}]

Author rebuttal:

Decision: Accept (Poster)