LLM DNA: Tracing Model Evolution via Functional Representations
Authors: Zhaomin Wu
Haodong Zhao
Ziyang Wang
Jizhou Guo
Qian Wang
Bingsheng He
Venue: ICLR 2026 Oral (2026)
Keywords: Large Language Model
Representations
Fingerprint
Embedding
Evolution
Phylogenetic Tree
DNA
Dimension Reduction
Primary area: unsupervised, self-supervised, semi-supervised, and supervised representation learning
TL;DR: We introduce LLM DNA, a low-dimensional representation of LLMs, uncovers undocumented relations and constructs phylogenetic tree for LLM.

Abstract: The explosive growth of large language models (LLMs) has created a vast but opaque landscape: millions of models exist, yet their evolutionary relationships through fine-tuning, distillation, or adaptation are often undocumented or unclear, complicating LLM management. Existing methods are limited by task specificity, fixed model sets, or strict assumptions about tokenizers or architectures. Inspired by biological DNA, we address these limitations by mathematically defining *LLM DNA* as a low-dimensional, bi-Lipschitz representation of functional behavior. We prove that LLM DNA satisfies *inheritance* and *genetic determinism* and establish its existence. Building on this theory, we derive a general, scalable, training-free pipeline for DNA extraction. In experiments across 305 LLMs, DNA aligns with prior studies on limited subsets and achieves superior or competitive performance on various tasks. Beyond these tasks, DNA comparisons uncover previously undocumented relationships among LLMs. We further construct the evolutionary tree of LLMs using phylogenetic algorithms, which align with shifts from encoder-decoder to decoder-only architectures, reflect temporal progression, and reveal distinct evolutionary speeds across LLM families.

Published as a conference paper at ICLR 2026
LLM DNA: TRACING MODEL EVOLUTION VIA FUNCTIONAL REPRESENTATIONS
Zhaomin Wu', Haodong Zhao?, Ziyang Wang', Jizhou Guo Qian Wang', Bingsheng He Department of Computer Science. National University of Singapore
2 School of Computer Science, Shanghai Jiao Tong University
ZZhiyuan College, Shanghai Jíao Tong University
zhaomin@nus edu sg, zhaohaodonge edu en, wangziyang&u, nus edu, $jtu1864098516
edu. on, e04 40493632uu. nus edu, dcshebênus. edu. sg
ABSTRACT
The explosive growth of large language models (LLMs) has created a vast but opaque landscape: millions of models exist, yet their evolutionary relationships through fine-tuning, distillation, or adaptation are often undocumented or unclear, complicating LLM management. Existing methods limited by task specificity, fixed model sets, or strict assumptions about tokenizers or architectures. Inspired by biological DNA, we address these limitations by mathematically defining L.M DNA as a fow-dimensional, bi- bi-Lipschitz representation of functional behavior. We prove that LLM DNA satisfies inheritance and genetic determinism properties and establish the existence of DNA. Building on this theory, we derive general, scal- able, training-free pipeline for DNA extraction. In experiments across 305 L.Mss DNA aligns with prior studies on limited subsets and achieves superior or com- petitive performance on various tasks. Beyond these tasks, DNA comparisons uncover previously undocumented relationships among LLMs. We further con- struct the evolutionary tree of LLMs using phylogenetic algorithms, which align with shifts from encoder-decoder to decoder-only architectures, reflect temporal progression, and reveal distinct evolutionary speeds across LLM families.
INTRODUCTION
The proliferation of large language models (LLMs) (Zhao et al. 2023 has produced vast and complex landscape of models. Hugging Face (Hugging Face, 2025 alone hosts millions of models spanning diverse families, architectures, and tokenizers, with the number increasing at an accelerating pace (Rahman et al.) (025). Amid this rapid growth, tracing the evolutionary paths of LLMs through fine-tuning, distillation, adaptation is critical across several areas: safety auditing-tracee
how security risks such as backdoors are transferred between LLMs (Cheng et al. 2024]; model governance verifying that adaptations and fine-tuning follow license require- ments (Duan et al. 2025 Lee et al. 20255 Xu et al. 2025); and multi-agent systems- guiding the
structure design and planning (Han al. (0024). This motivates a central research question:
Can we develop a frumevvrk to define and analyze the "DNA" of LLMs, enabling us to systematically characterize their functional similarities and evolutionary relationships?
frumework
Existing approaches to defining and extracting LLM "DNA" remain premature. Most prior work compresses LLMs into task-specific representations for model routing (Ong et al. (025) or ensemble learning (Huang et al. 2024). These representations are trained for specific objectives and neither generalize across tasks nor capture overall model characteristics. [Zhuang et al.) (2025) propose EmbedLLM, the first task-agnostic approach, which learns a compact representation for multiple downstream tasks. However, this representation is defined relative to fixed set of LLMs-it changes when new models are added-and thus is not an intrinsic DNA each model. Nikolic al.) (2025) and Zhu et al. (2025) further explore how to measure provenance independence between two LLMs by calculating token similarity or parameter correlation, but these methods do not gen-
dna.xtra.science
pypi.org/noojeetllm-mmna

---

Published as a conference paper at ICLR 2026
eralize to diverse LLMs with varying architectures and tokenizers. Collectively, these approaches do not furnish a formal and general definition of LLM DNA or a practical method for its extraction.
To address this gap, we introduce the concept of LLM DNA: compact, low-dimensional represen- tation of model's functional behavior. By analogy to biological DNA, it should satisfy two founda- tional properties: inheritance- -maul perturbations the model do not alter the DNA; and genetic determination- models with similar DNA exhibit similar characteristics. If such a DNA is well defined and extractable, established phylogenetic tools can be leveraged to analyze LLM evolution.
The challenges of defining and extracting of LLM DNA are two-fold. The theoretical challenge includes mathematically formalizing DNA that satísfies both properties- inhiritanie and genetic determination and proving the existence of DNA under such definition. The practical challenge to design, given this definition, a general and scalable framework that efficiently extracts DÑA across heterogeneous LLMs with distinct architectures, tokenizers, and prediction paradigms.
In this paper, we formally define LLM DNA as low-dimensional vector that obtained via a bi bi-Lipschitz map from the LLM functional space. We prove that this definition naturally satisfies inheritance and genetic determination and, using the Joinson-Lindenstrauss (JL. lemma (Johnson et al. 1984), formally establish the existence of such DNA. To practically extract LLM DNA, we derive random linear projection as an effective extraction method from these theoretical foundations and design RepTrace-a general, scalable, training-fiee free pipeline for DNA extraction. Finally, we build the evolutionary tree for 305 LLMs using Reparace and report various new findings.
Contributions. (I) We formally define LLM DNA and prove that it satisfies inheritance and genetic determination properties, proving the existence of LLM DNA. (2) Building on these results, we propose general, scalable, training-fre frree pipeline for extracting LLM DNA. (3) Experiments 305 models show that LLM DNA outperforms existing baselines on specific tasks and aligns with prior findings. (4) Using LLM DNAs, construct a phylogenetic tree that aligns with known architectural and temporal shifts and reveals undocumented relationships among LLMs.
2 RELATED WORK
LLM DNA relates prior work using terms such as representation and fingerprint. We group this literature into two views: (i) macroscopic view, which manages large groups of models by mapping each model to representation for model routing and ensemble learning; and (ii) microseopiic view, which uses fingerprints to assess whether two LLMss are related for provenance or identity.
Representation of LLMs. This direction manages large LLM collections by encoding each LLM
into a representation. Most methods are task-specific: Ong et al. (2025): Wang et al. (2024) learn representations for model routing, while |Huang et al.] (2024): Fang et al. (2024) encode models for ensemble learning. For instance, HybridLLM (Ding et al. 2024), trains binary "easy-hard" classi- fier to route queries to a small or larger model. Route (Ong et al. 2025) learns a supervised router that selects between stronger and weaker models based on query features. MetaLLM (Nguyen) al. (024) formulates routing as a multi aalli- bandit, adaptively trading off expected answer qual- ity against invocation cost via online feedback. I.LMMensemble (Fang et al. (0044) learns the weights for different LLMs to predict a specific attribute value. These task-specific approaches rely on su- pervised training for particular tasks, making their learned representations not generalizable across tasks. Zhuang et al. (2025) propose EmbedLLM, the first compact embedding supporting multi- ple tasks (e.g., model routing and benchmark prediction), but it is defined over fixed model set; adding models requires retraining and can shift the learned embeddings. Yax et al. (222) propose PhyloLM, a state-of-the-as approach for constructing LLM phylogenies. It defines ioter-model distances in tokenizer-awar manner: for models with the same tokenizer, it uses Nei's genetic distance over output token distributions; for different tokenizers, approximates distance using the first four characters of concatenated tokens. As a result, the cross-tokenizer metric is dominated by carly-token behavior and may not reflect sentence-lewe distributions. In contrast, our LLM DNA is a stable and generalizable sentence-level intrinsic property extractable independently for each LLM.
Fingerprint of LLMs. This line of work evaluates dependence or similarity between pairs of LLMs by mapping each model to representation, or fingerprint. Unlike watermarks, which actively insert a fingerprint during training (Zhao et al. 2024] Liu et al., 2024 Tang et al.) (025). fingerprinting is post hoc and analyzes identifiable model properties without modifying training. Prior studies
2

---

Published as a conference paper at ICLR 2026
verify whether remote LLM API serves the claimed model using strategic queries Pasquuiii al,
2025 or specialized prompts (asi et al. 2025) with response classification; Fu et al. (2025) further trams classifiers to capture stylistic signatures. Others go beyond text, leveraging parameters or
token embeddings for provenance prediction (Nikolic et al. 2025 Zhu et al. 2025). However, these
pairwise methods often structure unnenure-dependent and hard to scale to heterogeneous L.Mss. LLM
DNA, however, is a general representation capturing structural relationships across diverse models.
3 DEFINITION AND EXISTENCE OF LLM DNA
This section provides formal definition of LLM DNA in Section 3.1 Furthermore, Section 3.2 proves the existence of DNA under this definition. Additionally, Appendix elaborates that this definition satisfies two properties analogous to those of biological DNA.
3.1 DEFINITION OF LLM DNA
We formally define LLM function from discrete sequence of input texts vector of continuous logits D.finitionn..II, This definition applies to general LLMs with diverse architectures and adopts an end-to-end perspective. For the theoretical analysis, assume the input length is finite, i.e., bounded by constant m. Notably, "finiteness" does not imply smallness; the sets involved may still be arbitrarily large. Meanwhile, our practical Section 4 does not depend any particular value of m. Under this bound, the input space becomes finite, which directly yields Lemma A.4] a key step in establishing the existence LLM DNA.
pipeline in
Definition 3.1 (Large Language Model). Let E be a finite character set. Let E' be the set of all sequences of characters (strings) from Σ. For maximum sequence length E Z+, we define the spuce of admissible sequences as Sm. := {a Σ* |x| S ml). Let |Smm. A Large Language Sm O that maps admissible input sequence to vector of
Model (LLM) is function real- real-valued logits, with each logit corresponding to a possible output sequence in Sm. We denote the output space of LLM as O RNN. The set of all L.Ms constitutes the LLM function space, F.
Definition 3.2 (DNA of an LLM). Let (F, ,d) be the metric space of LLM functions. The DNA of an LIM f E F is a low-dimensional vector, denoted T; E 'D where the DNA space D = R. equipped with the standard Euclidean distance (T1, I| T22122 The mapping from F to DD must satisfy bi-Lipschitz condition; that is, there exist constants C2 > C1 > 0 such that for any LLMs fi, E F with corresponding DNAs Tfi and T2p it holds that: ei du ddr(S1, f2) S
dr dr(T), T12) S e2 ddr(f1.. (2)
Definition 3.2 guarantees that functionally similar LLMs have proximate DNAs, and vice versa. These lower and upper bounds in the bi-Lipschitz condition naturally guarantee two key properties of LLM DNA: (1) Inheritance-a lineage operation on an I.LM, such as fine-tuning, produces with similar DNAs exhibit
descendants with similar DNAs; and (2) Genetic determinism similar functional behaviors. Both properties are formally stated and proved in Appendix A.l
3.2 EXISTENCE OF LLM DNA
This section provides a formal proof for the existence of the LLM DNA. The core idea is that an LLM's function can be represented as a vector nigh-dimensional Hilbert space (Lemma A.4). This allows us to apply the Lohnson-Lindenstraus (JL) Lemma (Johnson et al. 1984), a powerful result in dimensionality reduction, to guarantee the existence low-dimensional embedding that preserves the geometry of the LLM functional space F.
Theorem 3.3 (Existence of LLM DNA), For any finite set of K LLMs Fx UJ,...., C
F. a DNA representation (Definition 3.2 with DNA space D Be_ exists, satisfying for all fo E FK. Ci dH(fi,f)) d, (Ti 02 (f( f,), where the target dimension L is T= O (102+ c1)
The proof of Theorem 3.3 appears in Appendix A.3 The required DNA dimension L trades off with
the bi-Lipschitz constants Cj C22 whose tightness captured by e = (C2 - c))(cc2 + c embedding distortion. High fidelity (small distortion) requires e c2 yielding very small and,
governs
3

---

Published as a conference paper at ICLR 2026
by L O log K), large L. Conversely, tolerating distortion (a larger gap between C1 and 02) increases e and permits a more compact, lower-dimensions DNA representation.
Theorem 5.3nno only guarantees the existence of an LLM DNA but also yiclds practical construc- tion. The proof leverages the constructive nature of the JL lemma: the required linear map E need not be specially engineered; scaled random linear projection satisfies the desired bi-Lipschitz property with arbitrarily high probability. This insight is formalized in the following corollary.
Corollary 3.4 (Construct LLM DNA via Random Projection). For any finite set of K LLMs FK [f...... fK] C. F, the LLM DNA in Theorem 3.3 can be obtained from random linear projection E: R with probability at least 2//K*
Beyond the probability bound established in Corollary 3.4 Larsen & Nelson (2014) proved that random linear projection is the optimal linear dimensionality reduction method. Leveraging both its optimality and computational efficiency. we adopt random Gaussian projection for extracting LLM DNA, with practical details introduced in Section 4
4 EXTRACT LLM DNA
Extracting LLM DNA in practice entails two challenges: (i) instantiating the output space O to reflect semantic content, since surface-fomm string comparisons brittle; and (ii) evaluating func- tional distance over the full input space Sm. which is computationally prohibitive. We propose an
extraction pipeline, RepTrace, which addresses these via a semantic response representation (Sec-
tion and an expectation-basel functional distance that admits efficient approximation (Sec-
tion
The full RepTrace pipeline provided in Section 4.3 and illustrated in Figure
Embeddling e E
Gaussian Random Matrix (atpp
p-t-
Embadding Madal
Project by AE
Embeddings
LLMs
Input Set samples) )
Responses samples)
Embeddings Ey E
DNA vector
Figure 1: Visualization of RepTrace: LLM DNA extraction workflow
4.1 CONSTRUCT SEMANTIC-AWAR REPRESENTATION
This subsection details how to build a semantic- aware vector representation (xf(x)) for each LLM response z; such that [] (z:))-<(ff (z:))]l2 ((2(())))
effectively approximates Il (2r,)-f(zz))) Purely string-based comparisons ignore semantic information: for example, the words "holiday" and "va- cation" are semantically similar yet differ completely character sequences. To convert textual outputs into vectors that capture semantic meaning, we use sentence-embedding model to map each response fixed-size embedding.
This design has three benefits: (1) it is LLM-agnostic for text-generation models, requiring no ac- cess to internal architectures or tokenizers as in many embedding/model based methods (aax al,, 2025); (2) it applies to closed-source LLMs because API-only models can still be queried for DNA computation; and (3) it accommodates new LLMs without recomputing existing DNAs, overcoming the limitation of EmbedLLM (Zhuang al. 2025)
4.2 APPROXIMATE LLM FUNCTIONAL DISTANCE
This subsection details our practical approach to measuring the distance between LLMs. Though direct computing functional distance is theoretically sound, its computation requires evaluating func- tions over the entire, combinatorially large input space Sm, which is infeasible. To bridge theory and practice, introduce tractable alternative: Stochastic Functional Distance. This metric is defined not over the full input space, but as a statistical expectation over a random sample of inputs, reflecting the practical scenario of evaluating models finite set of representative prompts.
4

---

Published as a conference paper at ICLR 2026
Definition 4.1 (Stochastic Functional Distance). Let fi, E be two LLM functions. Let be a probability measure on the input space Sm. and let t E 2* be a fixed sample size. Let S = (21....,,I:) [z) denote set of t independent and identically distributed (i.i.d.) random variables representing inputs drawn the distribution The Stochastic Functional Distance d [11, is defined as the expected Euclidean distance between the concatenated semantic representations of the LLMs' outputs over random sample St: dy(,i, == -ESIpp [V V Emyes, [1)()) f2(x,))
concentration bound provides a formal reliability guarantee for our empirical distance. The the- orem below shows that the estimate concentrates sharply around the true Hilbert distance dh: for any deviation level, the tail probability decays exponentially the sample size t. Thus, the estimate offers high-confidence measure of the underlying functional distance.
Lemma 4.2 (Concentration of Empirical Functional Distance). Let f1, E F be two LLM func- tions. Let d² = -E²= (x)) f2(2z))l be the squared empirical functional distance cal- culated from sample of l i.i.d. inputs {215-..,,44) Let d be the true squared Hilbert dis- tance. Assume the squared Euclidean distance between
two output vectors is bounded such that [lf((z) f2(x))11 S Cwax for any input x and for all functions of interest. Then, for any > 0, the
following bound holds for the sample average: P (
dz 2 26) I 2eexp
24c
C
The proof of Lemma 4.2 is included in Appendix A.4 Based on the theoretical bound, we employ the empirical estimate in Definition 4.1 to compute the functional distance in practice. The process begins by sampling a set of representative prompts, S, = {II,,.., Te.) from distribution that reflects real-world inputs. For given LLM f. we collect the textual response for each prompt Iy E St and encode it into a semantic vector as described in Section[4.]] These response vectors are then concatenated end-to- eo-dd. This procedure yields a single, high-dimensional vector that serves as functional representation for the LLM over the sampled inputs. 'he squared Euclidean distance between two such representations approximates the squared empirical functional distance df. which further a robust and computable estimation of the true squared Hilbert distance dyg.
4.3 OVERALL PIPELINE
This section presents RepTrace algorithm, training-free end-to-end DNA extraction pipeline. The procedure is detailed in Algorithm Iarithhm[aand and visualized in Figure We begin by selecting a sample input S, of prompts drawn from real-world datasets. Each input x; is fed an LLM f to generate a textual response Yi (line 4), which is then encoded into a fixed-size semantic vector using a sentence-embe -entence-embeddiigg model (line 5). These embedding vectors are then concatenated form single, nigh-dimensional functional representation E, for the LLM (line 6). Finally, a pre-computed random Gaussian projection (line 1) linearly projects this high-dimensional representation Ey into a low-dimensional vector Ty E L (line 7), referred to as the DNA of the LLM.
Algorithm 1: RepTTrcc:: LLM DNA Extraction Pipeline
Input
Max length me, sample size DNA dimension L; sentence-embeddding model Σ" R'; sample input set S, {aı,..., ........ 21) drawn uniformly from real-world datasets; A collection of LLMs Jx = //I....,It C.F
Output: DNA vectors (Ty E R² Fx).
Initialize random Gaussian projection: draw A E (x(p-)) with Aya 1/0,1
foreach f E Fx do
for + to do
< (x;)
// Textual response of f on input iy // Semantic embedding
0()) RPP
E5 < [e1, (a.,,,,.,,,. R
// Concatenate response embeddings // Project into DNA space
Ti + A AE E R
return (Ty FK}
5

---

providing data- Published as a conference paper at ICLR 2026
5 EXPERIMENTS
This section provides a rigorous empirical validation LLM DNA as robust functional fingerprint. Following our experimental setup across 305 models ($5.1 $B) we demonstrate DNA's efficacy in relationship detection ($5.2) and training-free model routing ($5.3). We verify the representation'
stability through stress tests involving data heterogeneity ($5.4), synthetic random inputs ($5.5), chat
template variations (55.6). and fine-tuning intensity ($5.7). Systematic ablations characterize the influence of bottleneck embedding models ($C.3), pre-aggregation dimensions p ($C.4), and DNA length L ($C.5) To address technical edge cases, we analyze parameter-size bias ($C.6), same- tokenizer performance ($C.7), and relationship "false positives" ($C.2) We conclude by leveraging these functional distances to reconstruct a phylogenetic tree of LLMs (5.8
driven map of the field's architectural evolution.
5.1 EXPERIMENTAL SETTING
Datasets. We evaluate across a diverse set of datasets covering question answering, common- sense reasoning, and general natural language understanding. Specifically, the pipeline integrates six widely used benchmarks, including SQuAD (Rajpurkar et al., 2016), CommonsenseQA Tal- mor et al. 2018), HellaSwar wag (Zellers al. 2019), Winogrande (Sakaguchi al., (020), ARC- Challenge (Clark et al. 2018). and MMLU (Hendrycks et al.] 2021). DNA extraction employs mixture of all six datasets, with each dataset contributing 100 random samples. Besides, Em- bedLLM (Zhuang et al. (025) dataset is also used to evaluate model routing, though not involved in DNA extraction. For each dataset, prompts are drawn from their designated text fields (e.g., question, sentence, or ctx).
LLMs. This study spans 305 LLMs released by 153 organizations (e.g. Qwen, Llama, etc.) on Hugging Face. The evaluation focuses on text-generative models, covering both decoder- only and encoder-decoder architectures, with parameter scales ranging from 100M to 70B. Both instruction-tuned base models are included in the evaluation. Text embeddings are tracted with the state-of-the-art open-source encoder Qwen/ (wen3 Embeddinng---b-ee 8B by de- fault, with performance comparison sentence-trrass forme s 1all1-mmmettbaseeee and pAAI/bbee-arggeeee
presented. More model details are included in Appendix
5.2 DETECT LLM RELATIONSHIPS
In this subsection, we validate DNA against Zhu et al. (2025), scale to 305 LLMs to demonstrate accurate relationship detection, and visualize the distribution to uncover undocumented model pairs.
Figure 2 shows the DNA distribution of the eight models (four correlated, four independent) in Table of Zhu et al. (2025). The decision boundary is computed by support vector machine (SVM) with an RBF kernel, indicating that the DNAs of correlated and independent models clearly separated. This result shows that the DNA structure is consistent with both the experiments in Zhu et al. (2025) and the corresponding Hugging Face documentation.
Table extends this experiment to 305 models. Using the official Hugging Face relationship (the "Model Tree"") ground truth for correlated models, we obtain 83 correlated pairs. We additionally sample 83 random pairs and treat them as independent. The resulting dataset is split 8:2 into training and test sets. Three baselines are compared alongside our approach: Rundom-uniformll guessing independent or correlated; Greedy Greedy-predicting that all within-organization pairs are correlated and orasaizgganiztiionn pairs are independent; and PhyloLM (Yax al. (025) -geneticc (Nei) distances to other LLMs in the training set are regarded as a signature vector. For PhyloLM and DNA, an SVM with RBF kernel is used for prediction. From Table T three observations emerge. First, DNA significantly outperforms the baselines and achieves high AUC of 0.992, indicating accurate detection of relationships among heterogeneous LLMs. Second, DNA attains higher recall than precision; this does not necessarily indicate flaw in DNA, but rather suggests the presence of undocumented relationships in the official model tree. A detailed case study of the misclassified pairs appears in Appendix C22 Third, DNA quality is robust to the choice of eembedcc- model. Using BGE (0.3B) and MPNet (O.IB). despite their much smaller parameter sizes, yields comparable-ant and in this relation prediction task even perfornepefo

---

Published as a conference paper at ICLR 2026
Destance Natria Compladtee, squad, Loc hajy9 Mastil
1:ME of DNA offereeee ES r6M0ii LMMW2-76611
20-
DNA distribution of LLMs evaluated by Zhu et al. (2025). "Independent" and "Cor- related" relative to Llama-2-7B-hff are based on public documents. The boundary is computed by an SVM with RBF kernel, indicating that the DNAs of "Independent" and "Correlated" mod- els are clearly separated.
Figure 2:
Mantel test between DNA ex- tracted from two disjoint datasets. Each point represents a single pair of models, plotted by their distance in the first dataset versus their distance in the second, showing a strong cor- relation (Pearson-r 0.7797) and high sta- tistical significance (p-value < 0.0001).
Figure 3:
Figure E presents the t-SNE visualization of all 305 models, resulting in three main observations. First, models from the same organization and model family cluster tightly (e.g.. Qwen3, Llama), confirming that DNA can be used to detect relationships among LLMs. Second, fine-tuned mod- els are closely related to their base models, confirming the inheritance property. For instance, DeepSeck-distille Qwen models locate near the Qwen3 cluster (right, purple). Third, some lationships that are not well documented appear closely in the visualization. For example, both Microsoft crca-2-1- Tree", state in their model cards that they are fine-tuned from Llama without specifying the ver- sion. In Figure 4 vicuna- 7b-vl. is located within the Llama-base cluster (left-middle, green), and orca-2213b lies in the Llama-char cluster (upper-mu (upper-middle, middle, green), suggesting their likely base versions in fine-tuning. This demonstrates the capability of DNA to detect related LLMs.
and LMSYS vi cuuna-7bvl... 1, both of which are not in the "Model
Table I: DNA LLM-relation detection test performance (mean +standard deviation across five seeds)
Method
Accuracy
Precision
Recall
F1
AUC
Random
0.493 +0.036
0,515 #0.034
0.516 =0,042
0,516 20.036
0.492 #0.036
Greedy
0.593 #0.018
0.759 #0.081
0.329 #0.000
0.458 #0.014
0.606 #0.023
PhyloI SVM
0.742 #0.058
0.741 #0.073
0.812 123
0.766 =0.057
0.788 #0.060
DNA (Qwen3)+ + SVM
0.919 #0.048
0.898
#0.049
0.957 #0.072
0.925 #0.047
0.979 #0.027
DNA (BGE) + SVM
0.934 +0.053
0.905 +0.068
0.982 #0.059
0.940 #0.050
0.992 #0.023
DNA (MPNet) + SVM
0.932 +0.048
0.914 #0.065
0.966 +0.069
0.937 =0.047
0.990+0.0220
Table 2: DNA-based routing accuracy test set
Random
Single Best
EmbedLLM (Matrix Factorisation)
DNA
0.402 20.016
0.607 +0.000
0.665 40.003
0.672 +0.008
5.3 MODEL ROUTING
This section investigates how DNA can benefit model routing. We use the same dataset, data split, baselines, and metric as EmbedLLM (Zhuang et al. 2025). EmbedLLM learns a compact LLM representation during routing training. To compare the quality of DNA with EmbedLLM-leamme embeddings, within the matrix factorization framework of EmbedLLM, we replace the LLM em- bedding layer and its subsequent linear layer with frozen DNAs. These DNAs directly compared
7

---

Published as a conference paper at ICLR 2026
SIIE projection ORA sgrature orgertaator
30
LA
CAA VEPUROA
Den Dale Thlcae
1-ONE Composee
-10
-20
-30
-20
-10
D
10
20
Component
Figure 4: Visualization of DNAs by t-SNE. Colors denote organizations releasing LLMs. Organi- zations with fewer than five LLMs are collapsed into "Others". Background regions are obtained by localized DBSCAN started where each organization forms group of more than three models.
with the query embeddings to make routing decisions. Notably, 11 models reported by Zhuang et al. (2025) either no longer exist or failed to run; their results are therefore omitted. Hence, our numbers do not perfectly align with those in Zhuang et al. (2025).
Table 2 reports routing accuracy, defined by Zhuang et al. (0255) as the fraction of queries routed to an LM that answers correctly. The results show that frozen DNA outperforms the embeddings learned by EmbedLLM. This result is notable because EmDedLLM explicitly learns representations on this routing dataset, whereas LLM DNA is training-frre and task-agnostic. Achieving parity with- out task-specific optimization validates the quality of DNA and highlights its superior scalability: unlike EmbedLLM,, our model supports arbitrary new models without retraining.
5.4 STABILITY OF DNA ON HETEROGENEOUS DATASETS
This subsection examines how the choice of sampled distribution S₁ influences the quality of the generated DNAs. We assess this effect with a Mantel test between DNAs extracted from two disjoint datasets. Specifically, we construct one dataset the union of 100 samples from SQuAD and 100 samples from CommonsenseQA (squad_oqa) and another as the union of 100 samples from HellaSwag and 100 samples from Winogrande (hs wg). We compute DNAs from the same set of models, then calculate the distances between randomly selected pairs of models, visualize them in Figure 3 and compute the Pearson correlation and its p-value. From Figure 3, we observe strong correlation (Pearson r > 0.75) between the two distance structures with very high statistical significance (p < 0.001). These results indicate that the structure of DNAs does not depend on the sampled data from which it is extracted, indicating the stability of our DNA extraction approach.
5.5 DNA EXTRACTED FROM RANDOM DATA
This subsection shows that LLM DNA remains discriminative even under random inputs. We gen- erate 600 synthetic wor-word prompts using wonderwords RandomWord (6x100, matching the main protocol), run each LLM to obtain outputs, embed them with Beeen---B-Ebeedee using the same DNA extraction pipeline, and evaluate DNAs via relation prediction. An exam-
8

---

Published as a conference paper at ICLR 2026
ple slice of the random inputs is: "disdain chapel intention gymnast activation codpiece hypothesis endothelium masonzy".
The results reported in Table Bl Interestingly, the random-data setting even slightly outperforms the standard benchmark. This highlights that the DNA extraction method is largely insensitive to input-dataset bias. The key intuition is that DNA analyses (e.g., relationship prediction) target rel- ative relationships among LLMs rather than absolute accuracy. Different datasets act as different "views" of the same model set: suboptimal view may obscure some relations, but it unlikely to fabricate spurious ones-for instance, it will not make two genuinely similar models appear unre- lated. This observation also aligns with the stability we see under random projection.
Table 3: DNA (Qwen3) relation prediction performance (DNA extracted from random inputs)
Method
Accuracy
Precision
Recall
AUC
PhyloLM + SVM
0.742 #0.058
0.741 #0.073
0.812 40.123
0.766 #0.057
0.788 #0.060
DÑA + SVM
0.919 048
0.898 +0.049
0.957 +0.072
0.925 +0.047
0.979 +0.027
DNA (Rand) + SVM
0.949 #0.048
0.931 #0.053
0.977 #0.062
0.952 +0.048
0.987 +0.038
5.6 ABLATION STUDY FOR CHAT TEMPLATE
We study how chat templates influence DNA extraction for instruction-tuned models. Chat templates inject the special tokens and formatting used during instruction tuning to elicit conversational be- havior. For an instruction-tuner model (e.g., Iamma-IInstuucee we compare two protocols: (i) bypass the template and feed the raw prompt directly (completion-style), and (ii) apply the template and compute DNA using only the extracted assistant's response. For non-chat (base/complction) models, the extraction pipeline is unchanged. We report relationship prediction performance under the same setting as Table Removing the chat template consistently improves accuracy by 1-3% across encoders, suggesting that for cross-family comparisons involving both chat-tuned and com- pletion models, omitting templates yields a fairer and more direct functional comparison by evalu- ating chat-tuned models through the same completion-style interface as their base counterparts.
Table 4: DNA relation prediction performance: with vs without chat templates
Encoder
Template
Accuracy
Precision
Recall
F1
AUC
w/ template
0.919 #0.048
0.898 +0.049
0.957 #0.072
0.925 #0.047
0.979 #0.027
Qwen3
w/o template
0.930 =0.045
0.916 #0.067
0.961 =0.071
0.935
#0.044
0.994 #0.021
w/ template
0.932 =0.04 00.048
0.914 #0.065
0.966
#0.069
0.937
#0.047
0.990 =0.020
MPNet
w/o template
0.958 #0.047
0.943 #0.060
0.982 #0.059
0,960
#0.047
0.994 #0.016
w/ template
0.934 #0.053
0.905 +0.068
0.982 =0.059
0,940 =0.050
0.992 #0.023
BGE
w/o template
0.945 #0.054
0,929 #0.073
0.974 #0.066
0.948 #0.052
0.992 #0.023
5.7
EFFECT OF FINE-TUNING ON THE DNA VALUES
This subsection quantifies how fine-tuning changes DNA values. Starting from -lama3-8B-IInstuct we fully fine-tune all parameters on subsets of the NVIDIA OpenMathInstruct-2 dataset (Toshniwal et al. 2024) with sizes 10, 100, 1000, 10000. Since 1.lama3 was released before OpenMathInstrruct- pretraining corpus nor our DNA extraction set. Figure 5 compares the DNAs of the original and fine-tuned models and reports their L2 distances to the original DNA.
this dataset appears in neither its
We draw two conclusions. First, the DNA distance increases with the amount of fine-tuned data, showing monotonic but nonlinear trend. Specifically, the distance rises sharply after fine-tuning on a tiny external subset (e.g., the first 10 samples causes L2 = 0.69) and then grows more gradually (up to 0.80 with 10,000 samples) additional samples are learned. Second, the global DNA structure remains stable: most subsequences preserve their dark/ight patterns with only minor variation, sug- gesting that fine-tuning alters local traits while leaving inherited, intrinsic behaviors largely intact.

---

Published as a conference paper at ICLR 2026
DNA Signature Heatmap: Original Finetuned Models
Rrstaned Cii
Node
TomA 12H07000
121-0 8014
DMA Dinerrsion
Figure 5: Shift of DNA values when (full-parameter) fine-tuning a. 3-83--3-8---- -aa-Instruct with rustt subsets of size
5.8 PHYLOGENETIC TREE OF LLM
Analogous to phylogenetic analysis in biology, once LLM DNA is available, we can construct a phy- logenetic tree to visualize model evolution. Specifically, we adopt the widely used Neighbor-Joining (NJ) method (Saitou & Nei, 1987) based on DNA distances, followed by the default midpoint- rooting strategy. Figure 6 visualizes this tree, with branches ordered by their leaf counts. Notably, without access to release dates, LLM DNA distances generally recover the actual evolution path, including: (i) architectural shifts from encoder-decoder (e.g., Flan-T5) to widely used decoder-only models (e.g., Llama, Qwen); (ii) temporal progression from 2023 to 2025; and (iii) lineage evolu- tion within LLM families (e.g., Llama 2 to Llama 3). Moreover, since longer branches represent greater functional distance between successive model families, the tree suggests distinct evolu- tionary speeds across model families; for instance, Qwen and Gemma evolve faster than the Llama series. more detailed phylogenetic tree with one model per leaf provided in Appendix C.1
Phylogenetic Tree LLM Model Groups by Architecture and Organization
Organizations teril
Owenz.5
Queen
Owee]
gemma3-it
Model Groups
Instruct
Qwen1.5
FB2
(333))
flan-ts
1202:
0.20
Evolutionary Distance
Figure 6: Phylogenetic Tree of LLM families built from DNA & distances with NJ algorithm
6 CONCLUSION
In this paper, we introduced LLM DNA, formal framework for representing the functional be- havior of large language models. We mathematically defined LLM DNA as a low-dimensional vector, proved its existence, and established its key properties of inheritance and genetic determina- tion. Based on this theory, we developed a practical, training free pipeline to extract DNA from a wide range of models. Our experiments 305 LLMs demonstrate that DNA effectively detect both documented and undocumented evolutionary relationships, and constructing a phylogenetic tree from these DNAs provides novel way visualize the evolution of the L.M ecosystem.
10

---

Published as a conference paper at ICLR 2026
REPRODUCIBILITY STATEMENT
To facilitate reproducibility. we release our source code at Xtra- Computing/LLM DNA and an interactive demo at https::/github... com/ We also provide a PyPI package, available at /ttp:::/pyyp..org//rrooo which can be installed via pip install 11m-dna. Regarding theoretical and experimental de- tails, proofs appear in Appendix A data specifications in Section 5.1 and hyperparameters, model details, and licenses in Appendix
ETHICAL STATEMENT
This study does not involve human subjects. All models used are open source, and we comply with their licenses; use is solely for research purposes. The technique does not pose any significant privacy or security risk.
ACKNOWLEDGEMENT
This research/project is supported by the National Research Foundation, Singapore and Infocomm Media Development Authority under its Trust Tech Funding Initiative. Any opinions, findings and conclusions or recommendations expressed in this material are those of the author(s) and do not reflect the views of National Research Foundation, Singapore and Infocomm Media Development Authority.
REFERENCES
Pengzhou Cheng, Zongru Wu, Tianjie Ju. Wei Du, and Zhuosheng Zhang Gongshen Liu. Trans- ferring backdoors between large language models by knowledge distillation. arXiv preprint 2988082008.098778 2024.
Peter Clark, Isaac Cowhey, Oren Etzioni, Tushar Khot, Ashish Sabharwal, Carissa Schoenick. and Oyvind Tafjord. Think you have solved question answering? try arc, the ai2 reasoning challenge. arXiv preprint arXiv: /003.0/800000555 2018.
Sanjoy Dasgupta and Anupam Gupta. An elementary proof of a theorem of johnson and linden- strauss. Random Structures & Algorithms, 22(1):60-65, 2003.
Dujian Ding, Ankur Mallick, Chi Wang, Robert Sim, Subhabrata Mukherjee, Victor Ruhle, Laks VS Lakshmanan, and Ahmed Hassan Awadallah. Hybrid I1m: Cost-efficient and quality-aware query routing. arXiv preprint arXiv:2404.14613 2024.
Tian Dong, Shaofeng Li, Guoxing Chen, Minhui Xue, Haojin Zhu, and Zhen Liu. Rai2: Responsible identity audit governing the artificial intelligence. In NDSS, 2023.
Moming Duan, Mingzhe Du, Rui Zhao, Mengying Wang, Yinghui Wu, Nigel Shadbolt, and Bing- sheng He. Position: Current model licensing practices are dragging into quagmire of legal noncompliance. In Forty-second International Conference on Machine Learning Position Paper Track, 2025.
Chenhao Fang, Xiaohan Li, Zezhong Fan, Jianpeng Xu, Kaushiki Nag. Evren Korpeoglu, Sushant Kumar, and Kannan Achan. Lim-ensemble: Optimal large language model ensemble method for e-commerce product attribute value extraction. Proceedings of the 47th International ACM SIGIR Conference Research and Development in Information Retrieval, pp. 2910-2914, 2024.
Zhiyuan Fu, Junfan Chen, Hongyu Sun, Ting Yang, Ruidong Li, and Yuqing Zhang. Fdllm: text fingerprint detection method for 1lms in multi-language, multi-domain black-box environments. arXiv e-prints, pp. 2501, 2025.
Shanshan Han, Qifan Zhang, Yuhang Yao. Weizhao Jin, and Zhaozhuo Xu. Lim multi-agent sys- tems: Challenges and open problems. arXiv preprint arXiv:2402.035778 2024.

---

Published as a conference paper at ICLR 2026
Dan Hendrycks, Collin Burns, Steven Basart, Andy Zou, Mantas Mazeika, Dawn Song, and Jacob Steinhardt. Measuring massive multitask language understanding. Proceedings of the Interna- tional Conference on Learning Representations (ICLR), 2021.
Jie Huang and Kevin Chen-Chuan Chang. Towards reasoning in large language models: A survey. In Findings of the association for computational linguistics: ACL 2023, pp. 1049-1064 2023.
Yichong Huang, Xiaocheng Feng, Baohang Li, Yang Xiang, Hui Wang, Ting Liu, and Bing Qin. Ensemble learning for heterogeneous large language models with deep parallel collaboration. Advances in Neural Information Processing Systems, 19838881-81988 2024.
Hugging Face. Hugging face. httpps://uugi.nggfat co. 2025. Accessed: 2025-09-24.
William B Johnson, Joram Lindenstrauss. al. Extensions of lipschitz mappings into a hilbert space. Contemporary mathematics, 26(189-206):1 1984.
Kasper Green Larsen and Jelani Nelson. The johnson-lindenstraus lemma is optimal for linear dimensionality reduction. arXiv preprint arXiv: 2014.
Sunbowen Lee, Junting Zhou, Chang Ao, Kaige Li, Xinrun Du, Sirui He, Haihong Wu, Tianci Liu, Jiaheng Liu, Hamid Alinejad-Rokny, et al. Quantification of large language model distillation. arXiv preprint urXiv22511.12619 2025.
Aiwei Liu, Leyi Pan, Xuming Hu, Shiao Meng. and Lijie Wen. A semantic invariant robust water- mark for large language models. The Twelfth International Conference on Learning Represen- lutions, 2024.
Quang H Nguyen, Thinh Dao, Duy C Hoang, Juliette Decugis, Saurav Manchanda, Nitesh V Chawla, and Khoa D Doan. Metallm: A high-performant and cost-efficient dynamic framework for wrapping llms, arXiv preprint ar arXiv:2407. 10834, 2024.
Ivica Nikolic, Teodora Baluta, and Prateek Saxena. Model provenance testing for large language models. arXiv preprint arXiv:2502.00766, 2025.
Isaac Ong, Amjad Almahairi, Vincent Wu, Wei-Lin Chiang, Tianhao Wu, Joseph E Gonzalez, M Waleed Kadous, and Ion Stoica. Routellm: Learning to route Ilms with preference data. In International Conference Learning Representations (ICLR), 2025.
Dario Pasquini, Evgenios M Kornaropoulos, and Giuseppe Ateniese. (LLMmap):: Fingerprinting for large language models. In 34th USENIX Security Symposium (USENIX Security 25), pp- 299-318, 2025.
Mohammad Shahedur Rahman, Runbang Hu, Peng Gao, and Yuede Hugginggraph: Understand- ing the supply chain of IIm ecosystem. arXiv preprint arXiv:2507.14240, 2025.
Pranav Rajpurkar, Jian Zhang, Konstantin Lopyrev, and Perey Liang. Squad: 100,000+ questions for machine comprehension of text. arXiv preprint arXiv: 1606.05500. 2016.
Mark Russinovich, Ahmed Salem, and Ronen Eldan. Great, now write an article about that: The crescendo (Multi Murti-TTrn-aa } (LLM} jailbreak attack. In 34th USENIX Security Symposium (USENIX 2025.
Security 25), pp.
Naruya Saitou and Masatoshi Nei. The neighbor-joining method: a new method for reconstructing phylogenetic trees. Molecular biology and evolution, 4(4):406-424 1987.
Keisuke Sakaguchi, Ronan Le Bras, Chandra Bhagavatula, and Yejin Choi. Winogrande: An adver- sarial winograd schema challenge scale. In Proceedings of the AAAI Conference on Artificial Intelligence, volume 34, pp. 8732-8740, 2020.
Alon Talmor, Jonathan Herzig, Nicholas Lourie, and Jonathan Berant. Commonsensega: A question answering challenge targeting commonsense knowledge. arXiv preprint 1xXvv:8811.0093, 2018.
Ling Tang, Yuefeng Chen, Hui Xue, and Quanshi Zhang. Towards the resistance of neural network watermarking fine-tuning, arXiv preprint 2rX0v:2505.01007, 2025.
12

---

Published as a conference paper at ICLR 2026
Shubham Toshniwal, Wei Du, Ivan Moshkov, Branislav Kisacanin, Alexan Ayrapetyan, and Igor Gitman. Openmathinstruct-2: Accelerating ai for math with massive open-source instruction data. arXiv preprint arXhv:2410.01566 2024.
Yun-Yun Tsai, Chuan Guo, Junfeng Yang, and Laurens van der Maaten. Rofl: Robust fingerprinting of language models. arXiv preprint arXiv:2505. /2682, 2025.
Xinyu Wang, Hainiu Xu, Lin Gui, and Yulan He. Towards unified task embeddings across multiple models: Bridging the gap for prompt- based large language models and beyond. arXiv preprint arXiv:2402. 14522, 2024.
Zhaomin Wu, Jizhou Guo, Junyi Hou, Bingsheng He, Lixin Fan, and Qiang Yang. Model-based large language model customization as service. In Proceedings of the 2025 Conference on Empirical Methods in Natural Language Processing, pp. 4904 4921, 2025.
Zhaomin Wu, Mingzhe Du See-Kiong Ng. and Bingsheng He. Beyond prompt-induced lies: In- vestigating Ilm deception on benign prompts. In Proceedings of the International Conference on Learning Representations (ICLR), 2026.
Zhenhua Xu, Xubin Yue, Zhebo Wang, Qichen Liu, Xixiang Zhao, Jingxuan Zhang, Wenjun Zeng, Wengpeng Xing, Dezhang Kong, Changting Lin, et al. Copyright protection for large language models: A survey of methods, challenges, and trends. arXiv preprint arXiv:2508. 151.:2508.11488, 2025.
Wenkai Yang, Xiaohan Bi, Yankai Lin, Sishuo Chen, Jie Zhou, and Xu Sun. Watch out for your agents! investigating backdoor threats IIm-based agents. Advances in Neural Information Processing Systems, 37:100938-100966 2024.
Nicolas Yax, Pierre-Yve Oudeyer, and Stefano Palminteri. Phylolm: Inferring the phylogeny of large language models and predicting their performances in benchmarks. Proceedings of the International Conference Learning Representations (ICLR), 2025.
Rowan Zellers, Ari Holtzman, Yonatan Bisk, Ali Farhadi, and Yejin Choi. Hellaswag: Can ma- chine really finish your sentence? arXiv preprint arXiv:1905.07880 2019.
Haodong Zhao, Jinming Hu, Peixuan Li, Fangqi Li, Jinrui Sha, Tianjie Ju, Peixuan Chen, Zhu- osheng Zhang, and Gongshen Liu. Nsmark: Null space based black-box watermarking defense framework for language models. arXiv preprint arXiv:2410.1390 2024.
Wayne Xin Zhao, Kun Zhou, Junyi Li, Tianyi Tang, Xiaolei Wang, Yupeng Hou, Yinggian Min, Beichen Zhang, Junjie Zhang, Zican Dong, et al. survey of large language models. arXiv preprint arXiv:2303.1822 18223, 1(2), 2023.
Sally Zhu, Ahmed M Ahmed, Rohith Kuditipudi, and Percy Liang. Independence tests for language models. In Forty-second International Conference on Machine Learning. 2025.
Richard Zhuang. Tianhao Wu, Zhaojin Wen, Andrew Li. Jiantao Jiao, and Kannan Ramchandran. Embedllm: Learning compact representations of large language models. International Confer- ence on Learning Representations (ICLR), 2025.
13

---

Published as a conference paper at ICLR 2026
Appendix
Table of Contents
A
Proof
14
A.1
LLM Properties: Inheritance and Genetic Determinism
14
A.2
LLM Functional Space is a Hilbert Space
15
A.3
Proof Theorem 3.3
16
Proof Theorem
A.5
Proof Corollary
17
B
Experimental Details
17
B.
Hyperparameters
17
8.2
Environment
18
C
Supplementary Experiments
18
C.I
Phylogenetic Tree of LLMs
18
Detailed Analysis of Falsely Predicted LLM Relationship
18
C.3
Ablation Study of Bottleneck Embedding Model
18
C.4
Ablation Study of the Embedding Model's Output Dimension p
19
Ablation Study
DNA Dimension
20
C.6
Effect of LLM's Parameter Size DNA Quality
21
7
Relation Prediction for LLMs with Same Tokenizer
21
D Broader Impact
22
Limitations
23
F Model Details
23
G Large Language Model Usage
24
A PROOF
A. LLM PROPERTIES: INHERITANCE AND GENETIC DETERMINISM
Before discussing the properties, begin by defining the evolution of an LLM.
Definition A.1 (Evolution). Let F be the LLM function space in Definition 3.1 and let d be the
Hilbert distance on F. For threshold 6 > an evolution is function F - F such that for any f F,
(f(f 8h.
If fo E(f)), we say fe is evolved from fi within ōh-
This notion covers wide range of operations that induce bounded shift in an LLM's functional behavior, including not limited to fine-tuning, distillation, and reinforcement learning, as they modify the LLM within bounded Hilbert-distance threshold.
The definition of LLM DNA gives rise to several properties analogous to biological DNA. The two most fundamental are inheritance and genetic determinism introduced at the start of this section. Drawing a parallel with biology, inheritance suggests that an evolutionary process acting on an LLM, such fine-tuning, produces descendants with similar DNAs. Genetic determinism indicates LLMs with similar DNAs behave similarly. Both properties can be mathematically derived from the definition of DNA (Definition 3.2 in following forms.
14

---

Published as a conference paper at ICLR 2026
Theorem A.2 (Inheritance).. Let LLM f2 F be derived from LLM fi F, with corre- sponding DNAs T/2 and Tn,- For any desired DNA proximity E > 0, there exists a Hilbert distance threshold 6n 0 such that du (f < ön. then d, (T TT) <.-
Proof. This property is a direct consequence of the bi-Lipschitz condition in Definition must show that for any & 0, we can find бн > 0 satisfying the implication:
3.2
We
(n(si, < 6н
d-(Tfi, Tf) 6
(1)
The bi-Lipschitz condition provides the upper bound d-(T,,, T/2) I 02 du (f1, f2) for some con- stant 02 > O. Let choose бн = c//22. Since c, > 0 and > 0, threshold 6y is also positive.
Now, assume du (fi. f2) < бн for some pair of LLMs. follows that:
(T) S 0o du (2).
(by the Lipschitz condition)
< 02 δн
(by assumption)
This confirms the implication Equation [ and the theorem holds.
Theorem A.3 (Genetic Determinism). Let SI, J2 F be two LLMs with corresponding DNAs
Tfi Tfp E D. For any desired functional proximity eg > 0, there exists a DNA distance threshold 6, such that if d, (T)2: 8G, then (f(f1, f2) EH-
Proof. This property follows from the lower bound of the bi-Lipschitz condition. We need to show that for any given е > O, there a 6, such that:
(T Th) 6- dn((i, cн
(2)
The bi-Lipschitz condition states that e₁ (H( S Rearranging this gives bound the Hilbert distance:
for some constant Ci > 0.
du(fi. < d (T),
(3)
C1
Let choose 8, = 6 C1. Since си > 0 and e₁ > 0, it follows that 5, 0. Assuming
d, (T) < 87, we have:
dz(fi.f2) <
d, (T)iTTa)
(from Equation 3
C1
d,
(by assumption)
C1
(en 01) = 6H1
Thus, the implication holds, proving the theorem.
A.2 LLM FUNCTIONAL SPACE IS A HILBERT SPACE
Lemma A.4 (LLM Functional Space is a Hilbert Space). Let F be the space of L.M functions mapping from the funite input space S. = {x1..... .INN to the vector space of outputs O = N Let μ. Sm + [0, 1] be a probability distribution on the input space, assigning a probability p(x,) to each sequence such that EN p(xx)) = 1. The pair (F,dg) forms Hilbert space, where the distance metric dy is defined as: (H( - VEM, ((z)))li(x;) f2(24)1B
Proof. We prove this by showing that F is a vector space equipped with a valid inner product.
Vector Space Structure. Any function f F is uniquely determined by the tuple of its outputs. We can represent f by its concatenated output vector V := f( E ON This establishes a natural isomorphism between and the vector space ON from which inherits its vector space structure.
15

---

Published as a conference paper at ICLR 2026
Inner Product Validation. We define weighted inner product on F as:
N
(2, Σ
(4)
where oo is the standard dot product on O R This function inherits linearity and symmetry from the standard dot product. Sesitive-defniteness is guaranteed by the condition that μu(z;) > for all i. The distance metric induced by this inner product is precisely d
d2) v(i 12 -2))
Σ p(aa)llsi f2(x:)1
(5)
As a complete inner product space, (F,dH) is Hilbert space.
A.3 PROOF OF THEOREM 3.3
Theorem 3.3 (Existence of LLM DNA), For any finite set K LLMS Fx (fi,..., fk) c
F. a DNA representation (Definition 3.2
with DNA space D RL exists, satisfying for all
f E FK, C1 (f(fi,f) S de (T T S c2 df fJ), where the target dimension L is a)) log
L= 0 (1(e++a)/()
Proof. We aim to construct mapping from the LLM function space F to low-dimensional DNA space 'D that satisfies the bi-Lipschitz condition in Definition (3.2 for any given constants C2 > e1 > This achieved by applying the Lohnson-Lindenstrauss (JL. Lemma.
First, we define a symmetric distortion parameter and a scaling factor a from our target constants C1 and e2:
(2
02
C2 +
2
Since C2 > a > 0, follows that E (0, 1), which is a valid distortion parameter for the JI. Lemma.
We invoke the JL Lemma with this The lemma applies because, shown in Lemma A.4 (F, dp) a Hilbert space, even when du is defined with the weighting function μu(z). The lemma guarantees the existence of linear map E F R2,, with dimension L ( log K), that embeds the vector representations of the functions. For any finite set (Ji,..., fK} c F, the following holds for any pair fe. fy:
(1 (f f.) S (E(A) E(f)l2 S (1 ()
(7)
Now, define our final DNA mapping by scaling the output of E. The DNA of LLM f is Ti := a E(f). The distance in the DNA space is therefore d, (T()),)))) TB)) = l|T Tú llz [[aE(()) - aEff)llz = a Eff)) E(f,)ll2-
By multiplying the inequality in Equation by our scaling factor a > 0, we get: - ( fj) s a ( F(f))l2 S a(1 + cd,
We can now check the bounds. For the lower bound:
a -) + 02
(1
C2 4
C1 + 02
C2 + ci 02 +
201 =
2
C2
t
2
22 + C1
2
For the upper bound:
+
)
(1 (1+
C2
C1 +
C2 + + C1
202
a(1+c)=
C2
2
C2 +
C + C1
Substituting these results and the definition of d, into our inequality, we obtain:
a du (fr. fz) < d.(T), T1) S (fs,
This explicitly provides the required inequality. Thus, valid DNA mapping exists for any choice of constants c2 C1 0.
16

---

Published as a conference paper at ICLR 2026
A.4 PROOF OF THEOREM 4.2
Lemma 4.2 (Concentration of Empirical Functional Distance). Let J1, E F be two LLM func- tions. Let d; = Ei=1 (1f( f2(2))11 be the squared empirical functional distance cal- culated from sample of t i.i.d. inputs (21....,21y 21). Let d be the true squared Hilbert dis- lance. Assume the squared Euclidean distance between any two oulput vectors is bounded such that (1 [2(x))11 & Cmax for any input and for all functions of interest. Then, for any >> O, the
following bound holds for the sample average: P ( dli 2 $ 2exp
21.
C2
Proof. Let the sample of t imputs, drawn iid. from the distribution be {x1...., It Let Zi...., Zj be a sequence of i.id. random variables, where each variable 2, represents the squared Euclidean distance between the output logit vectors for randomly drawn input
Zy = (f( f2(2;)112
By the assumption, each Zj is bounded within the range [0, Cmax]. The true squared Hilbert distance is, by definition, the expectation of this random variable:
d E[Z,]
The sample mean of these variables is precisely the averaged squared empirical functional distance:
1 + = 2j
-
Σ
Hoeffding's inequality provides bound on the probability that sample mean of bounded, inde- pendent random variables deviates from its expected value. For a set i.id. variables (Yi,..., Y.} bounded in [a, b), the inequality states:
P Σ
- E 2 < 2exp (-
2te²
(- a)3
We apply this inequality to our variables Zj. In our context, the sample mean d... the expectation is dlg, and the range (a, 6] is [0, Cmax], making (b aa)² C Substituting these into the general form directly yields the desired result:
( di 2e
2tc²
2 exp
C2
This completes the proof.
A.5 PROOF OF COROLLARY 3.4
Corollary 3.4 (Construct LLM DNA via Random Projection). For any funite set of LLMs Fk =
(J,,..., JK) C F, the LLM DNA in Theorem 3.3 can be oblained from random linear projection
E: - R with probability at least 2/K2.
Proof. Since LemmalA. C.mmal/A.4]sonn that is Hilbert space, the JL lemma applies. Invoking standard formulation of the lemma (e.g., Theorem 2.1 in Dasgupta & Gupta (2003)) for the set of vector representations of LLMs F& yields desired projection and associated probability bound. O
B EXPERIMENTAL DETAILS
B.1 HYPERPARAMETERS
Unless otherwise specified, adopt the following default configuration. The DNA dimensionality reduced to 128 using Gaussian Random Projection by default. Text embeddings are extracted with the state-of-the-art open-source encoder Qwen www.wwemeeeeeeeeee ing-eddiii---- 8B, using maximum
17

---

Published as a conference paper at ICLR 2026
imput length of 1024 tokens with cls pooling. This model is also used to encode questions in routing experiment for semantic consistency. For text generation, we set max.length, = 1024, cemperature 0.7, top 0.9, if these options applicable. Models with fewer than 7B parameters are run in BF16/FP16 precision without quantization, while larger models are automat- ically quantized to 8-bit for memory efficiency. For instruction-tuned models, the provided chat template is applied and only the responses are used for DNA extraction. All other parameters fol- low HuggingFace defaults, All applicable experiments conducted on five seeds with mean and standard deviation reported.
B.2 ENVIRONMENT
DNA extraction for each model conducted on a single GPU. Models with up to 30B parameters executed on NVIDIA A100 H100 GPUs with 80GB memory, while models between 30B and 70B parameters are deployed on H200 GPUs with 140GB memory. In total, the storage footprint of the 305 models amounts to approximately 20 TB.
C SUPPLEMENTARY EXPERIMENTS
C.I PHYLOGENETIC TREE LLMs
Analogous to phylogenetic research in biology, once the DNA of LLMs is available, we can con- struct a phylogenetic tree to visualize how L.MS evolve. Unlike existing phylogenetic trees based on release histories documentation, our phylogenetic tree is derived directly from model behav- ior. This enables us to uncover latent relationships between models, including close connections that may not have been explicitly documented.
To construct the phylogenetic tree, we adopt the widely used Neighbor-Joining (NJ) method from biology (Saitou & Nei 1987). The core assumption of this method is to find a tree that minimizes the total "evolutionary effort, quantified as the sum edge lengths. The evolutionary root is estimated by the midpoint of the longest path in the tree. We apply this method to LLM DNA by treating each I.LM as a species and using Euclidean distance between DNA vectors as the pairwise distance measure. As an expanded version of Figure 6 the full phylogenetic tree is shown Figure from which several interesting observations emerge:
1. LLMS from the same family are consistently colocated (e.g., Google Flan models, Qwen3 models, Llama 3 models), confirming that the DNA representation captures series-level similar- ity.
2. The DNA-based model tree aligns with the temporal evolution of LLMs, progressing (i) from Gemma to Gemma 3 along root-to-leaf paths and (ii) from early encoder-decoder architectures (e.g.. Flan-T5) to prevalent decoder-only architectures. This indicates that DNA is an effective tool for studying LLM evolution.
C.2 DETAILED ANALYSIS OF FALSELY PREDICTED LLM RELATIONSHIP
To investigate the causes of DNA false positives, we manually examined these cases with notable examples listed in Table We found that a substantial portion involve pairs absent from the Hugging Face model tree yet verifiably related based on public documentation, or belonging to the same that many "false positives" arise
families and thus likely related. The pairs listed in Table from incomplete labeling of the model-tree data rather than from the DNA method itself. Moreover, LLM DNA offers principled way enrich the model tree.
C.3 ABLATION STUDY OF BOTTLENECK EMBEDDING MODEL
This ablation study evaluates the effect of the bottleneck embedding model on the computed DNA. In addition to Qwen- 8B- 4B-Embedding, we compute DNA using two popular rentence-embedding models from different organizations and with different sizes: all-mpnet-base and boe-barreeee en--geeeell..... 5 (0.3 parameters). We then perform a Mantel test a standard statis- tical test for correlating two distance matrices matrices-pairwise across the three DNA variants. Specifically,
(O.IB parameters)
18

---

Published as a conference paper at ICLR 2026
Organizations
gemma---1100 1-66-
genmee-3-00-4
EleutherAl
Qwen
Qwerd--U-- Saaro-----hriiingg2255 7honking2507---PP
Qwen3-4B Thiniinnnn 2007
google
Qwena-4B 3-8
meta-llanm -lama
Qwenn--0
Qwen8-14B
QuQq322
Qward-0.6B
Queer 5-Math-1 58-iintruct
Qwer2.5-Maith M-t.s----------- ---aitt--B--ssttuu
Qwerd 5-78
Qwent 30B A3B insiiinctt-2000 FPB
Qwena-48 insirinsttttoo
Qwen2.5-2 5-D-2 7D-Instruct
Qwen2. 5-38 Instruct
Qwant.5:2 5-7.i..57
Owen2.6 Coden-BB instruct
Quend-7B-inssne
Quend 5-32B-Char
Qwend. Www.5-44B--hhh Ch4B-Chot
I 2-18-Inssnt
Usma-332 2-36-nnstrut
Ulma-33 1-88-insttuct
Metar Lama--a 3-BB-Instruc
Liama-2 2-70b-chatht ht
Jemme-213b--hhtt--
CodeLiama
Codeliama-77 Insttnstuctthi ht
lama-2-7b-chatttt
gamme-7b-i
S6-Imstruct
Qwan2. 5-1 1..5B-nnsttn
Querd 50-5ttrrttt
Qweni.6-4D-C0 5-4D-Chat
Owen1.5-0.55 0.5B-Chat
pyhits2290b
pyttia 12b
Pyttiaa
pythia-1b
pythia 1.40
gol-neo-1.39
gpt--db
gpt-netx-20e
Llmma--76--V
lemma 34b
Meta-Liama-2 Limm--300
LlbmmaGuard-7
Meta Lamaa Guard-2-08
flan-iS base
Figure 7: Phylogenetic tree of representative LLMs constructed using the Neighbor-Joine algo- rithm their DNA embeddings.
sample 10,000 random DNA pairs, compute their distances, and report the Spearman correlation and p-value in Figure g All pairs of embedding model exhibit strong agreement, with high correlations (r 0.75) and strong statistical significance (p 0.0001). These results indicate that the choice of embedding model has marginal impact on the overall DNA structure; even a small (0. embedding model yields comparable DNA.
C.4 ABLATION STUDY OF THE EMBEDDING MODEL'S ' OUTPUT DIMENSION p
This ablation study investigates the effect of the pre-aggregation embedding dimension p from Qwen3-8B- Dowe3-8B-Embeddee Embedding on DNA quality. We vary p E 64, 128, 1024, 4096 and evaluate each setting the same relationship prediction task used in Table The results shown in Figure 9
19

---

Published as a conference paper at ICLR 2026
Table 5: Examples of Undocumented Relationships Identified by LLM DNA
Model
Model
Twen/Qwen2.5-775 Instruct
Qwen/Qwen3-L.77
Qwen/QQwen2.5-7BBnnnntte
Owen/Qwen3-14B
Qwen/Qwen2.5-77B Instruct
Qwen/Qwen3-SB
Qwen/Owen2.5-Codee Instructt
Qwen/Qwen3-4B
Qwcn/Owwe22.5-Coder-7
Qwen/Qwen3-it
Qwen/Qwen3-8B
Owen/Owee2.55-0.5BBmmmm
Oven/OWeen34B--hhnnkee
Qwen/Qwen2.5577B
Qwen/Qwen3-88
Qwenn(Qwen3-4BTTinne
DribiiMLmmdds/tttr-CCC Coaee--Corrr---e Sitder-Qwen--00 3-BBBB
Qwenn/Qwen2.55-ooler77
pridhivMLLmdds//ttar Codeesrrr--t---rrrr--ere Coder-Qwen3-0 Qwen3-0.6B
janhq/Jan-v1-4B
prihiiWll..oods//star-CCC http:///ttaar-re----/eeeeee-- Coder-Qwwe3-00.68
prithvvML.mods//Sarr -
openai/gnt-oss-2000
tiiuacffalcon-12
-raellccoo-----nee b-instruct
ceen/Owen2,5-Codeee
tomg-grouph www.g-roroouuunnnnneee www.uuumdDDDDyarrreeeee
Qwen/Qwen3-Ne
sentence www.aatemmmalllllmnneeeeeeeeeeee
@wen:Qwen] Embedding
BAAlbge-iarge-ee harg----rrr--e....
Ower/Qwen Baaee//eeen-mmbeeteeeeeee Embedddnn--8
RAdihgee large en-v3.5
-1
330233
F-
CECOCC
3.75
F-
000000
2.50
0.8
0.00
3.00
0.3
0.75
0:2
0.25
0.
3.00
5.00
sentanoela traddoreeradd Emaandi B2a0-12
teuckdear)
Figure 8: Mantel test result across three different bottleneck sentence-embeediing models
Across all metrics, performance remains largely unchanged, indicating that p has only a marginal impact. This suggests that DNA quality is not primarily driven by the embedding model's out- put dimension, but by the subsequent aggregation process. also reinforces the observation in Table I that even simple eebeddig models achieve strong relation prediction perfor- mance.
Accuracy
Score
1.5
1000
1325
032
Sentenca-Embeeddigg Model Output Dimerrooss
Sentance-Erreeee Eww.dddbbbbdnggo.... Model Outpse Dimeesion
Sentanca- Conce--nbedddingggggg Hodel Outputt Dimersion
Sendance-EEb Eenooa--bbddiiingg Model Outpot Dimsension
Sanriance Eww.dddnngggggoom Model Outgut Dimeession
Figure 9: Relationship prediction performance (Table D under different output dimensions p of sentence-embeddiigg model. Results averaged over five random seeds; error bars denote standard deviation.
C.5 ABLATION STUDY OF THE DNA DIMENSION
This ablation study examines how the DNA dimension L affects DNA quality. We vary L in 4, 32, 128, 1024, 4096 and evaluate each setting on the same relationship prediction task used in Table I] The results shown in Figure 10 DNA quality improves L increases and converges around = 128. Very low dimensions (e.g.. L 4) underperform because they discard struc- tural information from the original high-dimensional space. Thus, selecting I, involves tradeoff
20

---

Published as a conference paper at ICLR 2026
between quality and efficiency:
larger dimensions preserve structural detail and improve curacy, but incurs increasing downstream computation and storage costs; smaller dimensions are cheaper and faster, but can overly compress the space and lose critical structure, leading to degraded DNA quality. We therefore use I. = 128 in all experiments. Note that this choice is specific to our current evaluation scale of approximately 300 LLMs; for substantially larger model sets, larger may be required.
Accurncy
105
395
Store
335
375
9
ORA
Dimorsiin
Figure 10: Relationship prediction performance (Table under different DNA dimensions L., Re- sults are averaged over five random seeds; error bars denote standard deviation.
C.6 EFFECT OF LLM'S PARAMETER SIZE ON DNA QUALITY
This subsection investigates whether the performance of LLM DNA is biased by model scalc-specifically, whether the method effectiveness varies significantly between small and large models. To rigorously test this, we conduct post-hoc eгror analysis completely separate from the DNA classification pipeline. The DNA classifier first predicts relationships using only functional embeddings, without access to metadata such as parameter counts. After fixing these predictions, we calculate the cumulative parameter size for every test pair (the sum of parameters of both mod- els). We then compute the Point-Biserial correlation (rpb) and fit an independent logistic regression model to determine if this cumulative size can predict the correctness the DNA classifier's s output (1 for correct, 0 for incorrect). Table 6 reports the performance metrics alongside the correlation coefficient and the p-value of the size coefficient derived from the regression analysis.
The results yield two key observations. First, DNA quality is statistically independent of LLM size: across all DNA-based methods, the Miint-Biserial correlations are negligible (ranging from -0.006 to 0.023), and the p-values for the size coefficient substantial (ranging from 0.63 to 0.69), well above the standard significance threshold of 0.05. This indicates that the method's ability to detect evolutionary relationships does not degrade for larger models nor improve for smaller ones. Second, robustness is consistent across different encoders regardless of their own size: the lack of size bias (high p-value) remains constant across the large Qwen3 (8B) encoder and the much smaller BGE (O.3B) and MPNet (B) encoders. This suggests that LLM DNA captures intrinsic functional behaviors that persist across model scales, rather than relying on complexity metrics associated with parameter count.
Table 6: Sint--Bieerial correlation (rpz) and statistical significance (p-value) between LLM size and relation prediction accuracy
Method
Accuracy
F1
AUC
Corr. (rys)
p-value (Size)
PhyloLM
0.742 +0.058
0.766 #0.057
0.788
+0.060
-0.097 #0.195
0.5202
DNA (Qwen3)
0.919 +0. +0.048
0.925
+0.047
0.979 +0.027
0.025 10.123
0.6349
DNA (BGE)
0.934 +0.053
0.940 40.050
0.992 40.023
0.013 10.125
0.6550
DNA (MPNet)
0,932 +0.048
0.937 20.047
0.990 40.020
0.023 #0.122
0.6903
C.7 RELATION PREDICTION FOR LLMs WITH SAME TOKENIZER
Though our sfate-of-theaar baseline PhyloLM Yax et al. 2025) is designed to compare general LLMs across both same- and different-tokeenzerr settings, introduces different comparison strate- gies for these two cases. In particular, when models share tokenizer, PhyloLM can directly leverage exact token distributions, which typically yields stronger performance than its cross-tor tokenizer vari- ant based on the first four characters (as shown in Table Ip. To provide fair and transparent com-
21

---

Published as a conference paper at ICLR 2026
parison under PhyloLM's same-tokenizer regime, further compare PhyloLM with LLM DNA on a single-tokenizer subset. This setting is more challenging because models with the same tok- enizer are typically closer in function and thus harder to distinguish. Concretely, filter the dataset to include only models using QwenZTookenizer, the most commonly adopted tokenizer among our evaluated L.MMs, and perform the same relation prediction task. The results reported in Table
Comparing Table 7 with Table yiields several key findings. First, all methods except PhyloLM exhibit a clear performance drop in the same-tokenizer setting: for example, DNA (MPNet) decreases from 0.93 to 0.86 in accuracy, and Greedy drops from 0.59 to 0.42. In contrast, Phy- loLM slightly improves its accuracy (0.742 to 0.755) and achieves notable precision gain (0.74 to 0.83), confirming that the sume-tokenizer case is indeed more challenging while PhyloLM remains robust. Second, despite the reduced gap, P still consistently outperformed by DNA. This may be because PhyloL. primarily evaluates the distribution of the first token, which may be insufficient to capture long-range sentence-level distributions that are better reflected in DNA. We note that this comparison used Qwen2Tokenizer to align the two methods instead of using the first 4 characters which is technically a slightly different version of PhyloLM We also emphasize that the comparison is conducted on relation prediction, task native to LLM DNA rather than the primary focus of PhyloLM. This does not diminish PhyloLMMss demonstrated effectiveness in its intended applications, such benchmark performance prediction. In future work, enable fairer and more comprehensive comparison, we will evaluate both methods on the datasets and tasks originally used to develop PhyloLM, such as model performance prediction benchmarks. Furthermore, we will investigate more deeply how sentence-level, long-range patterns affect LLM representations.
Table 7: DNA relation pediction performance for LLMs with Qwen2Tokenizer
Method
Accuracy
Precision
Recall
F1
AUC
Random
0.511 #0.054
0.682 =0.073
0.529 =0.054
0.593 =0.047
0.501 #0.068
Greedy
0.428 #0.043
0.687
101
0.294 #0.000
0.410 #0.018
0.500
#0.049
PhyloLM SVM
0.755 13.
0.832 #0.080
0.800 20.204
0.801 10.149
0.760 180
DNA (Qwen3)+ + SVM
0.834 +0.092
0.863
#0.073
0.907 #0.123
0.878 #0.072
0.926 105
DNA (BGE) + SVM
0.869 #0.078
0.865 +0.069
0.966 #0.094
0.909 #0.059
0.944
+0.087
DNA (MPNet)+ SVM
0.866 #0.088
0.879 +0.099
0.948 #0.098
0.906 =0.065
0.960 #0.079
D BROADER IMPACT
The utility of the phylogenetic tree extends beyond historical visualization, offering actionable in- sights into three key aspects of model management.
License Governance. As the number of models grows rapidly, non-compliance with model li- censes is becoming a critical concern. Such behaviors include fine-tuning or distillation from a model without giving proper credit or respecting license terms. For example, it has been alleged that certain proprietary models are derived from open open-source counterparts like Qwen-2.5-14B often difficult to verify through metadata alone. While existing license-analysis tools such as Mod- elGo primarily rely on explicit claims, our phylogenetic analysis provides a functional indication of lineage. Similar methods have been explored in other domains; for instance, RAI2 (Dong et al.) 2023) develops identification techniques for Computer Vision (CV) models to accurately identify model provenance and detect unauthorized reuse. In the language domain, our tree clusters mod- els like Microsoft's orca-2-133 and LMSYS's vicuna-7b-v1.1 tightly, identifying a shared lineage that is not explicitly documented in Hugging Face. Ultimately, the phylogenetic tree offers data- driven framework for flagging potential license violations and strengthening governance across the AI ecosystem.
claim
Model Selection. Our framework facilitates informed model selection by analyzing the interplay between functional inheritance and iteration velocity. First, the phylogenetic lineage allows users to identify families where specialized capabilities such reasoning patterns (Huang & Chang 2023) or domain knowledge (Wu et al. 2025)- are likely transferred down from ancestor models, ensuring the retention of desired baseline traits. Second, the "evolutionary rate" (branch length) serves
22

---

Published as a conference paper at ICLR 2026
proxy for Iteration Velocity, quantifying the magnitude of functional change between versions. This reveals a fundamental trade-off between novelty and stability: while a high iteration velocity offers significant behavioral shifts advantageous for experimental use cases, models with lower veloe- ity (higher backward compatibility) are often preferable for production anvirommnntsppariiculle multi-agent systems to minimize the risk of breaking established agent workflows prompt de- pendencies.
Risk Mitigation. Finally, the phylogenetic tree facilitates proactive risk mitigation by tracing the propagation of safety risks. Our findings on inheritance suggest that vulnerabilitie--uude planted backdoors (Yang et al. 2024). jailbreak susceptibility (Russinovich et al.] [2025], or decep- probably transferred down model lineages. Consequently, the
im-
tion behaviors (Wu et al. 2026 tree allows auditors to propagate known risk profiles from ancestor models to their descendants. If parent model is flagged as unsafe, its descendants identified by the tree can be immediately prioritized for safety interventions. This enables auditors to mitigate deployment risks efficiently, focusing resources on high-risk lineages without requiring exhaustive ground-up testing for every new model iteration.
E LIMITATIONS
This section discusses the limitations of our work.
Traits and DNA subsequences. Our formal definition of LLM DNA does not assign mathemat- ical meaning to DNA subsequences. However, in Section 5.7 we observe that fine-tuning alters only specific subsequences while leaving the overall pattern largely intact. This suggests that certain DNA subsequences may encode distinct model traits (e.g., mathematical reasoning). A theoretical characterization of how different evolutions affect such traits an important and challenging direc- tion for future work. In the long run, this could enable more precise understanding, management, and even targeted modification of LLM DNA, drawing a closer parallel to how biogenetics studies and manipulates inherited properties.
daptive attacks on DNA. Our current DNA extraction is not designed to be robust against adap- tive attacks. If an attacker knows the extraction data, they could deliberately train a model toward malicious objective-e.g. e.g., producing a substantially different DNA while functionally copying an- other model-in order evade lineage detection, patents, or license constraints. straightforward mitigation is to keep extraction datasets closed-source or to re-extract DNA using a fresh dataset when needed. A more principled direction is to scale the extraction set so that overfitting to be- comes prohibitively expensive and would noticeably degrade general performance. Developing such security guarantees remains a key topic for future work.
Data bias and contamination. Our DNA extraction relies six public datasets, which may in- troduce evaluation bias or contamination. Bias could cause us to overlook certain differences or overestimate similarity among some model groups, while contamination could arise if parts of the extraction data were seen during pretraining. Nonetheless, their impact on our evaluation is limited. As shown in Section 5.5 DNA remains effective even when evaluated on purely random input data. This observation supports that DNA primarily captures relative relationships among LLMs, rather than depending on absolute accuracy on any particular dataset. One remaining challenge is that cer- tain model APIs, such as the Claude series, may reject outputs generated from random strings; these cases require more advanced randomized methods to bypass such safety or formatting barriers.
F MODEL DETAILS
In this section, we present the full list of models used, including their architectures, parameter counts, and licenses. This paper focuses on text-generation models, including decoder-only models (e.g., GPT) and encoder-decoder models (e.g.. T5). For completion models, the completed text is treated as the response; for instruction-tunat models, the entire response extracted from the template is used. This may include any mener-ggneertedd reasoning traces (c.g.. in Qwen3), which are also considered part of the response. All uses comply with the respective model licenses.
23

---

Published as a conference paper at ICLR 2026
G LARGE LANGUAGE MODEL USAGE
Large language models assisted in verifying theoretical results, implementing source code, and pol- ishing this paper. The theoretical results and proofs were drafted by humans and verified by Gemini- 2.5-Pro. Code implementation was primarily assisted by Claude-4-Sonnet and GPT-5-Codex, and the code's correctness and alignment with the paper were checked by Gemini-2.5 2.5-Pro.. Manuscript polishing was assisted by Gemini-2.5- Pro and GPT-5.
24

---

Published as a conference paper at ICLR 2026
Table 8: Full list of used 305 models, including their architectures, parameter counts, and licenses
Architecture
Parameters
License
01-ai/Yi-1.5 L5534B--C.
Decoder Only
34B
Apache-2.0
Decoder Only
Apache-2.0
01-8W/Y1-44B-hatt
Deceder Only
34B
Apache-2.0
Devoder Only
Apache-2.0
0llaa/Yi-6fn-2000
Decoder Only
6B
Apache 2.0
01-ai/Yi6BB-Chal
Decoder Only
Apache-2.0
01-ai/Yi 9B 200K
Decoder Only
9B
Apache 2.0
AdaptI.LMM S.aapt..MM/fnaane-hhaa
Decoder Only
7B
Llama-2
MddppLLLM/mcdiiine-LLL
Decoder Only
Unknown
AdaptLLM M/niLL//iiiiiii-LMMMMM
Deceder Only
13B
Apache-2.0
AdaptLLM/mmediicin
Dewoder Only
Llama-2
Aurore- Deveill/KoonSSmll-7..
Decoder Only
8B
MIT
BioMistal/BieMMsrral-8
Decoder Only
Apache-2.0
Decoder Only
7B
Apache-2.0
Blominier-AUUNNlMA-Neeaa
Decoder Only
7B
MIT
Only
MIT
Causal M/3-alLM/33-bbbue
Devoder Only
34B
GPL-3.0
Dewoder Only
Apache
Decoder Only
9B
MIT
ConvssA//Lmminex334B-00000
Decoder Only
34B
Other
Convex AI/ 2mminxx344B-x022
Decoder Only
34B
Other
Cortical wttpaa/wwwkkiiiiboooooooooo-oooo
Decoder Only
7B
Apache-2.0
Cultri//Ncurallisssbfft
Only
Apache-2.0
DularuuAA/DaaruusS R1-14114- preview
Devoder Only
155
Apuche 2.0
DavidAt//Qwee&HHoenrr Inserre--eette-t-eooeed..l Uncensored
Decoder Only
Apache-2.0
DepHa/DXepHas-V1-1B
Decoder Only
8B
Apache-2.0
HleutheAA/gpt---bb
Decoder Only
Apache-2.0
EleaberAAiightnne--..3
Devoder Only
MIT
HleuherrAiigtt-nooo-200
Deceder Only
21B
Apacho-2.0
HlcuberAA/llemmmmBBBb
Omly
Llama-2
HeutherA//llemma 7b
Decoder Only
7B
Llama-2 2
BleuherAA/polyglootkor
Decoder Only
3.8B
Apache-2.0
BecuherAUipythia-I..
Decoder Only
2B
Apache-2.0
HeutherAUlpythia-12
Decoder Only
12B
Apache-2.0
BleutherAUiyythia.
Decoder Only
Apache 2.0
HccutherAliyythaa-2..6
Deceder Only
3B
Apache-2.0
EleutherAAipythiia
Dewdder Only
Apuche-
FallenMeric//MN
Decoder Only
12B
CC-BY-4.0
FelisChao/llmma2-BBbnnnii
Decoder Only
13B
Unknown
Decoder Only
7B
Apache-2.0
FelisChaolvicumma755 phhysssss
Decoder Only
7B
Unknown
Decoder Only
24B
Unknown
Decoder Only
7B
Unknown
HuugiingFacell4/eeepyyr
Devoder Only
HogginggPcc/TBBSmmilllM 135M
Decoder Only
0.1B
Apache-2.0
HuggingFace 1eeeingFPss///SSmoLLMM--1-133
Decoder Only
0.1B
Apacho-2.0
HuggingFaweTB//Smol...M
Decoder Only
3B
Apache-2.0
Intel/neural chail 7b-v3.3
Decoder Only
7B
Apuche 2.0
<-inelliggemce/Midd--2 Mww.-aaa---------nnm---e
Decoder Only
12B
MIT
KoboldAL/GPP 2 LN
Devoder Only
2.7B
Unknown
Only
LGAALEXXONEEXXXON--0O000
Decoder Only
IB
Other
LLM360/Amber
Decoder Only
Apacho-2.0
LatituddCames/VWa 2are--2-222
Decoder Only
12B
Apuche-2.0
Leol.M/leo hessianii 13h
Decoder Only
13B
Unknown
LiquidAM.FM22.
Decoder Only
Other
AquidAA/LFM2233500
Decoder Only
0.4B
Other
Mariyarfamahh/Wizarll
Decoder Only
AGPL-3.0
Ncko-Institutt-of- Instituts-of-SSiccee//iee Scinncc/nechhhtarrs-
Decoder Only
7B
Unknown
Decoder Only
715
Unknown
Nexuslow//Surtingg L.-7B
Decoder Only
7B
Apache 2.0
Nineday
Decoder Only
2.7B
Unknown
NousResaacc/Hesms-44111
Decoder Only
425k
Apacho-2.0
VousResentc/leeme--444.....11
Decoder Only
15B
Apache-2.0
NousResoach/NoosHHHHess--BB
Decoder Only
13B
gpl
Decoder Only
11B
Apache-2.0
Nouskeveentt/Vaas-lleeeee------
Decoder Only
341
Apuche-2.0
Decoder Only
24B
Unknown
Decoder Only
128
Apache-2.0
Only
34B
Unknown
Decoder Only
7B
Apache-2.0
Plaban8
Decoder Only
24B
Apacho-2.0
PocketDoo/Dans-tP Pow.unssssPssosatttyllllaa V.3.0-244
Decoder Only
24B
Apache-2.0
PygmalionAlpygmaale
Dewder Only
2.7
creativent creativeml-openniilll
QuixiA//Wizardl M.13B. Uneensoedd
Decoder Only
13B
Other
25

---

Published as a conference paper at ICLR 2026
Table 8
Full list of used 305 models (continued)
Model
Architecture
Parameters
License
QwenrQwQ-32B
Decoder Only
33B
Apache-2.0
QweniQwenl 1-.00.B-CH Chat
Devoder Only
0.68
Other
QweniQwenl 14B-Chat
Decoder Only
14B
Other
QweniQwent 5-32B-Chat
Decoder Only
33B
Other
Qwen/Qwenl 5-CB-Oa 5-4B-Chat
Only
Other
OwentOwenl_S- 1-n1--5----hha
Devoder Only
8B
Other
Qwen/Qwen2
Dewoder Only
Apache-222
QweniQwen2-7B. 1B-Isttttt
Decoder Only
8B
Apache-2.0
Qwen/Qwen2.5 0.5B Instruct
Decoder Only
0.51
Apuche 2.0
Qwen|Qwen2.5. 2.5.1...........
Decoder Only
2B
Apache-2.0
Decoder Only
2B
Apache-2.0
Owen/Owen2.5336
Only
QwentOwen2255-77
Devoder Only
8B
Apuche 2.0
Qwen/Qwen2.55
Desoder Only
Apache-2.0
Sween/Owe225-Codee--.
Decoder Only
8B
Apache-2.0
Qwen/Qwen2.5 2annn255-Cneer-1 7B--nstetttt
Decoder Only
Apuche 2.0
Qwen/Qwen2.5-Maine 1.5--Mat--..1---ssee 1.5tnteettt
Decoder Only
Apache-2.0
Decoder Only
8B
Apache-2.0
Owen/Owen2-0.68
Omly
Apache-2.0
@wen/Owen3-113 1.7B
Devoder Only
2B
Apuche 2.0
Qwen/Owen3-14p
Decoder Only
15B
Apache-2.0
Decoder Only
31B
Apache-2.0
Qwen|Qwen3-4B
Decoder Only
4B
Apache-2.0
VwennQ0een2-4B-lnsstreee22
Decoder Only
Apache-2.0
SeenKQOwen--B-Tiinkiee
Deceder Only
4B
Apache-2.0
Owen/Owen3-4BB
Devoder Omly
Apuche
Qwen(Qwen2-88
Decoder Only
8R
Apache-2.0
S4nfs/Nceto-1
Desoder Only
CC-BY-NC-4.0
SUSTecWSSUS-Cha:34BB
Decoder Only
34B
Apache-2.0
SciPhilSciPh-MMisrall----22
Decoder Only
7B
MIT
Only
MIT
JestaccUUGGENN--4BB-4777
Deceder Only
4B
Apache-2.0
Dewoder Only
Apache
whatsopp
Encoder Only
Unknown
Unknown
TheBBokkCCoooLLaaa-000-----
Decoder Only
69B
Llama-2
ThoBBokk/koobb-338--Ht
Decoder Only
13B
Other
TheBlokchula-30B--pp66
Decoder Only
30B
Other
Only
14B
Unknown
Devoder Only
0.18
Apuche-2.0
Taminnd65MMemalBBARRTT
TigocRescarchviigrroo-- 13b-basc
Decoder Only
13B
Apache-2.0
Thay Maaaaa/aaaalaaaaa L.w.aiiiaLaaaaBBBB---- 1.--hat-.1-
Decoder Only
Apacho-2.0
MnyLLam/TTiaLylunn I. 1B-Chat-11.0
Decoder Only
Apache-2.0
TinyI TenyTTinylllal Llamaaaaaa.-..
intermediate step
Decoder Only
Apuche 2.0
TinyLlama coan//iiyllaa Llama_v1.1
Only
Apache-2.0
LalLLalaTTTyLLaamnig v. wtt.maawwl..mmmh..omm code
Devoder Only
1B
Apache 2.0
Vihhroodels/iikhheNeeee I --220099---2100-24
Decoder Only
Apache-2.0
VisionCAARRvvies
Decoder Only
Unknown
Unknown
Sitex//Luna-Neeonn
Decoder Only
12B
Unknown
/ Shalow- 12B
Decoder Only
12B
Unknown
WhiteR WhiiRRahhiNNewWWiiiRdhhiii 13B.vl
Decoder Only
13B
IJama-2
WizaniLM/WizrnLL
Devoder Only
Llama-2
(ncride/(wee2.5.77. Instruct-R RI forffnaacee
Decoder Only
8B
Apache-24 2.0
adamo ha0
Decoder Only
Apache-2.0
Decoder Only
15B
MIT
Decoder Only
7B
Apuche-2.0
allenai/tlu 70b
Decoder Only
69B
Other
squif-niaaquif-3.5..m
Decoder Only
8B
Apache-2.0
ask Jug.haskgghuueemmmmhhheeeeennne
Only
augmxnttkhisa -b-e---bvvvl
Decoder Only
8B
Apache-2.0
Decoder Only
CC-BY-4.0
Serkeley- MaM-aaliiinn-----------a LM-7B-alpha
Decoder Only
7B
Apache-2.0
beruniy/LLaama- 3.2. 3B- Instruct-
Devoder Only
laam3322
heyoni/
Decoder Only
MIT
bigcodeeoottocoerr
Decodar Only
16B
OpenRAIL-M
higuollektancode22..
Decoder Only
OpenRAIL-M
Mipcciencebblemm555i
Decoder Only
0.6B
BigScience RAIL 1.0
bipscience/llomm-77
Decoder Only
BigScience RAIL 1.0
bigscience/bloomzzlbb
Decoder Only
BigScience RAIL 1.0
higscience/bloxmm.... 1b7
Deeder Only
2B
BigScienze RAIL 1.0
hipsiincee/leamm-55000
Decodar Only
0.6B
BigScience RAII. 1.0
bigscianse/bleome-70
Decodar Only
7B
BigSeicnce RAIL 1.0
In-Insttett
Only
chatitcloud/UZ21
Decoder Only
0.38
Apacho-2.0
cisco-
cisco-al/mini-butt-g..
Encoder Decoder
Unknown
Apache-2.0
Dewoder Only
19B
CC-4Y-NC-49
wwweaeeeCeeeeeeeeeeeeeeee Seck-33B
Decoder Only
33B
Other
codellama/Codel 3b-Insttetttttt
Decoder Only
13B
Llama-2
26

---

Published as a conference paper at ICLR 2026
Table 8
Full list of used 305 models (continued)
Model
Architecture
Parameters
License
codellama/Code LodelaaaaCCCLLaaaaa 3-b-instruct-hff
Decoder Only
34B
Llama-2
Coallaann/CCollaaaaa 7b-hi
Devoder Only
7B
Llama-2
culeparniicnlepamme. small code-to-text
Encader Only
Unknown
Apache-2.0
Encoder Only
Unknown
Apache-2.0
continucdelinssiiett
Only
Apache-2.0
Devoder Only
24B
Apuche-2.0
dutabrickw/ūolll-.u
Dewoder Only
CC. NC-BY-NC-4400
dinahiicksäooly-4222222
Decoder Only
12B
MIT
dalahridkw4äallly.22.533
Decoder Only
MIT
datahrickw/lally-v2-71
Decoder Only
7R
MIT
degsickkal/DDeepssst-oooo V2-Loeee----------- In-sitt-Inttruct
Decoder Only
16B
Other
deppaeckai/DeeppSeettB 00528-Qeen3-88
Only
MIT
leepseek RI Distill Llama 8B
Devoder Only
8B
MIT
deppoee--ai/DeeSSee...
Owen-1.5BB
Desoder Only
MIT
Decoder Only
8B
MIT
aieeepSeea V2eLite
Decoder Only
16B
Other
deppeck-aiideepssekkoooee 3.3bbase
Decoder Only
Other
Decoder Only
7B
Other
deepseek- aideepeee 67b-chat
Devoder Omly
deepseek. aiddrepsekk Ilm 7b hase
Devoder Only
7B
Other
leessek-aiddeeccek-llee
Decoder Only
Decoder Only
7B
Other
dicta -iddiiiiiiaamm2.
Decoder Only
7B
Apache-2.0
Decoder Only
24B
Apacho-2.0
coi-io/climaateegt-77
Deceder Only
7B
Other
jayxa/ELYYAAjjaaaeeseee Liama-2-7b
Devoder Omly
Llama-2
Mlya/EELY7AAjjpaaeee
Decoder Only
7B
Llama-2 2
clyzalLJoma3--FLLZZA-PPP
Decoder Only
llama3
Decoder Only
7B
CC-BY-NC-4.0
facchookigalactico-1.33
Decoder Only
CC-BY-NC-400
Decoder Only
34B
Apacho-2.0
Juealy//HuentyyOwe31-17777
Deceder Only
28
Apache-2.0
futurebouse/elhert)
Devoder Only
Apache
golaxy/gowizardlm
Decoder Only
Unknown
Apache-2.0
google-t5h5-lage
Encoder Decoder
0.7B
Apache-2.0
Encoder Decoder
60.5M
Apache-2.0
google/fan-tS-beee
Encoder Decoder
0.2B
Apache-2.0
Decoder
0.8B
Apache-2.0
google/ gemma 2b-it
Unknown
3B
Gemaaa
google! petele/ggmmm....
12B
Poglle/gemm--333
Unknown
IB
Gemma
google/gemma-3334 3emma-3-4b-it
Unknown
Gemma
google/ aaagll/gemm--aae 7b-it
Unknown
9B
Gemmna
gradientai/ lama Instruct- 2622
Decoder Only
8B
laam3
heegyu/kogptt--kus.
Decoder Only
huggyllan huggyllama/llama. 7b
Devoder Only
7B
Other
huihni-ai/Huihhui Jan-
Decoder Only
Apache-2.0
bbvibivallpccdddogoo--22
Decoder Only
72B
Other
Decoder Only
12B
Unknown
janhuglan-v11448
Decoder Only
4B
Apache 2.0
janhg/lanvvl-edge
Decoder Only
2R
Apache-2.0
jiawei weerduuu///wwee2.555511 O.ee
Only
jxm/gnt -0s- hase
Unknown
20B
Unknown
kevin009/llamaR
Decoder Only
Apache-2.0
Saifcyyunu(Cyoonavv44..
Decoder Only
24B
Apacho-2.0
kyujinpy/Sakura SOLLaaaa--SLCAAAAA
Decoder Only
11B
CC-BY-NC-SA-4.U
Igaalves/gpt
Decoder Only
0.
MIT
liyuesen/dmugpt
Encoder Only
Unknown
GPL-3.0
Imsys/vicuma. 1.3.vv1.55
Devoder Only
13B
Llama.2
Imkys/vicuma-333...
Decoder Only
33B
Unknown
Jmsys/vioua-77-vi.1
Decoder Only
Unknown
Imsys/vicua-7b-v1.55
Decoder Only
7B
Llama-2
Devoder Only
7B
Llama-2
m--a-piChhrMMsscciann
Decoder Only
7R
MIT
moce-laaaa/Cokellaae-- 1a-
Unknown
13B
Llama-2
ht..smmmaaa-..nssct.tttttt Insttrcuct
Unknown
Sean-Ilama/.lllmaa-332 1-taa-22-3b-hht-hhh 3b-chat-hf
Unknown
13B
Llama-2
Nutta-lamaa/Laaaa--700---
Unknown
69B
Llama-2
meta bi
Unknowa
7B
Llama-2
meta. Ilama/l. 2.
Unknown
7B
Llama
metn-llama/1 Jama-3. I-st-nstt
Unknown
8R
1amma.11
mota-Ilama/LLanaae 2tt.:LL.......---mmmsse 2-1-Innrruct
Unknown
1B
laama3 2
mnco-laama/Llamaiaaadd Lla.aaGaraaaa
Unknown
7B
Llama-2
meta-llama/Me Mrtt-Maaaa--a-a---
Unknown
71B
llama3
Ilama/Meeia Llama Instreet
Unknown
71B
laaa3
metn. meta-Ilama/Mer Ilaaa/MMaaeaa lamm-3-88
Unknown
8B
laaa3
Unknown
llama3
27

---

Published as a conference paper at ICLR 2026
Table 8
Full list of used 305 models (continued)
Model
Architecture
Parameters
License
metn-
Maae-llla//Mee----- L Jammn CBaBdd2-8B
Unknown
8B
Ilama3
Aesta-math/MettMaaa Mct.aMaa--MMLena------ Llemma-77
Decoder Only
Apacho-2.0
meta-math/MeiMMaahh Mistaall-7
Decoder Only
7B
Apache-2.0
microofl/DiaaoCPP mediumm
Encoder Only
Unknown
MIT
micmosoftMMdiiP Phi-Insttiss
Decoder Only
4R
MIT
microsoff/Orca-22 2-1b--
Devoder Only
1.3
Other
micrssoft/Oraa-2276
Desoder Only
Decoder Only
4B
MIT
microosoft/Phi---mmiiii tilrooott/hhi---uutl---ntttttttt
Decoder Only
MIT
Prrtroof//Phi-3.-mmii-
Decoder Only
4B
MIT
microooft/Pii instruct
Decoder Only
4B
MIT
microsoff/Phi-4mmiiiieeess
Decoder Only
MIT
microsoff/Dinozel..-----6
Decoder Only
0.8B
MIT
microsoft/phi-1
Decoder Only
MIT
microsoft/phi-1.5
Decoder Only
1B
MIT
microsoft/hhi-2
Decoder Only
3B
MIT
microsoftphii-4
Decoder Only
15B
MIT
mimmind
Decoder Only
14B
Apache-2.0
mistolaiiMiiitall-B--nntcccee
Unknown
Other
Unknown
7B
Apache-2.0
Unknown
Apache-2.0
mistrala/Miittal-k337BB
Unknowa
47B
Apache-2.0
mlabonne/AAphaaMmmrchhh
Decoder Only
7B
CC-BY-NC-4400
mlx community//Aperuus 8B Instruct-2099 hf16
Decoder Only
8B
Apache-2.0
mookiezi/Discond Micac-8B-Previsee
Decoder Only
8B
Ilama3
Only
llaaa3
mosaicml/mpt 30b-chat
Decoder Only
30B
CC-BY-NC-SA-AUU
mosaieml/mppt- 30b-instruct
Decoder Only
30B
Apacho-2.0
mosaicml/mpt: Saappt------oooy-- 7b-ttorywieer
Decoder Only
7B
Apache-2.0
mkoodame/BBau Orchid 2x7b
Decoder Only
13B
Apuche 2.0
nomic-niggptaall- 33b-snorry
Decoder Only
13B
gpl
Decoder Only
2B
CC-BY-4.0
gpant-communityyyquuaagggg
Only
0.18
MIT
upenai/pt--ss- 20b
Devoder Only
22B
Apuche-2.0
opeabn/UUnraMM-130
Decoder Only
13B
MIT
openchat/openchaa-.. 3.5-01006
Decoder Only
7B
Apache-2.0
openchatkopenchal....
Decoder Only
7B
Apuche
ogentn-reseearccvooenn llama.
Decoder Only
13B
Apache-2.0
pecati-cccauttlogeelllama...
Decoder Only
7B
Apache-2.0
prithiv
Decoder Only
0.68
Apache-2.0
project buize/baize
Devoder Only
13B
CC: BC-BY.N. BC NC.440
quantumaiko/Korean
Decoder Only
7B
Unknown
mshiraj/CasPPT
Decoder Only
7B
Apache-22 2.0
monenekdmmTinnSSooieee IM
Decoder Only
IM
Unknown
sail/Sailor-7B
Decoder Only
Apache-2.0
sarvanat/saraa--
Deceder Only
3B
Unknown
scb10x/ly
Dewoder Only
Apache-2.0
segolilylahw/t.i
Cyherrycurity---- v0.2 v0.2
Decoder Only
7B
Apache 2.0
soquelbox/gpt-oss--20 DES-Reasoning
Decoder Only
21B
Apache-2.0
staailityaistablllm. tuned alpha-7h
Decoder Only
7B
CCCBY-NC NV-NC-AA-.00
Encoder Only
Unknown
BigScience RAII. 1.0
HaccentHunyuan-11 8B-lnstruct
Only
Unknown
uncent/Hunyuan-7BB 7B-Instruct
Deceder Only
8B
Unknown
theprinl/ TTam Qwen2.5.0.BB
Dewoder Only
Apache
thesephist/contr- http:///ccoootaaattteeee..e www.:heenecc----aa----i-iii
Fncoder Decoder
Unknown
MIT
diuuacFalconn- 100-o033---BB-ttteeee
Decoder Only
10B
dinue/Falcon33. B Instmuct
Decoder Only
2B
Other
tiinac/Faccon333---nnn
Decoder Only
3B
Other
diuuec/alccon3-7B-Bmne
Only
tiuue//alcon- 1IB
Devoder Only
11B
Unknown
diumeffalcon.
Decoder Only
Apache
Decoder Only
7B
Apache-2.0
tomg-gooup-ummd//DynaCCuuaaa
Decoder Only
Apacho-2.0
Decoder Only
Apache-2.0
(omg-group-uund//ymannuu
Decodar Only
8B
Apacho-2.0
unslotl/gpt-oss-2000
Only
Apache-2.0
upstage/SOL.AR 10.7B Instruct-v1.0
Devoder Only
11B
CC NC.40.4
upstage/SOI.As
Decoder Only
Apache-2.0
Minivinalllmma-h-chhat
Decoder Only
7B
Llama-2
wwwe-orddissond/sllyy--002
Decoder Only
12B
Apache-2.0
yann:ellggExperriee226-77
Decoder Only
Apache-2.0
Zal-ory(LLM+--22-0511
Deceder Only
33B
MIT
zai-orgiglm-4-9b-dae
Only
Zhenge/MixTAO Mox
MofE.v.
Decoder Only
13B
Apache 2.0
28


Reviews:
[{"review_id": "6WkK4QT2Qh", "reviewer": "Reviewer_5DVv", "summary": "The paper proposes LLM DNA, a mathematically grounded, low-dimensional representation intended to capture an LLM’s functional behavior. The approach: (i) define DNA via a bi-Lipschitz map over a space of LLM functions, (ii) justify existence with a Johnson–Lindenstrauss (JL) argument, and (iii) provide a training-free pipeline. Empirically, DNAs cluster related models and support simple routing.", "questions": "- How can the trees be useful in practice?\n- Since there is only a small performance difference, are there other benefits to using this method over others?\n- How do results change with different sentence-embedding models and embedding dimensionalities?\n- What is the empirical trade-off between projection dimension d, distortion of pairwise distances, and downstream tasks (lineage/routing)?", "limitations": "", "rating": 6, "confidence": 2, "soundness": 3, "presentation": 3, "contribution": 3, "strengths": "- The bi-Lipschitz view + JL existence yields a clean, theory-backed rationale rather than an ad-hoc embedding and is well principled.\n- The training-free pipeline is straightforward, practical, and appears to work across heterogeneous and API-only models.\n- The high-level theory (bi-Lipschitz definition, JL-style existence, and concentration arguments) appears reasonable for finite settings.\n- The method is agnostic to architecture and tokenizers, and doesn't require rebuilding an embedding space when adding LLMs to the evaluation set. This means it can be extended to the closed-source LLMs and can be used to uncover undocumented relationships between models (p7-8, Appendix C.2), which is an advantage over EmbedLLM.", "weaknesses": "- \"DNA\" quality likely hinges on the chosen sentence-embedding backbone and its dimensionality. Robustness to encoder swaps (and multilingual/domain shifts) is not yet demonstrated.\n- There is an insignificant difference in performance for the model routing test between the standard baselines.\n- The motivation for the benefits that this type of representation unlocks (namely, phylogenetic analysis on models) is not sufficient. The authors show and discuss a phylogenetic tree, but discuss the genetic evolutionary rate without drawing the connection to its significance in this context and seem to be borrowing from authority of biological genetic evolutionary analysis.\n- The mathematical guarantees are for finite sets and bounded inputs; it’s unclear how distortion and stability behave as input lengths or the set of models grow. Please discuss the unbounded-input case and evolving model pools.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "hHKiPvl7XW", "reviewer": "Reviewer_Jnw8", "summary": "This paper introduces LLM DNA, a formal framework to identify and trace the evolutionary relationships of large language models (LLMs). The authors address the challenge that with millions of LLMs existing, their origins through fine-tuning, distillation, or adaptation are often unclear.", "questions": "-the paper sets the final DNA dimension $L=128$. How was this number chosen? How sensitive is the DNA's quality to this choice? If $L=32$, would the system lose the resolution to distinguish highly similar models (e.g., Llama-3-8B vs. Llama-3-8B-Instruct)? \n\n-How sensitive is the method to minor fine-tuning? If I fine-tune Llama 3 on just 1,000 new samples, how much will its DNA $\\tau_{f_2}$ change from the original $\\tau_{f_1}$? Is the DNA distance $d_\\tau$ predictably correlated with the amount or domain of fine-tuning data?\n\n-The study includes models from 100M to 70B parameters. Is the method equally effective across this entire range? Does using a powerful embedding model $\\phi$ on the (often lower-quality) outputs of a 100M model \"over-beautify\" its functional representation, making its DNA seem more capable than it is?", "limitations": "", "rating": 6, "confidence": 3, "soundness": 3, "presentation": 3, "contribution": 3, "strengths": "-one of the paper's  strengths is its rmathematical grounding. It doesn't just propose a heuristic; it formally defines LLM DNA using the Bi-Lipschitz condition and uses the Johnson-Lindenstrauss (JL) Lemma to prove its existence.\n\n-Unlike methods that require retraining to compare models, the LLM DNA extraction pipeline is fixed. Once the random projection matrix $A$ and prompt set $\\mathcal{S}_t$ are defined, any new model can have its DNA extracted independently4. This makes the system highly scalable and durable.\n\n-The method does not rely on model internals like architecture, weights, or tokenizers. Since it only evaluates text outputs , it works on black-box API models (like GPT-4) just as easily as open-source models, and it can compare models of completely different types", "weaknesses": "-While the Mantel test shows stability for general datasets, the DNA is still a snapshot of the model's behavior on only $t=600$ prompts. If this prompt set were narrowly-scoped (e.g., all prompts were Python coding problems), the resulting DNA would likely represent a \"coding DNA\" rather than the model's general functional behavior.\n\n-.The DNA measures functional similarity (i.e., \"do these models behave alike?\"). If two models from different companies were trained independently but happened to achieve a very similar functional state (convergent evolution), their DNAs would be nearly identical.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "inU5yJ7BnR", "reviewer": "Reviewer_vzx5", "summary": "This paper proposes LLM DNA, a low-dimensional representation of large language models obtained through random projection on model outputs. The authors formally define DNA as a bi-Lipschitz embedding, prove its existence via the Johnson-Lindenstrauss lemma, and demonstrate its utility for detecting model relationships across 305 models. Experiments show 95.7% AUC in relationship detection and reveal previously undocumented model relationships through phylogenetic tree analysis.", "questions": "1. Phylogenetic trees in biology represent species-level divergence, yet your tree shows iterative model versions like Qwen2.5 and Qwen3 as separate branches. Do you think the biological evolution analogy genuinely fits the nature of LLM development, or does it create conceptual mismatches?", "limitations": "", "rating": 4, "confidence": 3, "soundness": 3, "presentation": 2, "contribution": 2, "strengths": "1. The paper addresses an interesting problem of tracing LLM evolutionary relationships through the DNA concept, demonstrating potential as a practical tool. \n\n2. The experimental scale of 300+ models is substantial, and the discovery of false negative samples shows promising value.", "weaknesses": "1. The paper's core issue is presenting standard dimensionality reduction through two layers of excessive framing that obscure rather than clarify the contribution:\n   - Mathematical formalization: Sections 3-4 occupy too much space establishing Hilbert space formulation, bi-Lipschitz conditions, and applying the Johnson-Lindenstrauss lemma. However, this entire framework is generic, applying identically to image classifiers, recommender systems, or any function class. Theorems 3.4-3.5 contribute no LLM-specific insights, merely restating that this is random projection with standard techniques.\n   - Biological framing: The DNA terminology (\"inheritance,\" \"genetic determinism\" in lines 108-115) lacks citation to biological literature supporting these characterizations. For the ML community, framing this as \"low-dimensional manifold learning\" would be far more transparent and align with established understanding of dimensionality reduction methods. Why is phylogenetic tree construction necessary for analyzing LLM relationships? Since the method only requires a distance matrix, are there simpler established methods that could achieve similar relationship visualization without requiring biological background knowledge? \n\n2. Related work inadequately contextualizes the approach. The \"Representation of LLMs\" in Section 2 mentions model routing and ensemble learning without explaining these tasks or their connection to LLM DNA extraction, leaving unclear how this work differs from or builds upon existing representation learning methods.\n\n3. Baseline comparisons are critically insufficient. This is fundamentally model similarity measurement, yet experiments only compare against \"Random\" and \"Greedy\" strawmen plus EmbedLLM.\n\nOverall, I recommend the authors reduce the extensive mathematical and biological framing that appears sophisticated but contributes no novel insights, and instead devote more effort to contextualizing the work within existing literature and conducting more rigorous experiments.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "rdMeHXMHjx", "reviewer": "Reviewer_ZryQ", "summary": "This paper defines a DNA to LLMs in order to build a distance metric between LLMs to produce very interesting mapping at a fairly low cost (only need a forward pass and no need to retrain a model when adding a new LLM to the map).", "questions": "- The method presented used classical benchmarks to test the completion of models on these benchmarks. Some models are suspected to be contaminated on classical benchmarks like MMLU. This could bias the metric making models with different origins appear with a similar DNA. Wouldn't it be better to use text not included in classical benchmarks to avoid this ?\n- The algorithm seems to capture very well some relationships between models in Figure 4. However some relationships are not captured such as Meta-Llama-3-8B that is not close to Meta-Llama-3-8B-Instruct while the latter is likely finetuned from the former. What could cause this difference ? (there are many other examples of models that are not close to each other in the figure)\n- In Figure 4, clusters are based on organizations that trained the models. However similarity between LLMs could either come from a similar base model (but trained on different finetuning sets) or because they come from different pretraining sets (but were finetuned on the same finetuning set). How does this plays out in the results ? Does one scenario lead to much higher similarity than the other ?", "limitations": "", "rating": 6, "confidence": 3, "soundness": 3, "presentation": 3, "contribution": 2, "strengths": "- importing concepts from genetics to apply them to LLMs is inspiring\n- the method is well documented and math supports the methods\n- the subject addressed is important as more and more models are published everyday with lower and lower transparency about training details especially for private models", "weaknesses": "- **Important related work missing** : This paper adresses a nearly identical topic with similar methods than another paper published last year in the same conference https://iclr.cc/virtual/2025/poster/28195 without discussing and comparing with it. An in depth comparison of both methods would be very important as it would provide a good baseline to compare both the theoretical framework as well as results in practice increasing the impact of the paper.\n- **Definition of LLM evolution unclear** :The paper defined clearly what is genetic determinism and inheritance but didn't clearly define what is evolution in LLMs while this is a crucial concept in the paper. Is it a model A finetuned from model B ? Is it the general evolution of pretraining sets ? Is it something else ? This concept is central to the analogy and could deserve a much more in depth explanation strengthening even further the already strong basis of the paper.\n- **No limitations discussion** : are there cases where the algorithm doesn't work as expected ? It is important to discuss the upsides of the methods but also the parts where it doesn't perform particularly well while explaining why. This can only make the paper more interesting for the reader as well as fostering follow up work.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}]

Author rebuttal:

Decision: Accept (Oral)