Every Language Model Has a Forgery-Resistant Signature
Authors: Matthew Finlayson
Xiang Ren
Swabha Swayamdipta
Venue: ICLR 2026 Oral (2026)
Keywords: fingerprint
watermark
language model
signature
accountability
cryptography
forgery
security
Primary area: alignment, fairness, safety, privacy, and societal considerations
TL;DR: We show that all language models impose elliptical constraints on their outputs, which can be used as a hard-to-fake signature to identify a model from its outputs.

Abstract: The ubiquity of closed-weight language models with public-facing APIs has generated interest in forensic methods, both for extracting hidden model details (e.g., parameters) and identifying models by their outputs. One successful approach to these goals has been to exploit the geometric constraints imposed by the language model architecture and parameters. In this work, we show that a lesser-known geometric constraint—namely that language model outputs lie on the surface of a high-dimensional ellipse—functions as a signature for the model, which be used to identify which model an output came from. This ellipse signature has unique properties that distinguish it from existing model-output association methods like language model watermarks. In particular, the signature is hard to forge: without direct access to model parameters, it is practically infeasible to produce logprobs on the ellipse. Secondly, the signature is naturally occurring, since all language models have these elliptical constraints. Thirdly, the signature is self-contained, in that it is detectable without access to the model input or full weights. Finally, the signature is exceptionally redundant, as it is independently detectable in every single logprob output from the model. We evaluate a novel technique for extracting the ellipse on small models, and discuss the practical hurdles that make it infeasible for production-size models, making the signature hard to forge. Finally, we use ellipse signatures to propose a protocol for language model output verification, which is analogous to cryptographic symmetric-key message authentication systems.

Published as a conference paper at ICLR 2026
EVERY LANGUAGE MODEL HAS A FORGERY-RESISTANT SIGNATURE
Matthew Finlayson, Xiang Ren, & Swabha Swayamdipta fmfinlays, xiangren, siangren,,wabbas)qun swabhas)@usc. edu University of Southern California
ABSTRACT
The ubiquity of closed-weight language models with public-facing APIs has generated interest in forensic methods, both for extracting hidden model details (c.g., parameters) and for identifying models by their outputs. One successful approach these goals has been exploit the geometric constraints imposed by the language model architecture and parameters. In this work, we show that a lesser-known geometric constraint-nammll that language model outputs lie on the surface of a high-dimensional ellipse-functions as a signature for the model and can be used to identify the source model of given output. This ellipse signature has unique properties that distinguish it from existing model-output association methods like language model fingerprints. In particular, the signature hard forge: without direct access to model parameters, it is practically infeasible to produce propoabbailliies (logprobs) on the ellipse using currently known methods. Secondly, the signature is naturally occurring, since all' language models have these elliptical constraints. Thirdly, the signature is self-contained, in that is detectable without access the model inputs or the full weights. Finally, the signature is compact and redundant, as is independently detectable in each logprob output from the model. We evaluate novel technique for extracting the ellipse from small models and discuss the practical hurdles that make it infeasible for production-scale models. Finally, use ellipse signatures propose protocol for language model output verification, analogous to cryptographic symmetric-key message authentication systems.
INTRODUCTION
The proliferation of closed-weight language models has incentivized the development of methods for language model forensics, i.e., post hoc methods to learn about language models and their outputs with limited access. Recent work in this area has introduced methods to exploit linear constraints imposed by the model architecture et al., 2024; Yang & Wu, 2024). These methods use model constraints as type of signature, where one can verify that an output came from specific language model by simply checking that the output (typically, the model's output log probability vector) satisfies the model's constraints. Thus the signature functions as a type of watermark or fingerprint (Xu et al., 2025b; Liu et al., 2024).
identify the source of generations through a limited API (Finlayson
Another, desser-known constraint can also function as model signature: the constraint that model outputs lie on a nigh-dimensional ellipse (a hyperellipsoid) (Carlini et al., 2024, Appendix G). Model outputs can be verified by checking that they lie the model ellipse. In this work, explain the ellipse constraints and show how they can be used to identify the model that generated an output.
We find that ellipse signatures have set of four unique properties, which differentiate them from previous signatures and other model identification methods. In combination, these properties fill a new niche in the landscape of output verification systems. We emphasize that signatures are not necessarily better than other methods, rather their properties make them more suitable in specific situations. First, ellipse signatures are forgery- resistant, i.e., the signature is hard to fake for -eighh- models.
More precisely, this applies to models with final normalization layer, which encompass virtually all widely used language models today.

---

Published as a conference paper at ICLR 2026
Model A ellipse
Logit outputs
Language model A
LM
Language model
Figure 1: Language model logits are subject to constraints that force them lie on a high-dimensional ellipse. This constitutes a signature because we can identify which model generated an output by checking which ellipse lies on. Among other unique properties, we show that ellipse signatures are forgery-resistant because it is computationally hard in practice to generate signed logits without access to the model parameters.
This differentiates the ellipse signature from previously known linear signatures (Finlayson et al., 2024; Yang Wu, 2024). Second, ellipse signatures are naturally occurring, because virtually all modern language models have unique ellipse constraint and therefore sign their outputs. In contrast, many watermark and fingerprinting methods require the model or inference provider to proactively implement the system (e.g.. Zeng et al., 2025; Cui et al., 2025). Third, ellipse signatures self- contained: output detection does not require access to the model parameters or input. Self-containment useful for situations where provider wants trusted third party to be able verify outputs from their private language model without giving away the model parameters or prompt. Finally, the ellipse signature compact and redundant because the signature is present and detectable in any single generation step.² Consequently, a single generation step is sufficient to identify the generating model. This property is unique because many existing output identification methods require outputs from multiple generation steps to gather evidence of the generator's 's identity (e.g., Kirchenbauer et al., 2023).
The most interesting and least obvious property of ellipse signatures is their forgery resistance (Section 3). We define forgery as generating logprobs that conform to the model constraints without direct access to the model parameters. Previously studied linear signatures (Finlayson et al., 2024; Ying et al., 2012) can forged by extracting the linear constraints from the model API and generating logprobs to satisfy them. In comparison, ellipse extraction from APl-protected language models is extremely difficult, and thus forgery is much harder. Ellipse extraction is difficult for at least two reasons: it is expensive, with O log d) query complexity in OienAI-like API, and actually fiuting the ellipse has O(d)) time complexity. We demonstrate these challenges by implementing a new ellipse-specific extraction method and testing it small models. We are aware of any ellipse forgery method that avoids having to fit the ellipse, though we cannot at this time mathematically rule the possibility. For this reason, we adopt the term "forgery resistance" rather than "unforgeability..
The forgery resistance of ellipse signatures, coupled with the relative ease of verifying outputs on the ellipse, presents an intriguing opportunity for an output verification system. We propose a system analogous to cryptographic message authentication (Pass & Shelat, 2010), where the model ellipse functions as the secret key. Parties with access to secret language model parameters (including the ellipse parameters) can generate logprobs that can in turn be verified only by those who also have access to the secret ellipse (Section 4). We argue that such a system has implications for model forensics, regulation, and accountability for opaque choices made by language model providers.
2 LANGUAGE MODEL ELLIPSES ARB SIGNATURES
We set up the mathematical formulation of the language model ellipse and show how it functions as signature. To begin, let us assume that our language model has typical architecture, like the one shown in Figure In particular, we are interested in models with vocabulary size v much larger than their hidden size d, that have final sequence of layers consisting of a normalization, followed by a linear layer Rd R
2 As opposed to wein-w models with publicly-relessed parameters. By "single generation step", mean the probabilities of every next-token candidate the vocabulary.
2

---

Published as a conference paper at ICLR 2026
Output probabilities
Softmax
Lincar
Strotches & rotstes
Norm
Maps to sphere surface
Neural net
Daecing
Input
805
Figure 2: A typical language model's ' final layers consist of normalization followed by a linear (or affine) transformation. Normalization has the effect of mapping the representations onto the surface of sphere, while the linear layer stretches and rotates this sphere, resulting in an ellipse. Transformer diagram credit Vaswani et al. (2017); Negrinho (2020).
2.1 LANGUAGE MODEL OUTPUTS LIE ON AN ELLIPSE
Language model outputs lie on an ellipse because the penultimate model layer normalizes the activations before projecting them linearly into R" Model architects tend to choose one of two normalization schemes: the poot-mean-qqult (RMS) norm (Zhang & Sennrich, 2019) or the layer norm (Ba et al., 2016). We will focus on the RMS norm because is simpler, though we revisit the layer norm in Section E for completeness. The RMS norm is formally defined
RMSNorm(x)
(I)
E
where & is small, positive term added prevent division by zero. We note that the magnitude of the normalized output is Vd. For pedagogical reasons, will proceed with simplified, scaled RMS norm without an & term norm(x)= x yd E[x²], so that the normalized outputs will have magnitude 1. Normalization has the property of mapping inputs onto the surface of d-dimensional sphere because it sets the magnitude to while preserving the direction. To simplify notation, we will denote norm(x) as &. It is also common apply a learned element-wise affine transformation after normalizing, by multiplying by E Rd then adding a bias term B e Rd, so that the output is y & B. We call the output of the normalization layer the (final) hidden state of the model.
To obtain a logit vector from the hidden state, the model multiplies the hidden state by the unembedding matrix W. Because the normalized representations & lie on a sphere, and W(y @ 8 B) an affine transformation, the logits lie on the surface of a d-dimensional ellipse. It is important to note that the ellipse, like the sphere, is d-dimensional, even if is projected into v-dimensional space. The d d-dimensional ellipse inhabits R" in the same way that comet's 2D elliptical orbit inhabits 3D space.
Finally, language model APIs usually return log- probabiliiies (logprobs). i.e., log softmax (W(y @X B)), rather than logits. Since the softmax function is invariant to scalar addition, the logprobs remain unchanged if we assume that the logits are centered, i.c., CW(y @ & + B) where C' = 1- J. This simplifying assumption makes recovering logits from logprobs possible, since C' log softmax ((y f + B)) = Cy @ & + B). Section A gives more in-depth explanation of this assumption.
2.2 LANGUAGE MODEL ELLIPSES ARE SIGNATURES
The model ellipse can be interpreted as a signature that associates outputs to the language model that produced them. To verify that a model produced output, one can check its distance the model's ellipse. If it on the ellipse, it likely came from the model; otherwise, it almost certainly did not, since both in theory and practice the likelihood of output falling the intersection of two ellipses extremely low. To measure the distance a logprob to the ellipse, we inspect the magnitude the ellipse's 's inverse affine transform applied the logprob, i.e., (W+ C+ BB/y, where . denotes
3

---

Published as a conference paper at ICLR 2026
Avg. distance to ellipse
10a
Olmo 2 7B
Olmo 2 (300) 7B
10-2
Llama 3.1 8B
Qwen 3 8B
10-6
GPT OSS 20B
Olmo 2
Olmo 2 (300)
Llama 3.1
Qwen 3
GPT OSS
Target model
Figure 3: Mcan distance to the model cllipse for logprobs generated from several open-weigtt models, projected onto each other's output spaces. Small distances indicate that the outputs are on the target model ellipse, and therefore came from the target model. Standard errors, shown as shaded regions, are mostly too narrow to see.
pseudoinverse. If the legprob came from the model, then this transformation should map the logprob back onto the unit sphere. Thus, we can interpret the deviation of the magnitude from as a distance from the ellipse (in linearly transformed space).*
To evaluate the effectiveness of language model signatures at identifying language model outputs, we generate set of logprob outputs from four popular open-weight language models: Olmo 2 7B (OLMo et al., 2025), Llama 3.1 (Grattafiori et al., 2024), Qwen 8B (Yang et al., 2025), and GPT oss (OpenAI al., 2025). Since most of these models have different vocabularies and column spaces, we find set of tokens that are common to all language models, then map each model's outputs onto the column space of each other model such that the cross-entropy between the original and projected outputs is minimized for the shared tokens. This essentially copies the linear signature of the target model onto the logprobs generated by another model. Finally, we apply the inverse affine transform and check the distance of the logprobs to the unit sphere for each model. Plotting these distances in Figure 3, we indeed find that the generating model always has the smallest distance to the unit sphere by several orders of magnitude. We also include the second-to-last Olmo 2 checkpoint (300) compare with Olmo 2. We find that even in this case, the ellipse signature cleanly identifies the generating model.
As mentioned in Section 1, ellipse signatures have several unique properties that set them apart from existing output verification systems. In particular, ellipse signatures are naturally occurring because all modern language models have final normalization layer; they self-contale cofttinnd because the signature-checking procedure does not depend on the model input or parameters (except W, y, and B); and they are compact and redundant because every single logprob output bears the ellipse signature.
2.3 COMPARISON TO EXISTING METHODS
To understand the niche that ellipse signatures fill, it is worthwhile to consider the breadth of existing methods for identifying models by their outputs, and how they differ from ellipse signatures. Many of these methods fall under the umbrella of language model fingerprinting. though we do not consider the ellipse signature itself to a fingerprint because it lacks certain properties, namely robustness and stealthiness (Xu al., 2025b, Section 2.3). Note that differences in properties do not make one method "better" than another we highlight differences to show model ellipses and the compared methods are useful in different scenarios. Various methods and their properties are summarized in Table 2 in Section B. We are unaware of any existing model identification systems that exhibit all of the ellipse signature properties simultaneously.
bext- watermarks (Liu et al., 2024), subclass of fingerprint methods, inject signals into the text generation process that can later be used identify the generating model. This usually accomplished via sampling strategies (e.g., Kirchenbauer et al., 2023; Christ et al., 2024; Hou et al., 2024) which are sometimes distilled back into the model (Xu al., 2025a, e.g..). Watermarks are not naturally occurring, since they require intentional implementation on the model provider side. They are also not compact, since they require gathering statistical evidence over many generation steps.
*Section H discusses of how logit transformations like temperature affect this result.
4

---

Published as a conference paper at ICLR 2026
One common approach to fingerprinting is to train backdoors into language models so that the model responds to specific inputs in a way that reveals its identity (e.g., Li et al., 2022). These are neither naturally occurring, nor self-contained. since they require special training, and require control over the model input in order to elicit the identifying response.
Some methods naturally occurring fingerprints in output text to identify the generating language model. These include analysis of patterns in chain-of-thougle outputs (Ren et al., 2025), as well as more straightforward training of classifiers for the task (Mitchell et al., 2023). Though closer in nature to ellipse signatures due to their natural occurrence, these are not compact because they require multiple generation steps to identify the generating model.
Sun et al. (2024) propose zero -noollnoee proof for language models (zkLLM), which can guarantee that language model produced an output without sharing any model details, making it self-contained and hard to forge (with strong guarantees). While this method gives much stronger guarantees about how an output was produced than model ellipse, comes with the drawback that it makes inference much more expensive. Naturally occurring model ellipses on the other hand, do not.
The most similar method to ours is the method proposed by Finlayson et al. (2024) and expanded upon by Yang & Wu (2024), which uses linear signature to identify models. The main difference between ellipse signatures and linear signatures that only linear signatures are easy to forge while ellipse signatures are hard, as we will show in the next section.
3 ELLIPSE FORGERY IS HARD
In the context of model signatures, forgery means generating new outputs that conform to the model constraints without direct access to the model parameters. Formally, given a set of model outputs .,......Kz and a black box signature verifying function f such that f(x) = I for all x forgery requires producing a new output & which passes verification, i.e., f()) 1. In the case of the ellipse signature, f(x) = 1 if and only if is on the model ellipse.
Linear signatures (Finlayson al., 2024: Yang & Wu, 2024) are easy to forge, due results by Finlayson et al. (2024); Carlini et al. (2024). Their methods can forge a linear signature by extracting the linear constraints from an API, then producing new logprobs that satisfy those linear constraints (as we did in $2). In this section, we will demonstrate that, while it is possible to forge ellipse signature by extracting the ellipse constraint from an API, it is practically infeasible for sufficiently large models using current known methods. Ellipse signature forgery resistance therefore relies on the idea that, as far as we are aware, there is no known method to produce new points on an ellipse without first fitting an ellipse the known points. Here, we will make the same API assumptions used in Carlini et al. (2024); Finlayson et al. (2024); Nazir et al. (2025); Morris et al. (2024), where the API allows users to specify a prompt and then returns logprobs for fixed set of at least d tokens at every generation step. These assumptions are reasonable, given that OpenAl's Completions and Chat Completions APIs meet the criteria, specifically because the logi fogit_bias bias parameter allows users to find the logprob of any token, as shown in Morris et al. (2024).
Carlini et al. (2024) present a method for extracting the ellipse from model outputs. The general idea is to use fitting algorithm to find the ellipse that these outputs lie on. The parameters of the ellipse of best fit correspond to the singular values, rotation, and bias values in the model's 's final layer. We summarize the procedure in Algorithm and present more comprehensive overview in Section C. We then show that ellipse extraction hard for at least two reasons. First, the query complexity of extracting enough logprobs from an API to determine the ellipse is O(d log, d), and second, the time complexity of the algorithm for fitting an ellipse to the outputs is (dd))
3.1 ELLIPSE FITTING ERRORS FROM SMOOTHING
In theory, the & term in the denominator of a normalizatoin layer can cause issues for recovering the model parameters. Instead of enforcing |]norm(x)]] ||2 1, these layers enforce [|norm(x)]]a < 1, meaning that outputs fall within the interior of the ellipsoid, instead of on the surface. This effect decreases for larger models (see Section G), and in practice,
cause ellipse fitting fail.
In our own experiments, we found that the SVD-based ellipse fitting from Carlini et al. (2024) would sometimes fail for smaller models because the parameter E output by the fitting algorithm was not
5

---

Published as a conference paper at ICLR 2026
Algorithm 1 Get output layer parameters of language model.
function GET PARAMETERS( logprobs f,....6y R*
WH = [4 Ln R
Create matrix from logprob outputs
HC
Find embedding size of model
A = Itdd
Choose down-pree www-pprojection
A (WH(AC(WH)))))
p Solve for up-projection
E, b = ELLIPSOIDFIT( (LLLPSODDFI(AACI..... (AC(........ ACtn)
> Solve for ellipse
U.L. =svd[Cholesky(E")
> Convert ellipse to affine form Return up-projection, stretch, rotation, and bias.
return A .
end function
Bias
Singular values
Rotation
2
True Predicted
10a
0
-2
100
-4
000
000
0
20
69
60
0
20 040
60
20
4001
60
Figure 4: The predicted and true values for bias and singular values, and the angles between columns of the predicted and true rotation matrices for a calllonn-arameee model. The angles must lie the range x]. and the true rotation matrix columns have angle O with themselves. Our predictions are highly accurate, demonstrating robustness normalization smoothing in ellipse fitting algorithms.
positive definite. The failure stems from the fact that SVD-based fitting is not ellipse-specific. The algorithm fits quadric surface to the points, which may or may not be an ellipse, especially in the presence of noise from the & term. To guarantee the validity of our recovered ellipse, we turn to cllipse-specific fitting algorithms. We use fast algorithms for multidimensional ellipse fitting using semidefinite programming (Calafiore, 2002; Ying et al., 2012), in particular the ellipse fitting method from Ying et al. (2012), which is straightforward to implement, fast, and stable. We further detail our implementation in Section F. general, we find that smoothing does not hurt ellipse fitting for models of sufficient size, confirming an observation from Carlini et al. (2024).
We test our ellipse-speeific method on a small, pretrained, open-source model. particular, we use 1 1-million-parameteer language model with embedding size of 64 and final layer norm with & = x 10-5 (Black et al., 2022; Eldan & Li, 2023). We obtain outputs from the model using the Pile dataset (Gao et al., 2020), then use the outputs to fit ellipse and compare the predicted and true parameters. Figure visually demonstrates their high similarity. As expected, we observe a consistent, slight underestimation of the model's 's singular values due to smoothing.
To measure the benefits of using more outputs to fit the ellipse, we quantify the similarity between the estimated and true rotation, stretch, and bias for varying numbers of outputs in Figure 5. We measure bias and stretch similarity using mean squared error (MSE). For the rotation similarity between the predicted rotation U and true rotation U", we use the geodesic distance tr(U' U*). Overall, find that using more output samples generally improves parameter predictions, but has diminishing returns as irreducible error due begins dominate.
3.2 THE SAMPLE COST OF MODEL ELLIPSE RECOVERY SUPER-CUBIC
Recovering the ellipse of an LM requires (dd) outputs. An ellipse is special case of quadric surface (or simply quadric), which has the general equation
d Q Mi
Σ
Pixi
(2)

---

Published as a conference paper at ICLR 2026
Stretch (MSE)
Bias (MSE)
Rotation (geodesic distance)
102
Distance
100
0.146
0.312
00157
10-2
20,000
40,000
20,000
40,000
20,000
40,000
Samples
Samples
Samples
Figure 5: Distance between predicted and true parameters for bias, stretch, and rotation (lower is better). Fitting the ellipsoid to more samples generally improves the predictions, though with diminishing returns.
Model
Hidden size d
Vocab size
Samples for Ellipsoid
Ellipse (S)
pythia-70m
512
50,304
131,327
babbage-002
1536
101,281
1,180,415
1056
gpt-3.5-turbo
4650>
101,281
10,813,574
150,699
1lama-33 3--:0--instrenee
8192
128,256
33,558,527
16,487,421
Confirmed size from Carlini al. (2024).
Estimated size upper limit from Finlayson et al. (2024).
Hypothetical cost based on inference cost for gpt-4-turbb.
Table 1: Models, their sizes, the number of samples required ascertain their output ellipse, and the cost of the samples, based on OpenAl inference pricing on September 16, 2025. The number of samples required grows quadratically with the hidden size of the model, and the price grows cubically.
where the symmetric matrix Q E IRdxd and P E Rd are parameters. An ellipse has the property that Q is positive definite. The set vectors Rd that satisfy this equation form the ellipse surface. Since the total number of terms in the above equation is d(d 3)/2, and the equation for quadric is linear in its parameters, a set Odd) points is required in the general case to uniquely define an ellipse. In fact, in the worst case (2d=) samples are required to find even a single new (not in the set of samples) point on the ellipse, since if the samples are in general position then for every point not in the samples there is an ellipse that includes the samples but not the point.
For a model with an AMS-norm and modest embedding size 2 = 512, we would need at least 217 3 28 = 131,840 outputs. For Llama 3 8B, the embedding size is 212 4096, and would need 8,394,752 outputs. This quadratic growth makes finding model's 's ellipse from its outputs much more expensive than extracting the model's column space, which only requires (d) samples.
In order to minimize cost, an attacker typically sends a single prefix token to an LM API for each sample. However, as the required number of samples surpasses the vocabulary size of the model it becomes necessary to send multi-token prefixes to the model in order to expand the number of unique prefixes. The number of tokens per sample grows logarithmically with the number of samples required, or O d). This would bring the overall API cost to (d² log d) queries.
If the API only reveals a constant number of token logprobs per query, as has been the case historically for many API providers, the attacker needs to send multiple queries to recover the full logprob vector. Here, the attacker can save on cost by collecting d full logprobs, then only collecting logprobs for the same subset of d tokens for subsequent outputs, solving for the missing logprobs numerically later on. Even so, the attacker will still make (d) queries per sample. all, this means that the cost of discovering the model ellipse grows at rate of ( d log d), where the vd term accounts for the cost of collecting the initial d full logprobs.
Since the cost grows super-cubically with the embedding size of the model, current API pricing makes it prohibitively expensive to obtain the ellipse of many popular LMs, as shown in Table 1. Though OpenAl's 'ss cheapest and smallest available generative model, babbage-002, would cost about $ $1000 to attack, gpt-3. 3. ttrrbo would cost over $ $150,000. A 70B-scale model with inference cost similar to gpt-4- 4---bbburb would cost over $1 $16 million.
7

---

Published as a conference paper at ICLR 2026
Liama, T0B
15,000
(minutes)
15
(16,167 yeans" O
Time (years)
10
10,000
5
0
5,000
100
200
OpenAl Hubbage 002 (1 years)
Lhma 8D (254
0
0
1,000
2,000
3,000
4,000
5,000
6,000
7,000
8,000
9,000
Dimension d
Figure 6: Extrapolated running time of our implementation of the ellipse extraction algorithm from Ying et al. (2012). We extract ellipses from several models with different hidden sizes (blue). Fitting a degree-6 polynomial to these points, parameters of larger, more popular models.
extrapolate to guess at the time required to extract the
3.3 ELLIPSOID FITTING TAKES SEXTIC TIME
Obtaining sufficient samples from an APl-protected language model is only the first step in finding the model ellipse. It turns out that the second step, fitting an ellipse to the samples, is prohibitively expensive computationally. Both the SVD-based and ellipse-specific fitting methods discussed in the previous section typically require (dd) time, the time required to solve (dd) equations of (dd) variables (Carlini et al., 2024). Faster methods, such as those based on Strassen's 's algorithm (s (n277)))) are possible in the non-ellipse-speciie speedups is Onn)) (whether this bound tight is an open question in computer science), which still leaves us with a complexity least (d*). The best known ellipse-specific method still requires (dd) time and Odd) space (Lin & Huang, 2016).
case, but the absolute lower bound for such
To evaluate the real-world practicality of ellipse fitting, we implement the algorithm from Lin & Huang (2016), and record the solving times for ellipses in dimensions ranging from 8 to 256. The times Figure 6 are obtained by running the algorithm using 64 CPUs. Extrapolating the best- -ft polynomial of degree 6, recovering parameters from a typical 70 billion parameter model would take thousands of years."
We leave it as an open question as to whether there exists fast algorithm for generating new outputs on an unknown model ellipse based on samples. We ourselves tried handful of optimizations to speed up ellipse fitting, such as parallelization on GPUs and approximation. Unfortunately, the memory requirements of ellipse fitting makes GPU acceleration infeasible for sufficiently large models, and approximation methods catastrophically degraded the accuracy of our recovered ellipse parameters.
4 ELLIPSE SIGNATURES ARE MESSAGE AUTHENTICATION CODES
The hardness of extracting ellipses, combined with the cheapness of checking logprobs against the ellipse, creates a type of trapdoor function which can be used to verify model outputs. particular, we can interpret the model ellipse as a secret key, and the logprob outputs as message, in analogy to keys and messages in cryptographic message authentication systems (Pass & Shelat, 2010).
In such systems, signer sends a message and message authentication code (MAC) to verifier. The verifier uses the MAC to confirm the authenticity of the message. This process works as follows: first, the signer shares secret key with the verifier. To send verifiable message, the signer generates a tag from the key and message, e.g., by hashing their concatenation, and sends the message and tag to the verifier. The verifier can confirm the authenticity of the message by replicating the tag with their own copy of the secret key, e.g., by again hashing the concatenation of the key and message.
In the of the model ellipse, the ellipse typically is chosen via the model training, but could also be chosen by sampling a random ellipse. Signing occurs when the model generates logprobs. The
'We run our experiments on AMD BPYC 7643 48-core processors. 6uch extrapolations should be taken with grain of salt-these numhers intended only give intuition.
8

---

Published as a conference paper at ICLR 2026
Alice
Secret key
Bob
Alice
Secret LM
Bob
Key
Key
LM
Ellipse
Inforence
Mcssage
Senck
Mcssage
Tag
Logprobs
Sendo
Logprobs
Generates
Generates
Tag
Tag
Same?
On ellipse?
Figure 7: A cryptographic message authentication system (left) uses shared secret key to validate messages. The message author, Alice, signs the message by generating a tag from the message and secret key. After receiving the message and tag, Bob verifies the message by generating another tag with his copy of the key. If the received and generated tags match, the message is authentic. In our proposed verification system (right), the ellipse functions as a secret key, and logprobs take the role of messages. The tag is encoded in the position of the logprobs in R", and can be verified by checking that they lie on the secret ellipse.
logprobs contain both the message (information about the next-token distribution) and the tag, which is encoded in the logprob's 's position in R". To verify the message, someone with the secret model ellipse can verify that the logprob's 's position is on the ellipse.
The security of MAC hinges on the infeasibility of forgery. For MAC, forgery means producing message-tag pairs that pass verification (and have not previously been generated) without access to the secret key. For the ellipse signature, would-be forger's task is to efficiently produce logprob that lies on the model ellipse. The most obvious forgery attack would be to steal the model ellipse by collecting outputs, solving for the ellipse, then using the ellipse to produce new logprobs. As discussed in Section 3. this is impractically hard for production-sin language models of today.
Because ellipse verification guarantees only that individual logprob vectors came from a specific model, it is possible that an attacker could create plausible token sequence g of length by saving a collection of logprob outputs from target model, then piecing them together a sequence {()) ((n)
such that the top token from each logprob corresponds to a token in g, i.e., arg max; / 8j There are two potential defenses against this type of attack. The first would be to give the verifier access to database of all outputs ever produced by the provider. At scale, this might be prohibitively costly to maintain. A second deterrent to this kind of attack is that logprobs have been shown to contain large amount of information about their prefix (Morris et al., 2024; Nazir et al., 2025), meaning is possible to train language model inverters that reliably reconstruct the prefix of all logprob outputs. If the verifier finds that the inverter assigns low likelihood to the first half of the sequence when conditioned on the second half, then it is likely that the sequence has been tampered with.
message authentication protocol for language models can be a tool for language model accountability. As hypothetical scenario, suppose that language model providers are required by law to share their ellipse with a trusted third party. Then, a user receives a harmful output from the model and sues for damages and the provider denies generating the output, the third party can provide convincing evidence about which party is correct due to the forgery-resistan property of language model ellipses.
5 DISCUSSION AND CONCLUSION
Because of their unique properties, ellipse signatures fill a new niche in the space of language model verification methods. This combination of their features is especially promising for doing language model forensics, since ellipse signatures can be used to identify any model with high certainty, even the output provider did not intend to sign their outputs.
Ellipse signatures for language models can be improved on at least three fronts. First, the hardness of forging ellipses is only polynomial, far from cryptographic security guarantee. It is likely possible to identify other constraints on model outputs that give stronger guarantees. Second, our proposed ellipse signature protocol requires that the API provides logprobs. As of this writing. OpenAl is the only major commercial provider that allows access to logprobs, and even this access limited to handful of models through ad hoc querying workarounds. Lastly, the ellipse signature does not.

---

Published as a conference paper at ICLR 2026
have the desirable property of being difficult remove (often goal for model fingerprints), since modifying the model outputs or parameters erases the signature by breaking or changing the ellipse constraints. Future work could explore other kinds of signatures for language models, which are hard to remove. We hope that ellipse signatures will offer an exciting new avenue for research into such signatures, which will greatly impact language model security, accountability, and forensics.
6 ACKNOWLEDGEMENTS
Matthew Finlayson is supported by fellowship from the National Science Foundation (NSF) Graduate Research Fellowship Program. Swabha Swayamdipta's research is supported part by the NSF under grant I1S2403437, the Simons Foundation, Apple, Intel and the Allen Institute for AL Part of this research was done when Swabha Swayamdipta and Matthew Finlayson were visitors at the Simon's Institute. Xiang Ren's research is supported in part by the Office of the Director of National Intelligence (ODNI) Intelligence Advanced Research Projects Activity (IARPA), via the HIATUS Program contract #2022-2207220000 the Defense Advanced Research Projects Agency with award HR001 12220046, and NSF IIS 2048211. We would like to thank all the collaborators in INK and DILL research labs USC for their constructive feedback the work.
REFERENCES
Jimmy Ba, Jamie Ryan Kiros, and Geoffrey E. Hinton. Layer normalization. ArXiv, abs/1600.066550 2016. URL wtthttpss//aaat att.p.ssmmaiiiscoollrrrr hwt.h/oo.larrrrooopruaoeeeeeeeee 82336311 8236317.
Sid Black, Stella Biderman, Eric Hallahan, Quentin Anthony, Leo Gao, Laurence Golding, Horace He, Connor Leahy, Kyle McDonell, Jason Phang. Michael Pieler, USVSN Sai Prashanth, Shivanshu Purohit, Laria Reynolds, Jonathan Tow, Ben Wang. and Samuel Weinbach. GPT-NeoX-20B: An open-source autoregressive language model. In Proceedings of the ACL Workshop on Challenges & Perspectives in Creating Large Language Models, 2022. URL https: //assi///xxvvv.// org/ abs/ 2204. 06745.
G. Calafiore. Approximation of n-dimensional data using spherical and ellipsoidal primitives. IEEE Trans. Syst. Man Cybern. Part A, 32:269-278, 2002.
Nicholas Carlini, Daniel Paleka, Krishnamurthy Dvijotham, Thomas Steinke, Jonathan Hayase, A. Cooper, Katherine Lee, Matthew Jagielski, Milad Nasr, Arthur Conmy, Eric Wallace, David Rolnick, and Florian Tramèr. Stealing part of a production language model. In Proceedings of the 41st International Conference on Machine Learning, volume 235 of Proceedings of Machine Learning Research, pp. 5680-5705, Vienna, Austria, July 2024. PMLR. URL https: //proceedings. mlr. press/v235 http:///////aaaniiaaa/aa wwt.ccaanniiiiie html.
Miranda Christ, Sam Gunn, and Or Zamir. Undetectable watermarks for language models. In Shipra Agrawal and Aaron Roth (eds.), Proceedings of Thirty Seventh Conference on Learning Theory, volume 247 of Proceedings of Machine Learning Research, pp. 1125-1139, PMLR, 30 Jun-03 2024. URL https: https::/proceedings.. htt.eeeediiiii....eeeeeeeeeeeeee press//247// / christtaa.ttaaaa htmll.
Xinyue Cui, Johnny Wei, Swabha Swayamdipta, and Robin Jia. Robust data watermarking in language models by injecting fictitious knowledge. Proceedings of the First Workshop on Large Language Model Memorization (L2M2), pp. 1000200..00 Association for Computational Linguisties, 2025. doi: 10. 10.116533/11/20252222-- 1.15. URL http: / doi. org/10. 186.5/11/222225 12m2 15.
Ronen Eldan and Yuan-Fang Li. Tinystories: How small can can language models be and still speak coherent english? ArXiv, abs/2305.07759, 2023. URL https:////ap semanticscholar arrg/CorssII: 258686446.
Matthew Finlayson, Xiang Ren, and Swabha Swayamdipta. Logits of api-protected 1lms leak proprietary information. In Proceedings of the Conference Language Modeling (COLM 2024), Philadelphia, PA, USA, October 2024, COLM Organization Committee. URL https: //openreview net:/formmidd-oRTThhaa Conference Language Modeling.
10

---

Published as a conference paper at ICLR 2026
Leo Gao, Stella Biderman, Sid Black, Laurence Golding, Travis Hoppe, Charles Foster, Jason Phang, Horace He, Anish Thite, Noa Nabeshima, Shawn Presser, and Connor Leahy. The Pile: An 800gb dataset of diverse text for language modeling. arXiv preprint arXhv:2101.00027, 2020.
Aaron Grattafiori et al. The Ilama 3 herd of models, 2024. URL https:t/aaxiii.rrrrtt 21783.
Abe Hou, Jingyu Zhang, Tianxing He, Yichen Wang, Yung-Sung Chuang. Hongwei Wang. Lingfeng Shen, Benjamin Van Durme, Daniel Khashabi, and Yulia Tsvetkov. Semstamp: A semantic watermark with paraphrastic robustness for text generation. In Proceedings of the 2024 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies Volume 1: Long Papers), pp. 4067-4082 Association for Computational Linguistics, URL http:// /tpp:://xxx www.doo.oooooooooooo.oooo 10653/
2024. doi: 10. naacl-long.222 226.
John Kirchenbauer, Jonas Geiping, Yuxin Wen, Jonathan Katz, Ian Miers, and Tom Goldstein. A watermark for large language models. In Andreas Krause, Emma Brunskill, Kyunghyun Cho, Barbara Engelhardt, Sivan Sabato, and Jonathan Scarlett (eds.). Proceedings of the 40th International Conference on Machine Learning, volume 202 of Proceedings of Machine Learning Research, pp. 17061-17084, PMLR, 23-29 Jul 2023. URLhhttps//prroceddii v202 /.0l//kirchenbauer23aaa html.
mlr.press/
Yiming Li, Yang Bai, Yong Jiang, Yong Yang, Shu-Tao Xia, and Bo Li. Untargeted back- door watermark: Towards harmless and stealthy dataset copyright protection. In S. Koyejo, S. Mohamed, A. Agarwal, D. Belgrave, K. Cho, and A. Oh (eds.), Advances in Neu- ral Information Processing Systems, volume 35, pp. 13238-13250. Curran Associates, Inc., 2022. URL https::/prroceedngg..nt newrpipss. www.ins..icccco.ooo. cwwpurips.cccpppppppp ccppcper_aappppeeeeeeeeeeeeeeeee Paperee---errre---eeeeeeeeeeeeee aaaappr_filesppppereeeeeeeeeeeee /paper/2022/filee
Zhouchen Lin and Yameng Huang. Fast multidimensional ellipsoid-specific fitting by alternating direction method of multipliers. IEEE Transactions on Pattern Analysis and Machine Intelligence, 38(5): May 6. ISSN 2160-9292. doi: 10.1 (009:/pam.22155.2422299 //dx. www//0........ 1169/ tpami. 2015 2469283.
URLhttp:
http:
Aiwei Liu, Leyi Pan, Yijian Lu, Jingjing Li, Xuming Hu, Xi Zhang. Lijie Wen, Irwin King, Hui Xiong, and Philip Yu. A survey of text watermarking in the era of large language models. ACM Computing Surveys, 57(2):1-36, November 2024. ISSN 1557-7341. doi: 10.1145/3691626. URL http://dx.doi.ore www. 10. 1tt..//....rr111111191111//11111 1145/3691626.
Eric Mitchell, Yoonho Lee, Alexander Khazatsky, Christopher D Manning, and Chelsea Finn. Detectgpt: Zero-shot machine-generated text detection using probability curvature. In International conference on machine learning, pp. 14950-24962 PMLR, 2023.
John Xavier Morris, Wenting Zhao, Justin T Chiu, Vitaly Shmatikov, and Alexander M Rush. Language model inversion. In The Twelfth International Conference Learning Representations, 2024.
URL /ttpps:/ooperreviek net. wow.moiidttdddiiiiii tddoum?id=t9WWWppckkj..
Murtaza Nazir, Matthew Finlayson, John X. Morris, Xiang Ren, and Swabha Swayamdipta. Better language model inversion by compactly representing next-token distributions. In Proceedings of the 39th Conference on Neural Information Processing Systems (NeurIPS 2025), San Diego, CA, USA and Mexico City, Mexico, dec 2025. Neural Information Processing Systems Foundation. URL httpps://operrveee war.eerrvveee.eee/ffrun//m/eeeee met.rn/omeoomomiiii-///eeceett poster.
net
Renato Negrinho. Sane Tikz. https://gii thub httpgtta..mmo//ggiiio//eaaaaa
2020.
Team OLMo, Pete Walsh, Luca Soldaini, Dirk Groeneveld, Kyle Lo, Shane Arora, Akshita Bhagia, Yuling Gu, Shengyi Huang, Matt Jordan, Nathan Lambert, Dustin Schwenk, Oyvind Tafjord, Taira Anderson, David Atkinson, Faeze Brahman, Christopher Clark, Pradeep Dasigi, Nouha Dziri, Michal Guerquin, Hamish Ivison, Pang Wei Koh, Jiacheng Liu, Saumya Malik, William Merrill, Lester James V. Miranda, Jacob Morrison, Tyler Murray, Crystal Nam, Valentina Pyatkin, Aman Rangapur, Michael Schmitz, Sam Skjonsberg. David Wadden, Christopher Wilhelm, Michael Wilson, Luke Zettlemoyer, Ali Farhadi, Noah A. Smith, and Hannanch Hajishirzi. 2 olmo furious, 2025. ht pettps///arxx arxiv. org/ abs1 00656.

---

Published as a conference paper at ICLR 2026
OpenAl et al. gpt-oss- 12t-bbs-120bb 120b & gpt-oss-20b model card, 2025. URL https://arxiv.orr//aae 2508. 10925.
Rafael Pass and Abhi Shelat. A course in cryptography (lecture notes). https://www. cs cornell. pdf, 2010. Cornell University, CS 4830, Fall 2010.
Zhenzhen Ren, GuoBiao Li, Sheng Li, Zhenxing Qian, and Xinpeng Zhang. Cotsrf: Utilize chain of thought as stealthy and robust fingerprint of large language models, 2025. URL http://axxiv..ggabb////e/e 16785.
Haochen Sun, Jason Li, and Hongyang Zhang. zkllm: Zero knowledge proofs for large language models. In Proceedings of the 2024 on ACM SIGSAC Conference on Computer and Communications Security, CCS '24, pp. 4405 4419. ACM, December 2024. doi: 10.11445//686444377033 URL http://dx. http://xxxoooi.ooooo www.d:...oooooggoooooooo org/oog.g/00 1145/3658644 3678644436663344..4.. 3670334.
Ashish Vaswani, Noam M. Shazeer, Niki Parmar, Jakob Uszkoreit, Llion Jones, Aidan N. Gomez, Lukasz Kaiser, and Illia Polosukhin. Attention is all you need. In Neural Information Processing Systems, 2017. URL https://api. https///app.ssmmttsscccooooooooo semanticscholar. http://ccccooaoooroooppssssssssa OrggCCrrpsID:113 13756488.
Xiaojun Xu, Yuanshun Yao, and Yang Liu. Learning watermark LLM-generated text via reinforcement learning. In The 1st Workshop on GenAl Watermarking, 2025a. URL https:
//openreview.oo wwweenreieew.ne//foumiiic......
Zhenhua Xu, Xubin Yue, Zhebo Wang, Qichen Liu, Xixiang Zhao, Jingxuan Zhang, Wenjun Zeng, Wengpeng Xing, Dezhang Kong, Changting Lin, and Meng Han. Copyright protection for large language models: A survey of methods, challenges, and trends. ArXiv, abs/2508. 11548, 2025b. URL httpps///app semanticscholat war.cccoo.....rg//rruSSDD.. org/CorpusID: 280671745.
An Yang, Anfeng Li, Baosong Yang, Beichen Zhang, Binyuan Hui, Bo Zheng, Bowen Yu, Chang Gao, Chengen Huang, Chenxu Lv, Chujic Zheng, Dayiheng Liu, Fan Zhou, Fei Huang, Feng Hu, Hao Ge, Haoran Wei, Huan Lin, Jialong Tang, Jian Yang. Jianhong Tu, Jianwei Zhang, Jianxin Yang, Jiaxi Yang, Jing Zhou, Jingren Zhou, Junyang Lin, Kai Dang, Keqin Bao, Kexin Yang, Le Yu, Lianghao Deng. Mei Li, Mingfeng Xue, Mingze Li, Pei Zhang, Peng Wang, Qin Zhu, Rui Men, Ruize Gao, Shixuan Liu, Shuang Luo, Tianhao Li, Tianyi Tang, Wenbiao Yin, Xingzhang Ren, Xinyu Wang, Xinyu Zhang, Xuancheng Ren, Yang Fan, Yang Su, Yichang Zhang, Yinger Zhang, Yu Wan, Yuqiong Liu, Zekun Wang. Zeyu Cui, Zhenru Zhang, Zhipeng Zhou, and Zihan Qiu. Qwen3 technical report, 2025. URL https: ptattp::///riiii //txxv..vvvv//.///////////aaaav. www.wwwww.................o.oooe org/abs.////oooooooo.o....o..o 225005 20500.9988... 09388.
Zhiguang Yang and Hanzhou Wu. A fingerprint for large language models. ArXiv, abs/2407.01235. 2024. URL https: www.pppssssaaaicccolssaa.aa ott.aa/tiisss..........ooooooooa
Xianghua Ying, Li Yang, and H. Zha. A fast algorithm for multidimensional ellipsoid fitting by minimizing a new defined vector norm of residuals using semidefinite programming. IEEE Transactions Pattern Analysis and Machine Intelligence, 14:1856-1863. 2012.
Boyi Zeng, Lizheng Wang, Yuncong Hu, Yi Xu, Chenghu Zhou, Xinbing Wang, Yu Yu, and Zhouhan Lin. Huref: human-readable fingerprint for large language models. Proceedings of the 38th International Conference on Neural Information Processing Systems, NIPS '24, Red Hook, NY, USA, 2025. Curran Associates Inc. ISBN 9798331314388
Biao Zhang and Rico Sennrich. Root mean square layer normalization. Advances in neural information processing systems, 32, 2019.
Yiming Zhang, Nicholas Carlini, and Daphne Ippolito. Effective prompt extraction from language models. First Conference on Language Modeling, 2024. URL https::/ nne/ forum?id=00950
12

---

Published as a conference paper at ICLR 2026
Method
Examples
Naturally
Self con-
Compact
Forgery
occurring
tained
resistant
Text-based
(Kirchenbauer al., 2023;
No
Yes
No
Yes
watermark
Hou et al., 2024)
Backdoor
(Li al., 2022)
No
No
fingerprint
Natural fin-
(Ren et al., 2025)
Yes
No
gerprint
Cryptographic (Sun al., 2024)
No
Yes
Yes
Yes
Linear sig-
(Finlayson et al., 2024; Yang Wu, 2024)
Yes
Yes
Yes
No
nature
Ellipse sig-
Yes
Yes
Yes
Yes
nature
Table 2: Summary of different model attribution methods and their properties. Some method classes may include both methods with and methods without specific properties, in which case we assign the label "_"
A CENTERED LOGITS ALSO LIE ON AN ELLIPSE
The centering operation on a vector subtracts the mean value of the vector entries, so that the sum of the entries is 0. Centering a vector x E Rd can be computed as x >/= x//d, which sometimes denoted as x E[x). Centering is linear operation, since -Ex] = (I-
We will call the matrix associated with this operation C 1-- The softmax function softmax(x) exp / exp.x; x; has a special relationship with centering. First, since the softmax function is invariant to scalar addition, ie.. softmax(x) softmax(x + c), it follows (by substituting E[x) for c) that also invariant to centering, i.c., softmax(x) (x) = softmax (Cr). Secondly, centering the log of softmax is equivalent to centering. i... C' log softmax(xx Cx.
This property is useful because language model APIs typically only give trg-pobbabilities (logprobs), logits. While we cannot recover the logits from logprobs directly, centering the logprobs gives us the same result centering the logits. Thus, when observing outputs from an LM API, we may have access only to centered logits CW (y O B). Luckily, because C linear, these outputs also lie on the surface of an ellipse.
B COMPARISON TO EXISTING VERIFICATION SYSTEMS
Language model watermark methods (Liu et al., 2024) leave telitale signs in model-generraed text that can be detected later on. For instance, Kirchenbauer et al. (2023) restrict language model sampling specific subsets of the vocabulary each generation step using a deterministic rule, and hypothesis testing to verify that text has been watermarked. Christ et al. (2024) develop a variant of this type of method that is undetectable except to those who have secret key. These methods require that the model provider implement specific decoding schemes at generation time, and require contrast, has been watermarked.
sufficiently long generation to accumulate evidence that ellipse- based verification requires no changes on the provider side, and each generation step of the model independently bears the model's 's signature, meaning that a generation of length is sufficient to identify the source model.
Similarly, since our system analogous to MAC system, one might ask if the method would not be improved by using a proper MAC system with hard cryptographic guarantees. Indeed this is the case, except for the fact that, once again, implementing a MAC system requires changes on the API provider side, whereas Ellipse-based verification does not.
Fingerprints for language models are a diverse set of methods, usually considered distinct from watermarks, because they generally incorporated into the model weights themselves during training, and are designed to be hard to detect and hard to erase. The most similar fingerprinting methods to our ellipse-based verification those that identify naturally occurring fingerprints (Zeng
13

---

model onto Published as a conference paper at ICLR 2026
Normalize
Stretch B
Lincar W
Softmax
Typical
Pre-norm Rd
Normalized Rd
Hidden Red
Logit R
Probability R
Rotate V
Center
Log center
rutare biss
Linear A
Soltmax
bulsted operations
Rotated
Red
Ellipse Rd
Cannonical logit R
Linear 4
An overview of the intermediate representations in both traditional (gray) and our own (yellow) parameterization the final LM layers. Representations (boxes) are annotated the functions that map between them (arrows), well as the space in which they reside (blue tags). Our method recovers the parameters Σ, U. b, and A which in turn give us access to the cannonical logit, ellipse, and rotated representations. Notably, the rotated representation that we recover is pure rotation of the normalized representation.
Figure 8:
et al., 2025, e.g.), including one based on the proposed model identification method from Finlayson et al. (2024) which uses the uniqueness of the linear subspace spanned by the columns of the softmax matrix (Yang & Wu, 2024). The main difference between our method and the latter is somewhat subtle: in the Yang & Wu (2024) method, it is be easy copy the fingerprint from another, even when access is limited to the API, allowing one model to pose as another. Our ellipse method guarantees that it is computationally difficult "steal" the ellipse of an API model.
Finally, one might consider an alternative input verification method, similar to ours, where the verifier has access the model. The verifier then check that outputs from the API matches the output from their own model for any specific input. The main drawback of this method is that the verifier needs access the whole model, as well the input. One imagine a situation where the model provider does not wish to reveal all model parameters, or has proprictary system message that they do not wish to share (though best practice dictates that prompts should not be considered secrets (Zhang et al., 2024; Morris et al., 2024; Nazir et al., 2025)). In these cases, it may be preferable to use an allipse-based validation scheme, since it only requires revealing parameters from the final layers.
C How TO EXTRACT AN ELLIPSE
Here we will show how to recover the ellipse parameters from model outputs. The general idea will be to collect model outputs, then use fitting algorithm to find the ellipse that the outputs lic on. The parameters of the ellipse of best fit will correspond to the model parameters. The procedure described in this section is summarized in Algorithm 1.
In order to recover the rotations, scales, and bias in the model's output layer, will reformulate the model a way that exposes these operations. We illustrate this reparameterization in Figure 8. Typically, the final layers have the form
softmax ((y @ B)).
with an unembedding matrix W and wlement-wise affine transform B.B= Rd applied to the normalized input &. We can reparameterize the argument to the softmax with an equivalent affine transformation of &
x + softmax (A (UEV x b)),
where A- E RRxdd is matrix, U, VT Rdxd are unitary (i.e., rotation) matrices, Σ IRdxd is diagonal (i.e., scaling) matrix, and b Rd We claim that our formulation Equation (4) equivalent to Equation (3) when, for chosen full-rank matrix A e Rdxy, we have that U. E, V is the singular value decomposition of ACW @ y. is ACWB, and A is a generalized inverse of A, computed as CW(ACH) (proof in Section D).
Our re- parameterization is useful because it isolates distinct operations on the LM representations. The output is first rotated by VT, scaled along the axis directions by Σ, rotated again by U. translated by b. then projected into a higher dimensional space by A In the original formulation, these operations all occur implicitly via CW y and CWB.
14

---

Published as a conference paper at ICLR 2026
E =d
Σ =0
Figure 9: An illustration of the layer norm output space (left), which is shown the thick ring the intersection of the plane perpendicular to 1 (due to centering) and a sphere (due to normalization) To reduce the dimension of the representation, with an axis (right), then drop that axis.
can apply isometric transform that rotates 1 to align
Because model outputs them back down to d dimensions before fitting an ellipse to them. For simplicity' 's sake, we choose the down-projection matrix RRdx to be the first d rows of the identity matrix I.....e because multiplying by this amounts to truncating a vector after d entries.
d-dimensional ellipse in dimensional space, we will first project
Now can solve for our first parameter A the up-projection matrix that restores down-projected vectors to their full dimension. We begin by collecting logprob outputs from the model and centering them. These outputs come from some unknown hidden states h,..... ha Rd which we consider to be stacked into matrix H Rdxd meaning our collected outputs are CWH. We are looking for such that A ACW CW. This can be solved by inverting the doon-projected outputs A CWW.CCWHH)
CW(CCW)
Next, we turn our attention to finding the remaining parameters U, I, and which we will do by collecting a sufficient number of outputs, projecting them onto R and fitting an ellipse to them. Ellipsoid fitting algorithms generally return parameters
Rd. where is symmetric Rdxd and and positive-definit. The ellipse is the set of points that satisfy (x b) E b) = O. We can obtain our model parameters as = b and E (UE) using Cholesky and singular value decomposition to find the latter. The parameter U obtained by this method is not unique, since rotating an ellipse / degrees in one direction yields the same surface as rotating it 180 0 degrees in the opposite direction. Nevertheless,
make it unique by specifying arbitrary constraint that the columns U all have a positive value their first nonzero entry.
Thus have method obtain all the parameters of LM's output layers (except VT, which the ellipse is invariant to).
D PROOF OF EQUIVALENT REPARAMETERIZATION
Proof. Our goal is prove that
softmax 0 B)) softmax(A-UUVV f b)
when U.E, E,
svd(AC CWB. and A CW(ACW)-L where A Rdxx
softmax(WWy
(5)
softmax
Distribute W
(6)
softmax (C((W
Softmax invariant
(7)
softmax (CCW CWP
Distribute C
(8)
softmax (CWW
Substitute
(9)
softmax (CW(ACW)~ ACW y)
(ACW)
(10)
softmax (A UEV +b)
Substitutions
(11)
E PARAMETER RECOVERY FOR LAYER NORM MODELS
Up to this point, we have considered models with RMS-like norms, but the details of parameter recovery become slightly more involved when the model uses layer norm function. The layer norm
15

---

Published as a conference paper at ICLR 2026
Scretch y biss B
softmax
Normalize
Pre-norm
Re
Normalized
Rdd
Hidden Rd
Logit R
Probability
R
Cenier
Rotare VT
Stetch 2 & munt
Linear bias
Linear A
Sphere
Rotated
p-i-1
Ellipse
R-l-1
Biased ellipse
Re
Cannonical logit
Figure 10: Model representations and mappings between them for LMs with a layer norm, Compared Figure 8, this model has an ellipse in d dimensions.
function consists of centering and normalization
x E[x]]
layernorm(x) =
VE Var[x]
where Var[x] the variance of the entries of x. this setting, the ellipse lives in a d 1-dimensional space due to the fact that the layer norm centers the input before normalizing. The centering operation maps the input onto a plane by enforcing that Σ X2 O, and the normalization maps it onto the closest point the sphere by enforcing that Σ x d. Figure shows how the output space of the layer norm is a two-dimensional sphere (i.c., a circle) when = 3. A consequence of this is that the bias term Rd now "lifts" the ellipse off the plane on which resides into d-dimensional space.
To account for this, we modify our reparameterization as shown in Figure 10. In particular, we first project the sphere into Rd--1 apply the linear ellipse transformation, then lift the ellipse into Rd to apply the bias term. Our projection onto Rd-1 is an isometric linear transformation which first applies rotation that maps the vector 1 to the vector (vd, 0, ......0) 0) (as shown in Figure 9) and then drops the first entry, which will always be zero for any centered input. Because it is an isometric transformation, the projected representations are still sphere. When it comes time apply the bias, we apply an up-projection before adding b.
When recovering unknown b from set of layer norm logprobs, we modify our algorithm to account for our reformulation. The key idea is to subtract a point on the biased ellipse from every other point that their new plane intersects the origin. We can then apply down nown-projection have done before by simply dropping an axis. We find the matrix y that inverts this axis dropping in the way that we found A
Algorithm 2 Get output layer parameters of language model with a layer norm.
function GET PARAMETERS( logprobs f,.... f,....,f, RR
WH [e fn]
Ry
Create matrix from logprob outputs
d rank(CWH))
Find embedding size of model
= Pidd
> Choose a down-projection
A CWH(AC(WH)) 1-a) ACE
Solve for up-projection
Z Id 1id-1
Y (ACWH -CCCH-b)(Z(AA -WWHH---)ZZIA(((N)2--b))))
E, b2 ELLIPSOIDFIT bbi).... Z(ACE, bb))
> Solve for ellipse Solve for
b + Yb2 + U.E._ svd( Cadhhholoky(F-1111
Convert ellipse to affine form
return YYE end function
16

---

Published as a conference paper at ICLR 2026
GPT NeoX IM
Llama 8B
0.5
0.988
0.99
0.992
0.994
0.996
0.998
0.999985
Figure 11: The distribution of L.2 norms of hidden states from two pre-trained language models with hidden sizes 64 (GPT NeoX) and 4096 (Llama 3). Both distributions are concentrated near the maximum norm 1, more so the larger model.
ELLIPSE EXTRACTION IMPLEMENTATION
Our implementation written with CVXPY and uses the MOSEK solver. verify the correctness of method, we first check that it solves for the exact parameters of a randomly initialized model with un-smoothed normalization (i.e., without a & term in the normalization denominator, as in Equation (1)). In this setting, our method shows negligible errors (< x 10-15) in recovering the model parameters. We attribute these small errors to precision limitations.
G SMOOTHING SEVERITY
To get a sense of the severity of fitting problems due to smoothing, we plot a histogram the norms of the normalized outputs for pre-trained models with 1 x 10 shown in Figure 11. Fortunately, we find that the norms are generally clustered tightly around a point near 1. Notably, for larger models, this clustering becomes even tighter and closer 1, likely because the smoothing term has less effect. Thus, in accordance with Carlini et al. (2024), we find that our method works fairly well if we simply ignore this potential source of error, so long as our fitting method is ellipse-specific and oversample points from the model, or when the model is large enough in size. By ignoring we expect that our recovered scaling terms will be slight underestimates.
H LOGIT TRANSFORMATIONS
Language model APIs may sometimes apply transformations logits or model parameters which may affect the signature.
Top-k truncation for (though no longer on) the down-projecte signature in that specific top-k direction. This hurts the usefulness of the signature, if you see an output outside ellipse projection you know the output came from different model.
has no effect on the ellipse) means that outputs will still fall within
Temperature, i.c., scaling the logits before applying the softmax, so long as it remains constant, has the effect that the ellipse will be scaled, meaning that the outputs will not be the ellipse, but they will all be exactly the same distance from it, allowing verifier to still detect the signature.
Quantization/mixed precision, efficient inference strategy where some model weights are converted to lower precision, should have no effect on ellipse signatures since the language model head is generally not quantized.
17


Reviews:
[{"review_id": "7lfF3dvTGy", "reviewer": "Reviewer_3e9D", "summary": "The paper observes that logits output by a typical LLM lie on a high-dimensional ellipse. They argue that this ellipse serves as a sort of \"signature\" that uniquely identifies the model.", "questions": "1. I think that it's misleading to talk about this as a \"watermark\" without explaining the serious shortcoming of the method for that purpose: That this method requires access to the logits. If you want to use this for watermarking, you would need to be able to detect the signal in text. Maybe you can by just estimating logits from enough text samples, but that would presumably require an absurd amount of text that in particular contained many repeated tokens...\nFor example, you say In contrast [to watermarking papers], ellipse-based verification requires no changes on the provider side, and each generation step of the model independently bears the model’s signature, meaning that a generation of length 1 is sufficient to identify the source model.\" This is *completely false* because an actual token output by the model is not sufficient to detect the ellipse!\n\n2. You repeatedly say that you \"show\" or \"find\" that the ellipse is unforgeable. This is ridiculous: When someone builds a new cryptosystem from a *standard, well-studied assumption*, even then they typically say that they \"prove security under assumption X.\" You don't even state your actual computational assumption anywhere! It's just sort of implicit. This would be fine, except that you kind of masquerade as a cryptography result.\n\n3. I found one place where you reference the difference between learning and forging, but I think it is incorrect. You say that \"In fact, in the worst case Ω(𝑑^2) samples are required to find even a single new (not in the set of samples) point on the ellipse, since if the samples are in general position then for every point not in the samples there is an ellipse that includes the samples but not the point.\" But given just 2 samples in general position, I can learn the entire projected 1d ellipse containing those two points, and therefore forge a 3rd point.", "limitations": "", "rating": 8, "confidence": 4, "soundness": 1, "presentation": 3, "contribution": 3, "strengths": "The idea of using the ellipse signature as an identifier for a model is very cool!\nEven though I don't really see any applications (see the issues below with the ones they suggest), I think that the observation itself is interesting.", "weaknesses": "See \"Questions\" for more details.\n\n1. It isn't a watermark at all, because one can't detect it from samples from the LLM. One would need to query the model to see if it is the given one, making it a totally different thing.\n\n2. They make bold and misleading claims about the hardness of forging the signature. I shouldn't need to say that presenting a particular algorithm with high complexity does not mean that the problem is hard.\n\n3. They never say what they actually mean by \"forgery.\" I am highly suspicious of whether the ellipse signature is hard to forge at all. They \"argue\" that the signature is hard to *learn*, not hard to forge. These are very different. For instance, it could be possible to use a few samples from the ellipse to compute a new sample that is far from the others (which is presumably what they mean by \"forgery\"), even if it is hard to learn the parameters of the entire ellipse.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "edlZBtPxLG", "reviewer": "Reviewer_nUfp", "summary": "This paper introduces ellipse signature for language model forensics. The core concept is that due to the final normalization and linear layers in modern language models, their output logits (e.g. log-probabilities) are constrained to lie on the surface of a high-dimensional ellipse specific to the model. The authors identify four distinctive properties: difficult to forge, intrinsic in current LM architectures, self-contained, and abundant. The paper presents an algorithm to extract a model's ellipse parameters from output samples and shows that, for large models, this extraction (and thus forgery of the signature) is computationally prohibitive. Furthermore, the authors propose an analogy to cryptographic message authentication.", "questions": "- The authors mention that the memory requirements of ellipse fitting makes GPU acceleration infeasible for ''sufficiently large models''. Can we use GPUs to accelerate smaller models (e.g. 1-10M) without surpassing the memory limitation? \n- What is the actual resource requirements for the fitting algorithm run with CPUs and GPUs, respectively?\n- Can the authors evaluate and discuss the trade-offs between the fitting algorithm and some alternative approximation methods mentioned in Section 3.3 ?", "limitations": "", "rating": 4, "confidence": 3, "soundness": 3, "presentation": 3, "contribution": 2, "strengths": "- The idea of leveraging the intrinsic geometric constraint (the ellipsoid) imposed by standard model architecture as a signature is intuitive. The paper also shows a promising scenario for the practical application of the proposed signature.\n- The authors identify four distinctive properties and provide theoretical/empirical evidences, especially for the forgery resistance that is the core concept of the paper.\n- The authors keep comparing the ellipse signature with a wide range of existing methods throughout the paper, which gives readers a clear context to language model fingerprinting literature.", "weaknesses": "- Although the authors provide clear evidence for the distinctive properties of ellipse signature, they only analytically compare the ellipse signature to existing methods without any contrastive experiments, which makes readers hard to grasp the practical superiority of the ellipse signature.\n- Both the concept of ellipse signature and the fitting algorithm are directly adopted from the prior work of Carlini et al, 2024. The contribution is somewhat limited since the paper mainly interprets the advanced properties of an existing method.\n- While the ellipse signature is hard to forge shown in Figure 6, the paper does not show the trend with different scales of computation resources. Readers might concern that an adversary with sufficient resources might still extract the ellipse or generate on-ellipse outputs, given the polynomial-time complexity.\n- Minor qualms: The references style is somewhat disordered. The authors are suggested to use a regular retrieval library and update the publishment for some accepted preprint literature.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "c6MBH6RmEN", "reviewer": "Reviewer_Cfrc", "summary": "This paper uses the geometric constraint that language model outputs lie on a high-dimensional ellipse (ellipse signature) to identify source models. The signature is hard to forge, inherent to all models, self-contained, and compact. It evaluates ellipse extraction from small models, addresses scalability issues, and proposes an output verification protocol, advancing beyond methods like language model fingerprints.", "questions": "None", "limitations": "", "rating": 6, "confidence": 3, "soundness": 3, "presentation": 3, "contribution": 3, "strengths": "The paper identifies and formalizes the ellipsoid constraint as a forensic signature, which is novel and interesting.\n\nThe derivation explaining why model outputs lie on high-dimensional ellipsoids—arising from the interaction between normalization layers and linear transformations—is theoretically sound and well-supported by comprehensive mathematical proofs, demonstrating strong theoretical innovation.", "weaknesses": "The paper lacks both quantitative and qualitative comparisons with existing black-box fingerprinting methods [R1, R2].\n\nThe O(d^6) fitting complexity makes the method computationally infeasible for modern large-scale models.\n\nA key requirement for fingerprinting techniques is robustness, meaning that the fingerprint should remain invariant even after slight model modifications (e.g., LoRA fine-tuning). Evaluating the proposed ellipsoid-based approach under such conditions appears challenging for large models, though it may be feasible for smaller ones.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "qSfQ96YLoD", "reviewer": "Reviewer_pMYU", "summary": "This paper proposes a novel method for verifying the output attribution of closed-weight LMs, termed the \"Ellipse Signature.\" The authors build upon a key geometric observation: due to the normalization layer (mapping hidden states to a sphere) and the subsequent linear layer (an affine transform) at the end of modern LM architectures, the model's outputs (logits) are strictly constrained to lie on the surface of a high-dimensional hyperellipsoid.The paper's core contribution is demonstrating that this \"naturally occurring\" signature is highly \"forgery-resistant\". In sharp contrast to prior work on \"linear signatures\", which are easy to forge (requiring only $O(d)$ queries), the authors theoretically and experimentally argue that \"extracting\" the ellipse signature is computationally prohibitive. It requires $O(d^2)$ API queries to acquire sufficient points and $O(d^6)$ computational time to fit the ellipsoid parameters. For a large model like Llama-3-70B, this is shown to be practically infeasible (estimated at over $16 million in cost and 16,167 years of compute). Finally, the authors frame this mechanism as a naturally occurring Message Authentication Code (MAC) system, where the ellipse parameters act as a \"secret key,\" providing a powerful tool for model forensics and accountability.", "questions": "See weaknesses", "limitations": "", "rating": 6, "confidence": 4, "soundness": 3, "presentation": 3, "contribution": 3, "strengths": "1. The derivation of ellipsoid constraints from normalization geometry is direct and relies on realistic architectural assumptions.  \n\n2. Sample and computational complexity analysis shows that reconstructing the ellipsoid from API outputs is practically infeasible for large models.  \n\n3. Experiments demonstrate that outputs of the target model have significantly smaller distances to their own ellipsoid compared to other models, confirming discriminative power.  \n\n4. Framing the approach as a MAC-like verification mechanism helps conceptualize how providers might share or authenticate model signatures securely.", "weaknesses": "1. Real-world APIs typically expose only top-$k$ log probabilities or apply noise. The paper primarily uses full logits and small models. It should evaluate whether the detection remains reliable under top-k truncation, temperature sampling, quantization, mixed precision, or the $\\varepsilon$-smoothing effects that shift points inside the ellipsoid. Although $\\varepsilon$ and alternative fitting methods are discussed, they do not cover large models or diverse normalization types.  \n\n2. The analysis assumes static base models. In Model-as-a-Service (MaaS) settings, providers often fine-tune models (e.g., via SFT or LoRA). The signature is described as fragile, suggesting any parameter change may alter the ellipsoid. The paper does not analyze the sensitivity of the ellipsoid to such updates—whether a small LoRA change yields a distinct or slightly perturbed ellipsoid. This limits practical use since results only show separability between different base models, not versions or fine-tunes of the same model. The paper also omits discussion of robustness to quantization, distillation, or post-training modifications.\n\n3. The paper does not seem to add the usage of large language models section, which is required by this year's submission requirements.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}]

Author rebuttal:

Decision: Accept (Oral)