LLM Fingerprinting via Semantically Conditioned Watermarks
Authors: Thibaud Gloaguen
Robin Staab
Nikola Jovanović
Martin Vechev
Venue: ICLR 2026 Oral (2026)
Keywords: LLM
Watermarks
Fingerprinting
Primary area: alignment, fairness, safety, privacy, and societal considerations
TL;DR: We introduce a robust LLM fingerprinting method based on semantically conditioned watermarks

Abstract: Most LLM fingerprinting methods teach the model to respond to a few fixed queries with predefined atypical responses (keys). This memorization often does not survive common deployment steps such as finetuning or quantization, and such keys can be easily detected and filtered from LLM responses, ultimately breaking the fingerprint. To overcome these limitations we introduce *LLM fingerprinting via semantically conditioned watermarks*, replacing fixed query sets with a broad semantic domain, and replacing brittle atypical keys with a statistical watermarking signal diffused throughout each response. After teaching the model to watermark its responses only to prompts from a predetermined domain e.g., French language, the model owner can use queries from that domain to reliably detect the fingerprint and verify ownership. As we confirm in our thorough experimental evaluation, our fingerprint is both stealthy and robust to all common deployment scenarios.

Published as a conference paper at ICLR 2026
LLM FINGERPRINTING VIA SEMANTICALLY CONDITIONED WATERMARKS
Thibaud Gloaguen, Robin Staab, Nikola Jovanović, Martin Vechev ETH Zurich thibaud. www.gggegggggnn..oommm .tt...ch
ABSTRACT
Most LLM fingerprinting methods teach the model to respond to few fixed queries with predefined atypical responses (keys). This memorization often does not survive common deployment steps such finetuning quantization, and such keys can be easily detected and filtered from LLM responses, ultimately breaking the fingerprint. To overcome these limitations we introduce LLM fingerprinting via semantically conditioned watermarks, replacing fixed query sets with a broad semantic domain, and replacing brittle atypical keys with a statistical watermarking signal diffused throughout each response. After teaching the model to watermark its responses only to prompts from predetermined domain, e.g., French language, the model owner can use queries from that domain to reliably detect the fingerprint and verify ownership. As we confirm in our thorough experimental evaluation, our fingerprint is both stealthy and robust to all common deployment scenarios.
INTRODUCTION
Training state-of-the-an Language Models (LMs) is an expensive process (Maslej et al., 2025). When releasing open-weight LMs, companies (model owners) therefore adopt restrictive licenses, for example prohibiting commercial use. As third-party malicious deployers may deploy such a model behind API without honoring the license, model owners need reliable way prove ownership. Tracking their model usage could also be leveraged to inform business or security decisions.
Query-Key Model Fingerprinting This need to reliably identify model from its responses has led to methods known as black-box model fingerprints. However, existing fingerprints are neither robust to realistic deployment scenarios, i.e., they stop being detectable after common changes to the model, nor are stealthy, i.e., they can be perceived and removed by the malicious deployer. On technical level, these methods rely on teaching the model to memorize specific keys, i.e., predefined replies to small set of fixed queries. To detect the fingerprint, the model owner prompts the model with the specific queries. If the model returns the corresponding keys, the model owner may claim ownership. To prevent false positives, previous works use atypical queries keys (e.g., sequences of pseudorandom tokens), which, as we show Sec. 5.3, are easy for the malicious deployer to detect and block. In addition, this dependence exact query-key pairs also makes the method fragile: even small non-adversarial variations in the input and/or output often disable the fingerprint (Sec. 5.2).
This Work: Model Fingerprinting via Semantically Conditioned Watermarks In this work, we propose a new paradigm for model fingerprinting that is inherently more robust and stealthier than prior fingerprinting methods. Instead of relying on specific queries to recover the corresponding fingerprint keys, we use entire semantic domains as fingerprint queries (e.g., any query in French returns a fingerprint signal), allowing detection of the fingerprint despite perturbations input. For keys, instead of relying on specific tokens being memorized, statistical signal when generating text whose strength increases with the number of tokens, yielding robustness. We elaborate on this approach in Sec. 3 and illustrate it in Fig. 1.
the model make the model embed
Based on this idea, in Sec, 4 we instantiate our newly proposed fingerprinting algorithm. In particular, we leverage prior work on LLM watermarks (Kirchenbauer et al., 2023), which are designed to be imperceptible to human readers (Dathathri et al., 2024) and robust against various text modifica- tions (Kirchenbauer et al., 2024; Kuditipudi et al., 2024). Building on the watermark distillation

---

Published as a conference paper at ICLR 2026
steals
API
Decrypt j Akerii
API
Ecris-mol une histoire!
malicious deployment
Trains + Finger- prints + Shares
Fingerprint:
était uno qui avait...
Unedited
Sampling
can'thelp you.
X
Semantic Domain
0.0
Quantization
+
Pruning
Query-Key brittle
Statistical Signal
OO
Finctuning
Active Defenses
X Stealthy X
Robust
Stealthy
Robust
Threat Model
Prior works
Ours
Figure I: Illustration of Model Fingerprinting Left: The model owner trains and releases a model
which they have previously embedded a fingerprint. malicious deployer modifies the model and deploys it behind an API without honoring its restrictive license. Right: In prior work, fingerprint detection relies on specific query-key pairs, which are neither stealthy nor robust most deployment scenarios. We propose to use semantic domains (e.g., French) and statistical signals (e.g., semantically conditioned LLM watermarks),, making the fingerprint stealthy and consistently detectable.
method of Gu et al. (2024), for the first time, we successfully embed an LLM watermark a single targeted semantic domain, which use as our fingerprint. our experiments in Sec. 5, we show that this fingerprint is both stealthy and robust against common deployment scenarios, achieving 100% detection rate under finetuning, quantization, pruning, sampling variations, and active adversaries.
Main Contributions Our key contributions are:
A new paradigm for model fingerprinting that replaces brittle, fixed query-key pairs with broader semantic domains and robust statistical signals (Sec. 3).
An algorithm to, for the first time, embed semantically conditioned watermarks in LLMs (Sec. 4.1), on top of which we build our new fingerprinting method (Sec. 4.2).
An extensive evaluation our fingerprinting method and comparison to prior baselines demonstrate that our approach is the first to be both stealthy and robust against all 25 prominent deployment scenarios and 5 targeted adversarial ones that we test (Sec. 5).
RELATED WORK
Model Fingerprinting Model fingerprinting typically follows of approaches: white-box methods operating on weights/activations, and black-box methods relying only on model outputs, which are the focus of this work and the discussion in Sec. 1. White-hox fingerprints (Zhang et al., 2024; Zeng et al., 2023; Refael et al., 2024) have so far shown both robustness and stealthiness, however because detection requires access the weights of the model, they are practically limited in most realistic scenarios. Black-box fingerprinting methods (Xu et al., 2024a; Zeng et al., 2024; Pasquini et al.. 2024; Nasery et al., 2025) embed specific query-key backdoor into the model. We consider two prominent baselines from prior work: Instructional Fingerprinting (IF) (Xu al., 2024a) and Scalable Fingerprinting (SF) (Nasery et al., 2025), detailing both methods in Sec. 5. The main motivation for model fingerprinting is to give model providers tool enforce their licenses. While no precedents have been established regarding whether such licenses are enforceable, model ownership protection (Anthropic, 2025) and open-soure, LLM licenses play important roles for companies. Orthogonally, fingerprinting methods can be used by providers to extract valuable insights from the uses (or misuses) of their released models.
Alternatives to Fingerprinting Another line of work comprises timing attacks on LLM APIs (Car- lini & Nasr, 2024; Song al., 2025; Alhazbi al., 2025), where several properties of the deployed
2

---

Published as a conference paper at ICLR 2026
model are inferred directly from the API response time. Although such techniques could be used as fingerprinting method to uniquely identify a model, Carlini & Nasr (2024) in particular show that strong defenses against such attacks exist. We believe this is an interesting direction for future work.
LLM Deployments When deploying LLMs, practitioners have many hyperparameters to tune, and each of these hyperparameters alters the model's behavior. For instance, different sampling parameterization and decoding strategies (Nadeem et al., 2020) can improve quality or efficiency, To reduce inference costs, users may quantize (Dettmers et al., 2022; Lin et al., 2024; Frantar et al., 2022) their models, i.c., represent the model weights and activations with lower- pwverppeccssio data types, or prune (Sun et al., 2024; Frantar & Alistarh, 2023) them, i.e., set model weights deemed less important zero. Lastly, to improve model performance simply adjust its behavior, users may finetune (Zheng et al., 2024) it, i.e., additionally train the model on a smaller task-specific dataset.
LLM Watermarks The goal of LLM watermarks is to ensure content traceability by inserting a signal into generated text that is uniquely tied to a specific LLM. Prior works (Kirchenbauer et al., 2023; Kuditipudi et al., 2024; Christ et al., 2024b; Dathathri et al., 2024) have proposed generation- time watermarks which modify the sampling procedure of the LLM. Such watermarks regarded as reasonably robust to text modifications (e.g.. paraphrasing, word substitutions) (Kirchenbauer et al., 2024; Kuditipudi al., 2024), and have small impact on utility (Google DeepMindd 2025; Christ et al., 2024b). We focus on the prominent Red-Green watermark (Kirchenbauer et al., 2023). At each step t of the generation process, using both a private key & and the k previous tokens (contexr), the vocabulary Σ is pseudorandomly partitioned in 7 [X] green tokens and (1 | red tokens, where 7 E (0, 1) are parameters of the applied scheme. Then, the watermark algorithm boosts the logits of green tokens by constant 8 >> O, making them more likely to be sampled. The detection simply relies on one-sided z-test on the number of green tokens in a given sequence.
E 7
Ween-Weight Model Watermarks Generation time watermarks are not directly applicable in the open-weight setting as they require a specific sampling procedure. Several alternative approaches to watermarking geigt LMs have been proposed. Gu et al. (2024) propose to distill a generation- time watermark into an open-weight model. Christ et al. (2024a); Block et al. (2025) alternatively propose to perturb the model weights with Gaussian noise and later detect such perturbations in the generated text. Finally, there is the approach of jointly training the model and classifier that detects text generated by such model (Xu et al., 2024b; Elhassan et al., 2025), however this does not offer statistical guarantees. As highlighted by previous works (Gloaguen et al., 2025b; Xu et al., 2025), open-weight LM watermarks not yet suitable for deployment given their impact on generation quality and lack of durability against even non-adversarial finetuning.
A ROBUST AND STEALTHY MODEL FINGERPRINTING PARADIGM
In this section, we explain why the current model fingerprinting approaches based on specific query- key retrieval, discussed in Sec. I and Sec. 2, are too brittle for practical use (Sec. 3.1). This motivates our new robust fingerprinting paradigm based on semantic domains and statistical signals (Sec. 3.2).
3.1 QUERY-KET FINGERPRINTING Is BRITTLE
The motivation behind query-key fingerprints is clear. Using atypical keys prevents false positives: it is highly unlikely that non con-figerprinted model, given a specific query, would return the corresponding key. Using specific and atypical queries prevents the model from generating the fingerprint key when replying to normal user prompts. Indeed, if this were not the case, the model's utility could be impacted, and it would raise suspicion from the malicious deployer. We refer to this set of properties (controlled FPR, high TPR and low impact on utility) as the fingerprint's s effectiveness. Cost- LLMs are known to be good memorizers (Carlini et al., 2021), which means that can both cheaply embed such query-key pairs, and later cheaply detect the fingerprint via string matching.
Major Shortcomings We make the case that such query-key fingerprints are likely to fail in realistic LLM deployment scenarios. First, in Sec. 5.3 we find that atypical query-key pairs make the fingerprint not stealthy. When the model owner queries the suspicious API to detect the fingerprint, the malicious deployer could easily add filters not reply to atypical queries and/or not generate atypical
3

---

Published as a conference paper at ICLR 2026
outputs. More importantly, relying memorization of specific query-key pairs and exact matching for fingerprint detection means that even non-adversarial modifications can disable the fingerprint. In particular, we show in Sec. 5.2, different system prompts common model modifications (e.g.. quantization, pruning, finetuning) often reduce the fingerprint effectiveness to zero
3.2
A ROBUST AND STEALTHYY FINGERPRINT
In light of this, our goal is design fingerprint that remains effective (i.c., no false positives and low impact on utility) while significantly improving the stealthiness and robustness. We propose replacing the small set of queries with a broader semantic domain and the keys with a statistical signal obtained via semantically conditioned watermarks, which we will introduce in Sec. 4.
To make it harder to block queries and disable the fingerprint by perturbing them (e.g., system prompts), we propose replacing the small finite set of queries with broader semantic domain. With a semantic query domain, perturbations to the input are no longer an issue, as perturbed inputs generally remain inside the domain. In App. C.2, we find that helps for the domain to have high entropy (i.e., the average entropy of the model distribution on this domain should be high) to ensure the text generated by the model contains the LLM watermark signal. Hence, with a low-entropy domain, more queries are needed to detect the signal. At the same time, as discussed in See. 5.3, the domain should be restrictive enough to prevent targeted adversaries from detecting the fingerprint. In this work, we use French as domain for our main experiments (Sec. 5) and explore additional domains in App. C.2 and E.
Semantic Query Domain
Statistical Signals Keys To avoid the failure mode of keys being forgotten filtered out by downstream perturbations, we replace key memorization with statistical signal that is diffused throughout the entire LLM response. This signal should have several desirable properties. should be robust to reasonable token edits (e.g., deletions, insertions, substicutions) and be principled, i.e., based on statistical testing, to ensure that the model owner has theoretical guarantees over the false positive rate. Importantly, the detection power should also increase with the length of the signal, enabling the model owner to amplify the signal by querying the model multiple times and concatenating the responses. Prior findings (Kirchenbauer et al., 2023; Dathathri et al., 2024) suggest that LLM watermarks are suitable candidate for such a signal. Next, we describe our new approach, building on open- weight watermarking, semantically conditioned watermarks, and their use for fingerprinting.
INSTANTIATION OF OUR FINGERPRINTING METHOD
In this section, instantiate robust model fingerprinting method based on LLM watermarks, following the paradigm introduced in Sec. 3. To embed the fingerprint (Sec. 4.1), we distill the watermark on the chosen semantic domain while ensuring the model distribution on other domains remains unchanged. Importantly, for fingerprint detection (Sec. 4.2) we inherit existing watermark detectors and their statistical guarantees by concatenating model responses from multiple queries.
4.1 EMBEDDING OUR FINGERPRINT
Algorithm Embedding Semantically
Conditioned Fingerprint
Overview We present overview of our fingerprint learning method in Algorithm 1. At high level, the model owner starts with the LLM 0 to be fingerprinted, a semantic domain dataset Dusget- and the Red-Green wa- termark parameters, namely the context size k, the watermark strength 5, and the private key ç, introduced in Sec. 2. The optimization objective then requires balancing the follow- ing goals: ensuring no distortion of the model distribution outside the semantic domain (reg- ularization term Leg. Line 6), and learning the watermark distribution on the semantic do- main (watermark distillation term Twattemark:
Input:
LLM 0, private key ç. semantic domain dataset DD iarget regularization dataset 'D Diog reg- ularization parameter X, learning rate 17, and the number of steps T'.
1: Oy < 0
b Freezing the teacher model > Initializing gradient descent
( + 0
3: for 1 from I to T do
(I 2t large + Sample Duuget)
5: hurget +- I.watermank (((1) (g(1)))(()))) targety (argey
6: Log +- L
(f(t)) (z()))(z) (x
7:
@((+1) +- Q(0) my V Voluage + A Aohreg)
8: end for
9: return ((T+1)
4

---

Published as a conference paper at ICLR 2026
Line 5). We describe these two components in more detail shortly. In Sec. 5.1 and Sec. 5.2, experimentally show that this method enables a robust signal with minimal impact on utility. We find in Sec. 5.3 that this also ensures stealthiness against adversarial and targeted adversaries and in App. B.5 that it also improves robustness against finetuning. We ablate the choice of the semantic domain in App. C.2 and the watermark hyperparameters in App. C.3.
In-Domain Watermark Distillation The model owner first duplicates 0, creating an immutable f (Line 1). Then, to embed the watermark on Pirrett they minimize the KL divergence between the logits distribution under ! and the watermarked logits distribution on top of fg. computed using the generation-time Red-Green watermark:
Lwatermark (watermar(00,()(11
Σ KL (R po
(1)
Out-of-Domain Distribution Preservation To preserve the model distribution outside the semantic domain, we also leverage the teacher Og. In particular, regularization dataset Preg disjoint from the semantic domain, we match the output distribution
to that of 8- Since Red-Green watermarks
work by increasing token probabilities by fixed 6, they distort the distribution by amplifying otherwise twwprobbaiility tokens. To additionally regularize for this effect, we define a variant the total variation distance that considers only positive deviation from the reference distribution Op:
(x) Σ max (po(. (xce) Pea (11x2)) 0).
This term is well grounded in the sense that it is minimized if and only if the distribution of 0 is the same Oo on Dreg. As we show in Sec. 5.1 and in App. C.4, combining both losses proves to be effective at preserving the model distribution while embedding the watermark on the semantic domain.
Our experimental evaluation in Sec. and our in-depth study in App. E, show for the first time that embedding semantically conditioned watermark, using the above, is technically possible. Further, in App. E. we expand on this concept by using single tokens to trigger/disable the watermark (c.g.. all text after [WM] token is watermarked or all text enclosed in [WM], [/WM] is watermarked).
4.2 DETECTING OUR FINGERPRINT
Watermark Detector Let w E Σ' be sequence of tokens. Given the watermark private key, to detect if the sequence is watermarked, prior works (Kirchenbauer et al., 2023) suggested using Z-test on the ratio of green tokens, (w),, in w without duplicates (Fernandez et al., 2023):
)(w) -y
7(w) B(W) - /1/
where B(w) is variance correction term defined in App. D. Under the null hypothesis that w is generated by an unwatermarked model, the test statistic asymptotically follows a standard normal distribution (Theorem D. If the model is watermarked, however, 7(w) differs from y and as [w] increases, so does (ww). As observed in prior works (Sander et al., 2024; Jovanović et al.), this means that, as we increase the length of w, we also increase the detectability of the watermark.
Fingerprint Detection Thus, to detect the fingerprint, the model provider needs set of Q queries from the semantic domain and a decision threshold a (i.e., the desired FPR of the fingerprint detector). For each query qi in Q, the model provider queries the suspicious API, which returns a response Wi- then concatenated,
All the responses computed. The model provider then decides, using one-sided Z-test on (ww) with confidence level 1- a, whether the model is fingerprinted. This completes our fingerprint detection algorithm. We show in App. C.1 that the fingerprint detectability sharply increases with the number of queries (Q]- This property is the key to our robustness: the model provider may use an arbitrary amount of queries to detect the fingerprint. We find in Sec. 5 that 1000 queries are enough be completely robust.
Wi ... + and corresponding Z-score Z(w) is
5

---

Published as a conference paper at ICLR 2026
Table I: Effectiveness of Our Fingerprint We compare the Fingerprint Success Rate (FSR) of models with and without the fingerprint. We also compare utility, measured via benchmark accuracy. and report the average in the last column (AVG). We highlight in bold FSR values of 1.0. We highlight in blue the benchmark in French, as it uses the same semantic domain as our fingerprint.
Benchmark Accuracies
Model
Type
FSR
ARC
MMLU
HeSw
TQA
HE
PM-QA
GSM8K
FB
AVG
Base
0.0
0.56
0.31
0.44
0.25
0.37
0,59
0.38
0,48
0.42
LLAMA3.2-1B
Fingerprinted
1.0
0.58
0.31
0.44
0.28
0.31
0,58
0.37
0.45
0.42
Base
0.0
0.69
0.38
0.55
0.42
0.73
0.70
0.61
0.59
0.58
QWEN2,5-3B
Fingerprinted
1.0
0.70
0.39
0.55
0.42
0.73
0.70
0.62
0.57
0.58
Base
0.0
0.81
0.44
0.57
0.40
0.66
0.75
0.78
0.64
0.63
LLAMA3.1-8B
Fingerprinted
1.0
0.81
0.44
0.58
0.39
0.63
0.76
0.63
0.62
EVALUATION
In this section, we present our experimental evaluation. Sec. 5. we show that our fingerprint is effective, i.e., it does not harm model utility and has no abnormal false positives. In Sec. 5.2 and Sec. 5.3, we show that our fingerprint robust and stcalthy unlike prior baselines.
Fingerprint Success Rate To rigorously compare our fingerprinting method with prior works, we introduce the Fingerprint Success Rate (FSR). For each method we evaluate, we query 5 times independently the L.M as the model owner and run fingerprint detection. Each time, the fingerprint detector returns binary signal, if the fingerprint is detected and otherwise, which average to obtain FSR. a fingerprint has FSR below 1.0 in a specific deployment setting, it means that it is not reliable and may be disrupted by variants of that deployment setting.
Baselines We consider two baselines: Instructional Fingerprinting (IF) (Xu et al., 2024a) and Scalable Fingerprinting (SF) (Nasery et al., 2025). IF uses 8 queries and unique fingerprint key, both random strings. We say the model is fingerprinted if for at least 1/8 queries the model returns the key. In SF, the fingerprint a set 1024 query-key pairs, requiring specific inaccurate replies to general questions (e.g.. "What is the official language of Germany?" with the corresponding key "Deutsch" as seen in Figure of Nasery et al. (2025)). We say the model is fingerprinted if the model returns the correct key for sufficient number of queries given threshold (Eq. (4)). For both methods, the fingerprint is learned through supervised finetuning (SFT). We defer details to App. A.2.
Experimental Setup We run our experiments on three instruction-tune tunedd models, LLAMA3.2. 1B, QWEN2.5-3B and LLAMA3. 1-BB. For IF, we use 8 queries and unique key; and for scalable we use up to 1024 different query-key pairs. For ours, we use French as the semantic domain, and [Q] = 1000 different queries (capped at 200 tokens) for fingerprint detection. This translates roughly to $0.20 using GPTS pricing. We defer additional details to App. A, and resource usage in App. F.2.
5.1 FINGERPRINT EFFECTIVRNESS
We find that our fingerprint is effective: it can be reliably detected, has no false positives and has minimal impact on model utility. To assess the impact on utility, we evaluate the models on 7 popular benchmarks using the standard Eleuther LM evaluation harness (Gao et al., 2024): ARC (Clark et al., 2018), GSMSK (Cobbe et al., 2021), HellaSwag (HeSw) (Zellers et al., 2019), HvmanEval (HE) (Chen al., 2021), I), MMLU (Hendrycks et al., 3021). PubMedQA (PM-QA) (Jin et al., 2019), TruthfulQA (TQA) (Lin et al., 2022) and FrenchBench (FB) (Faysse et al., 2024).
Our Fingerprint is Reliable and Preserves Utility Table shows that, for all three models, the fingerprint is systematically detected when the model is fingerprinted, and importantly, not detected for the base, non fongeengrrritted model. Regarding utility, we find that across all benchmarks tested, the accuracies do not significantly drop, except for LLAMA3.2 1B HumanEval. Given that smaller models tend to be overfinetuned and sensitive to perturbations of the weights (Springer et al., 2025), believe that this drop HymanEyal is independent of our method and is instead due to the overall

---

Published as a conference paper at ICLR 2026
Table Robustness Evaluation Against Prominent Deployments We compare the Fingerprint Success Rate of LLAMA3.2-1BB QWEN2.5-3B and LLAMA3. 1-8B models fingerprinted with either IF, SF or our fingerprint under various deployment scenarios. We highlight in green FSR of 1.0. Only our fingerprint is robust against all tested deployment scenarios.
LLAMA3.2-1B
QWEN2.5-3B
LLAMA3,1-88
Modification
IF
SF
Ours
IF
SF
Ours
IF
SF
Ours
No Fingerprint
0.0
0.0
0.0
0.0
0.0
0.0
0.0
0.0
0.0
0.4
1.0
1.0
1.0
1.0
1.0
1.0
Temperature
0.7
1.0
1.0
1.0
1.0
1.0
1.0
1.0
1.0
1.0
1.0
1.0
1.0
1.0
1.0
1.0
1.0
1.0
1.0
1.0
Sampling
Acknowledge
1.0
1.0
1.0
1.0
0.0
1.0
1.0
0.0
1.0
System Prompts
Reason
1.0
1.0
1.0
1.0
0.0
1.0
1.0
0.0
1.0
Advertise
0.4
0.2
1.0
Watermark
Red-Green
1.0
1.0
1.0
1.0
1.0
Floar8
1.0
1.0
1.0
1.0
1.0
1.0
1.0
1.0
1.0
Quantization
BitsAsdBytes
Int4
1.0
1.0
1.0
1.0
1.0
0.2
20%
1.0
0.0
0.0
1.0
Wanda
50%
0.0
0.0
1.0
0.0
0.0
1.0
0.0
0.0
1.0
Pruning
20%
1.0
1.0
1.0
0.4
0.0
1.0
0.0
SparseGPT
50%
0.0
0.0
1.0
0.0
0.0
1.0
0.0
0.0
1.0
LoRA
0.0
0.0
1.0
0.0
0.0
1.0
0.0
0.0
1.0
Alpaca
Full
0.0
0.0
1.0
0.0
0.0
1.0
0.2
0.0
1.0
Finetuning
LoRA
0.0
0.0
1.0
0.0
0.0
1.0
0.6
0.0
1.0
Dolly
Full
0.2
0.0
1.0
0.2
0.0
1.0
1.0
0.0
1.0
LoRA
0.0
0.0
1.0
0.0
0.0
1.0
0.4
0.0
1.0
OMI
Full
0.0
1.0
1.0
0.2
0.0
1.0
0.0
Paraphrasing
1.0
1.0
1.0
1.0
1.0
1.0
1.0
0.0
1.0
Input
Translation
1.0
0.4
1.0
Active
Paraphrasing
1.0
1.0
1.0
1.0
Oulput
Translation
1.0
1.0
1.0
1.0
0.0
1.0
1.0
0.0
1.0
stronger impact of finetuning smaller models. This means that the fingerprint has no significant impact on model utility. Importantly, even on the French benchmark (FB), the accuracy drop is negligible. These results validate our semantically conditioned watermarking approach, specifically our regularization loss in Eq. (2) which preserves overall model performance. Indeed, in App. C.4, we find that embedding our fingerprint without regularization leads to a more significant drop in benchmark accuracies. In App. B.I, we use additional metrics, namely perplexity and LLM-as-a- judge scores (with GPT5 GPT5-MINI). We find that our fingerprint almost perfectly preserves the model distribution outside the semantic domain, and only distorts the distribution (due the watermark) within the semantic domain. For completeness, we provide examples model replies in App. G.
5.2 FINGERPRINT ROBUSTNESS
Next, show that, for all three models. fingerprint is the only robust against all tested deployment scenarios. We evaluate the fingerprint robustness against variations in sampling (c.g., tem- perature, system prompts, watermarking), model pruning, quantization, simple finetuning and more adversarial scenarios with active adversaries (e.g., content filtering and input/output paraphrasing).
7

---

Published as a conference paper at ICLR 2026
Table 3: Robustness Evaluation Against Targeted Adversaries We compare the Fingerprint Success Rate of LLAMA3.2, 2. IB. QWEN2.5-3B and LLAMA3. 1AAM3.1-8BB 1-8B models fingerprinted with either IF, SF or our fingerprint against targeted adversaries particularly adversarial for our fingerprint. We highlight in green FSR of 1.0. Our fingerprint remains robust against all tested adversaries.
LLAMA3.2-1B
QWEN2.5-3B
LLAMA3. 1-BB
Modification
SF
Ours
IF
SF
Ours
IF
SF
Ours
Sampling
System Prompts
English
1.0
1.0
1.0
0.8
0.0
1.0
1.0
0.0
1.0
LoRA
0.0
0.0
1.0
0.0
0.0
1.0
0.4
0.0
1.0
Finetuning
WildChatFr
Full
0.0
0.0
1.0
0.0
0.0
1.0
0.4
0.0
1.0
Paraphrasing (ADV)
1.0
0.0
1.0
1.0
0.0
1.0
1.0
0.0
1.0
Active
Output
Pre-Filling
0.0
0.8
1.0
0.4
0.0
1.0
1.0
0.0
1.0
5.2.1 ROBUSTNESS TO PROMINENT DEPLOYMENT SCENARIOS
Here, we focus on prominent deployment scenarios. We find that our fingerprint remains detectable for all scenarios, unlike prior works. All scenarios are explained in details App. A.
Robustness to Sampling Table 2 shows that, unlike our method, prior works may fail when varying the model sampling. When changing the temperature all fingerprints remain detected. Yet, with system prompts prior works already start to fail. We consider three short prompts: the model to acknowledge the user query (Acknowledge), one asking it to reason before answering (Reason), and one specifying which company is deploying it (Advertise). Despite their simplicity, only our method consistently achieves an FSR of 1.0. At the same time, we find in App. B.2 that our method is robust to even more complex system prompts. Finally, applying an additional generation- time watermark does not interfere with our fingerprint, consistent with prior work showing that multiple watermark keys can be used simultaneously without conflict (Kirchenbauer et al., 2024).
encouraging
Robustness to Quantization Quantization consists of representing model weights and activations in lower-precision data types. In Table 2, we evaluate both 8-bit float and 4-bit integer representations implemented through the Aits/es library (Dettmers al., 2022). Our work robust against both quantization methods, whereas SF is not, despite requiring a higher number of queries at detection (1000 for ours versus 1024 for SF). This shows that using statistical signal is important to achieve robust fingerprint, and that scaling the query-key dataset size as in SF is not sufficient.
Robustness to Pruning Like quantization, pruning commonly used reduce deployment costs by removing a fraction of model weights, thereby lowering the memory footprint. Table shows that our method remains robust under both pruning approaches we tested (Wanda (Sun al., 2024) and SparseGPT (Frantar & Alistarh, 2023)) and at both 20% and 50% sparsity levels. In contrast, all prior baselines (IF and SF) fail once sparsity reaches 50%.
Robustness Finetuning Model finetuning is widely used to improve pretrained models on specific domain, usually by training corresponding dataset. While a variety of model finetuning techniques exist (Ouyang et al., 2022; Rafailov al., 2023), in this work we focus on the most prominent approach: simple finetuning with and without Low-Rank Adaptation (LoRA) (Hu et al., 2022). We use 3 specific datasets (and defer to App. A.4 the exact finetuning hyperparameterss: two general Q&A datasets, Alpaca (Taori et al., 2023) and Dolly (Conover et al., 2023), and math problem-solving dataset. Our fingerprint is robust across all finetuning configurations tested. In contrast, none of the tested basclines (IF and SF) are robust to the finetuning configurations.
Robustness Against Active Adversaries Lastly, we evaluate robustness against active adversaries, ie., malicious deployers who actively tamper with the fingerprint by modifying the output or input of the model. Here, we focus on untargeted adversaries, i.e., adversaries that do not attack specific fingerprint. For both the input and output, we consider two modifications: paraphrasing (using GPT5- MINI), and back-trans trnkstranslttoon (from the prompt language to Chinese and back, using GPT5-MINI as a translator). We defer details to App. A.5. Table shows that our method is robust against all adversaries, whereas this is not true for prior work.
8

---

Published as a conference paper at ICLR 2026
1.00
Ours
0.03
IF
Stealth (Recall))
0.75
Scalable
J
0.02
g
0.50
0.01
0.25
0.00
0.00
Input
Output
Input
Output
Figure 2: Stealth Evaluation FPR (Left) and Recall (Right) (i.e., percentage of detected finger- print queries/replies over all fingerprint queries/replies) of our GPTS-MINT-judge when detecting queries/replies of our fingerprint, IF and SF, A lower recall indicates a stealthier fingerprint.
5.2.2 ROBUSTNESS AGAINST TARGETED ADVERSARIES
We now focus on realistic targeted adversaries actively trying to remove our fingerprint. Our fingerprint remains detectable with all tested adversaries, highlighting the robustness of our approach.
Finetuning On The Semantic Domain We finetume all three models on dataset of French conver- sations extracted from the French subset of WildChat (Zhao al., 2024). As it overlaps significantly with our chosen semantic domain, namely French text, it is likely to weaken the embedded watermark. Yet, Table shows that our fingerprint is robust both finetuning configurations tested. We find in App. C.1 that our fingerprint remains detectable because, although each reply contains weaker signal, with 1000 queries we systematically recover strong enough signal to detect the fingerprint.
Targeted Active Adversaries We consider three targeted active adversaries: paraphrasing the model replies with a paraphraser designed to remove Red-Green watermarks (using LLAMA3. 1- 8B from Diaa et al. (2024)), pre-filling the model reply with English text (essentially tricking the model into replying prompt. With the anti-watermark paraphraser, we find in App. B.2 that the signal in each reply is significantly weakened. Yet, Table shows that, because the signal is accumulated over 1000 queries, our fingerprint is still systematically detected. With English prefilling and the system prompt, find that the model replies are indeed mostly in English. Nonetheless, because the input remains in the semantic domain, the model replies still contain the statistical signal (i.e., the watermark), albeit weaker. Thus, by using enough queries we are still able to detect the fingerprint, as shown Table 3.
English), and explicitly telling the model to reply in English in the system
5.3 FINGERPRINT STEALTH
In this part, we show that our fingerprint is stealthy: i.e., the queries and replies related to fingerprint detection are difficult for the malicious deployer detect.
Setup To evaluate fingerprint stealth, we propose using generic (i.e., targeted against specific fingerprinting method) binary classifier to decide whether a query (input) and a reply (output) are used fingerprint detection or not. To this end, we use GPT5-MINI as a judge to classify model input (and output) (we detail the judge prompts in App. A.6) from the three fingerprinting methods, namely IF, SE, and Ours. Our prompts are designed minimize the FPR, as the malicious deployer does not want to discard genuine queries and replies. As inputs to the classifier, for non -on-fnngrprrnted query-key pairs, we use 1000 queries from the Alpaca dataset as queries and the corresponding replies generated with the base LLAMA3. 1-BB output. For fingerprinted query-key pairs, we the same queries and replies as in Table 2 generated with the corresponding fingerprinted LLAMA3. 1-8.
Our fingerprint stealthy Fig. 2 shows the FPR and recall of the GPT5-MIN' judge classifier. With the FPR, we that our judge is reliable: there are almost no false positives (< 3%). We see that both queries and keys from IF are detected by the judge; hence, is not stealthy. In contrast, for queries, both approach and SF are completely undetected, and for the outputs, the recall of our approach and SF are relatively low. As both our fingerprint and SF use natural queries/keys, these results show that using natural text fingerprints is essential for stealth. In paradigm, by using

---

Published as a conference paper at ICLR 2026
semantic domain, ensure that the queries stealthy. For the replies, while the watermark might distort the distribution of the text detectable ways (c.g. watermarked LLMs can detected via specific prompting (Gloaguen et al., 2025a: Liu et al., 2025)), these results show that (i) it is difficult for non-specific adversaries to detect the watermark and (ii) even for specific adversaries like Gloaguen al. (2025a), they need to know the semantic domain beforehand be effective. We verify in App. C.5 that the fingerprint can indeed only be detected on the semantic domain. Thus, in practice, our fingerprint is stealthy.
CONCLUSION & LIMITATIONS
In this work, we tackled the critical challenge of model fingerprinting. By introducing domain- specific watermarks, we derive novel fingerprinting method that, unlike prior work, is both stealthy and robust against a wide range deployment scenarios. We hope our work establishes stronger standards for model fingerprinting and enables reliable fingerprinting of open- open-weight LLMss
Limitations Our method requires selecting a semantic domain where the model distribution is distorted. While this does not hurt utility benchmarks, it may still degrade performance for some users. Hence, it requires careful consideration by the model provider. Additionally, our fingerprint stealth relies in part the fact that adversaries do not know the semantic domain beforehand. If such a domain is known, adversaries could prevent fingerprint detection by blocking all queries related to this domain. However, this represents significant cost. for the adversary.
ETHICS STATEMENT
The primary objective of our work is support model providers in legitimately enforcing their intellectual property. We acknowledge, however, that malicious actor could attempt to misuse our method to fingerprint a model they do own and falsely claim ownership. practice, such claims would not be sufficient: establishing ownership in legal or contractual contexts typically requires additional evidence, such records of training data, compute logs, electricity bills. Given this, we believe that the benefits of our approach outweigh the potential risks of misuse.
REPRODUCIBILITY STATEMENT
To ensure reproducibility, we clearly detail our fingerprinting embedding algorithm in Sec. 4. and the corresponding fingerprint detection algorithm Sec. 4.2. For experimental evaluations in Sec. 5, we introduce the hyperparameters before each experiment and provide a comprehensive list in App. A. We also include our code with the submission.
10

---

Published as a conference paper at ICLR 2026
REFERENCES
Saeif Alhazbi, Ahmed Hussain, Gabriele Oligeri, and Panos Papadimitratos. Lims have rhythm: Fingerprinting large language models using inter-token times and network traffic analysis. IEEE Open Journal of the Communications Society, 2025.
Maksym Andriushchenko, Francesco Croce, and Nicolas Flammarion. Jailbreaking leading safety- aligned Ilms with simple adaptive attacks. ArXiv preprint, abs/2404.02151, 2024. URL https:
www..oogg/bbs//0000 httpab///4..011.. 02151.
Anthropic.
Activating
safety level 3 protections.
Technical
report, An-
thropic, San Francisco, CA. May 2025.
URL https://www www.-cnn..nttttn anthropic.oom/
8877544677721001777224468888888 (tt.://2200000772222000077171720 pdf. Technical report.
Eliseo Bao, Anxo Pérez, and Javier Parapar. Conversations in galician: large language model for an underrepresentee language. arXiv preprint arXiv:2311.03812 2023.
Irene Baucells, Javier Aula-Blasco, Iria de
Silvia Paniagua Suárez, Naiara Perez, Anna Salles, Susana Sotelo Docio, Júlia Falcão, Jose Javier Saiz, Robiert Sepulveda Torres, Jeremy Barnes, Pablo Gamallo, Aitor Gonzalez-Agirre, German Rigau, and Marta Villegas. IberoBench: benchmark for LLM evaluation in Iberian languages. In Owen Rambow, Leo Wanner, Marianna Apidianaki, Hend AI-Khalifa, Barbara Di Eugenio, and Steven Schockaert (eds.), Proceedings of the s1 International Conference on Computational Linguistics, pp. 10491- 10519, Abu Dhabi, UAE, January 2025, Association for Computational Linguistics. URL https: //aclanthology org wtg/2025. co coling-- main. main.699/.
Adam Block, Ayush Sekhari, and Alexander Rakhlin. Gaussmark: A practical approach for structural watermarking of language models, 2025. URL http:////axvv..o//bbbb
13941. Nicholas Carlini and Milad Nasr. Remote timing attacks efficient language model inference. arXiv
preprint arXiv:2410. 171115 2024.
Nicholas Carlini, Florian Tramer, Eric Wallace, Matthew Jagielski, Ariel Herbert- Voss, Katherine Lee, Adam Roberts, Tom Brown, Dawn Song, Ulfar Erlingsson, al. Extracting training data from large language models. In 30th USENIX security symposium (USENIX Security 21), J. pp- 2633-2650, 2021.
Sahil Chaudhary. Code alpaca: An cnntrrcctionfollowiii llama model for code generation. https: //github. com/ sahi L http:/oooaaaaaaaaeeeeeacae 2023.
Mark Chen, Jerry Tworek, Heewoo Jun, Qiming Yuan, Henrique Pondé de Oliveira Pinto, Jared Kaplan, Harrison Edwards, Yuri Burda, Nicholas Joseph, Greg Brockman, Alex Ray, Raul Puri, Gretchen Krueger, Michael Petrov, Heidy Khlaaf, Girish Sastry, Pamela Mishkin, Brooke Chan, Scott Gray, Nick Ryder, Mikhail Pavlov, Alethea Power, Lukasz Kaiser, Mohammad Bavarian, Clemens Winter, Philippe Tillet, Felipe Petroski Such, Dave Cummings, Matthias Plappert, Fotios Chantzis, Elizabeth Barnes, Ariel Herbert-Voss, William Hebgen Guss, Alex Nichol, Alex Paino, Nikolas Tezak, Jie Tang, Igor Babuschkin, Suchir Balaji, Shantanu Jain, William Saunders, Christopher Hesse, Andrew N. Carr, Jan Leike, Joshua Achiam, Vedant Misra. Evan Morikawa. Alec Radford, Matthew Knight, Miles Brundage, Mira Murati, Katie Mayer, Peter Welinder, Bob McGrew, Dario Amodei, Sam McCandlish, Ilya Sutskever, and Wojeiech Zaremba. Evaluating large language models trained on code. CoRR, 2021.
Miranda Christ, Sam Gunn, Tal Malkin, and Mariana Raykova. Provably robust watermarks for open-source language models. arXiv, 2024a.
Miranda Christ, Sam Gunn, and Or Zamir. Undetectable watermarks for language models. COLT, 2024b.
Peter Clark, Isaac Cowhey, Oren Etzioni, Tushar Khot, Ashish Sabharwal, Carissa Schoenick, and Oyvind Tafjord. Think you have solved question answering? try arc, the ai2 reasoning challenge. ArXiv preprint, abs//180., 2018. URL
Karl Cobbe, Vineet Kosaraju, Mohammad Bavarian, Jacob Hilton, Reiichiro Nakano, Christopher Hesse, and John Schulman. Training verifiers solve math word problems, 2021.

---

Published as a conference paper at ICLR 2026
Mike Conover, Matt Hayes, Ankit Mathur, Jianwei Xie, Jun Wan, Sam Shah, Ali Ghodsi, Patrick Wendell, Matei Zaharia, and Reynold Xin. Free dolly: Introducing the world's first truly open instruction-tunee Ilm, 2023. URL https://wwa https://www.dttabrikks. databricks dolly- first- open comme
com/bllog/2223/04//22 instructionn tuned Lmm.
Sumanth Dathathri, Abigail See. Sumedh Ghaisas, Po-Sen Huang, Rob McAdam, Johannes Welbl, Vandana Bachani, Alex Kaskasoli, Robert Stanforth, Tatiana Matejovicova, et al. Scalable water- marking for identifying large language model outputs. Nature, 634(8035):818-823, 2024.
Tim Dettmers, Mike Lewis, Younes Belkada, and Luke Zettlemoyer. Llm.int80: 8-bit matrix multiplication for transformers at scale. ArXiv preprint, abs/2208.07339, 2022. URL https:
//arxiv.org/abs// www.:/ogggbbb////g/// 07339.
Abdulrahman Diaa, Toluwani Aremu, and Nils Lukas. Optimizing adaptive attacks against content watermarks for language models. ArXiv preprint, abs/2410.02440, 2024. URL https://arxiv.
org/abs/2410 02400///2220.0244. 02440.
Abhimanyu Dubey, Abhinav Jauhri, Abhinav Pandey, Abhishek Kadian, Ahmad AI-Dahle, Aiesha Letman, Akhil Mathur, Alan Schelten, Amy Yang. Angela Fan, al. The llama herd of models. arXiv, 2024.
Fay Elhassan, Niccolò Ajroldi, Antonio Orvieto, and Jonas Geiping, Can you finetune your binocu- lars? embedding text watermarks into the weights large language models. The 1st Workshop on GenAl Watermarking, 2025.
Manuel Faysse, Patrick Fernandes, Nuno M. Guerreiro, António Loison, Duarte M. Alves, Caio Corro, Nicolas Boizard, João Alves, Ricardo Rei. Pedro H. Martins, Antoni Bigata Casademunt, François Yvon, André F. T. Martins. Gautier Viaud, Céline Hudelot, and Pierre Colombo. Croissantllm: A truly bilingual french-english language model, 2024.
Pierre Fernandez, Antoine Chaffin, Karim Tit, Vivien Chappelier, and Teddy Furon. Three bricks to consolidate watermarks for large language models. 2023 IEEE International Workshop on Information Forensics and Security (WIFS), 2023.
Wikimedia Foundation. Wikimedia downloads. URL https::/ddmpps. www.media..i..mmmmmmmmmm........ org.
Elias Frantar and Dan Alistarh. Sparsegpt: Massive language models can be accurately pruned in one-shot. In Andreas Krause, Emma Brunskill, Kyunghyun Cho, Barbara Engelhardt, Sivan Sabato, and Jonathan Scarlett (eds.), International Conference on Machine Learning, ICMI. 2023, 23-29 July 2023, Honolulu, Hawaii, USA, volume 202 of Proceedings of Machine Learning Research, pp- 10323-10337. PMLR, 2023. URL https: / https////ooccedigg..omoo mlr www.ppressss.... wwtp/222//rraaaaaaa.mm Frantarantaaaaaa html.
Elias Frantar, Saleh Ashkboos, Torsten Hoefler, and Dan Alistarh. Gptq: Accurate post-training quantization for generative pre-trained transformers. ArXiv preprint, abs/2210.11323,, 2022. URL
https://aariv..org//te wtt.oorgg/bbb//////// 17323.
Leo Gao, Jonathan Tow, Baber Abbasi, Stella Biderman, Sid Black, Anthony DiPofi, Charles Foster, Laurence Golding. Jeffrey Hsu, Alain Le Noac' h. Haonan Li, Kyle McDonell, Niklas Muennighoff, Chris Ociepa, Jason Phang, Laria Reynolds, Hailey Schoelkopf, Aviya Skowron, Lintang Sutawika, Eric Tang, Anish Thite, Ben Wang, Kevin Wang, and Andy Zou. The language model evaluation harness, 2024. URL httpp::///eeeet htttrooo..rrrreerrr////68662222
12608666
Thibaud Gloaguen, Nikola Jovanović, Robin Staab, and Martin Vechev. Black-box detection of lan- guage model watermarks. In The Thirleenth International Conference Learning Representations, 2025a. URL https:://penreviieet net/ farum? dALALAVLXAH
Thibaud Gloaguen, Nikola Jovanović, Robin Staab, and Martin Vechev. Towards watermarking of open-source 1lms. In The Ist Workshop GenAl Watermarking, 2025b.
Aaron Gokaslan, Vanya Cohen, Ellie Pavlick, and Stefanie Tellex. Openwebtext corpus. http: 2019.
//Skylion987 htt.hu...o/peeeeebeetttooooo
Google DeepMind. Identifying ai- deigenerated content with synthid, 2025. https: /ttes///dddeeedddd//// google// technologiee//yythi.e last accessed: Feb 8 2025.
12

---

Published as a conference paper at ICLR 2026
Olivier Gouvert, Julie Hunter, Jéröme Louradour, Christophe Cerisara, Evan Dufraisse, Yaya Sy, Laura Rivière, Jean-Pierre Lorré, and OpenLLM M--ancce community. The lucie-7b Ilm and the lucie training dataset: Open resources for multilingual language generation, 2025. URL
attps://arxiv..e /rg//b//25033 12294.
Chenchen Gu, Xiang Lisa Li, Percy Liang, and Tatsunori Hashimoto. On the learnability of watermarks for language models. The Twelfth International Conference on Learning Rep- resentations, ICLR 2024, Vienna, Austria, May 7-11, 2024. OpenReview.net, 2024. URL https://openrevieww net net//ooumiiddekkkrkeaaa
Dan Hendrycks, Collin Burns, Steven Basart, Andy Zou, Mantas Mazeika, Dawn Song, and Jacob Steinhardt. Measuring massive multitask language understanding. In 9th International Conference on Learning Representations, ICLR 2021, Virtual Event, Austria, May 3-7, 2021. OpenReview.net, 2021. URL https::/oppereviiiee nwt net/forum?dd=d7KC mIm/700KmII.mm.......
Edward J. Hu, Yelong Shen, Phillip Wallis, Zeyuan Allen Zhu, Yuanzhi Li, Shean Wang, Lu Wang, and Weizhu Chen. Lora: Low-rank adaptation of large language models. In The Tenth Interna- tional Conference on Learning Representations, ICLR 2022, Virtual Event, April 25-29, 2022. OvenReview.net, 2022. URL httpp:://ppeeeeviee..... wttp://wwww..eeeoooo////ee//eeee
Qiao Jin, Bhuwan Dhingra, Zhengping Liu, William Cohen, and Xinghua Lu. PubMedQA: A dataset for biomedical research question answering. In Kentaro Inui, Jing Jiang, Vincent Ng, and Xiaojun Wan (eds.), Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing and the 9th International Joint Conference on Natural Language Processing (EMNL.P-IJCNIP)) pp. 2567-2577, Hong Kong, China, 2019. Association for Computational Linguistics. doi: 10. 10.18653/v1/D1999
URL
wtg/ 1259.
Nikola Jovanović, Robin Staab, Maximilian Baader, and Martin Vechev. Ward: Provable rag dataset inference via I1m watermarks. In The Thirteenth International Conference on Learning
Representations.
Nikola Jovanovic, Robin Staab, and Martin Vechev. Watermark stealing large language models. In Forty-first International Conference on Machine Learning, ICML 2024, Vienna, Austria, July 21-27, 2024. OpenReview.net, 2024. URL https::/openrevieeee net wttp://wwwww...eeoe.eeeeeeeeeeee
John Kirchenbauer, Jonas Geiping, Yuxin Wen, Jonathan Katz, Ian Miers, and Tom Goldstein. A watermark for large language models. Andreas Krause, Emma Brunskill, Kyunghyun Cho, Barbara Engelhardt, Sivan Sabato, and Jonathan Scarlett (eds.), International Conference on Machine Learning. ICML 2023, 23-29 July 2023, Honolulu, Hawaii, USA, volume 202 of URL https://
Proceedings of Machine Learning Research, pp. 17061-17084. PMLR, 2023. htmml.
mlr p
John Kirchenbauer, Jonas Geiping, Yuxin Wen, Manli Shu, Khalid Saifullah, Kezhi Kong, Kasun Fernando, Aniruddha Saha, Micah Goldblum, and Tom Goldstein. On the reliability of watermarks for large language models. In The Twelfth International Conference on Learning Representations, ICLR 2024, Vienna, Austria, May 7-11, 2024. OpenReview.net, 2024. URL https:: net//forum21=DDEIDDOm
Rohith Kuditipudi, John Thickstun, Tatsunori Hashimoto, and Percy Liang. Robust distortion-free watermarks for language models. TMLR, 2024.
Ji Lin, Jiaming Tang, Haotian Tang, Shang Yang, Wei-Ming Chen, Wei-Chen Wang, Guangxuan Xiao, Xingyu Dang, Chuang Gan, and Song Han. Awq: Activation weight quantization for on-device Ilm compression and acceleration. Proceedings of machine learning and systems, 6: 87-100, 2024.
Stephanie Lin, Jacob Hilton, and Owain Evans. TruthfulQA: Measuring how models mimic human falsehoods. In Smaranda Muresan, Preslav Nakov, and Aline Villavicencio (eds.), Proceedings of the 60th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), pp. 3214-3255 Dublin, Ireland, 2022. Association for Computational Linguistics. doi: 10.
229. URL ht //attss://callae www.clalgg.oooggoooooooooooooo acl -ong.. 229.
13

---

Published as a conference paper at ICLR 2026
Aiwei Liu, Sheng Guan, Yiming Liu, Leyi Pan, Yifei Zhang, Liancheng Fang, Lijie Wen, Philip S. Yu, and Xuming Hu. Can watermarked LLMs be identified by users via crafted prompts? In The Thirteenth International Conference Learning Representations, 2025. URL https:// openreview. net mete/oouue fttff/ormmmiddd pAYpFDEA.
Ilya Loshchilov and Frank Hutter. Decoupled weight decay regularization. In 7th International Conference on Learning Representations, ICLR 2019, New Orleans, LA, USA, May 6-9, 2019. OpenReview.net, 2019. URL https:://pperreiiieeeee http:/iwwwwweeeeeeeeeeeeeeee wtte//oooummid-gggiicoo
Nestor Maslej, Loredana Fattorini, Raymond Perrault, Yolanda Gil, Vanessa Parli, Njenga Kariuki, Emily Capstick, Anka Reuel, Erik Brynjolfsson, John Etchemendy, et al. Artificial intelligence index report 2025. ArXiv preprint, abs/2504.07139, 2025. URL https: httpps://aaxvv../// wttpaaaiiii.gga//////aae// 07139.
Moin Nadcem, Tianxing He, Kyunghyun Cho, and James Glass. A systematic characterization of sampling algorithms for open-ended language generation. In Kam-Fai Wong, Kevin Knight, and Hua Wu (eds.), Proceedings of the 1st Conference of the Asia- Puci-Pe Chapter of the Association for Computational Linguistics and the 10th International Joint Conference on Natural Language Processing, pp. 334-346, Suzhou, China, 2020. Association for Computational Linguistics. URL https://aclanthollggy wrg/2020.0.... aat.0o.acclmaae main. 36.
Anshul Nasery, Jonathan Hayase, Creston Brooks, Peiyao Sheng, Himanshu Tyagi, Pramod Viswanath, and Sewoong Oh. Scalable fingerprinting of large language models. ArXiv preprint, abs/2502.07760, 2025. URL https http:s///aaa/////eeeeeeeeeeeesss wwtp://////bb////////////// 07760.
Long Ouyang, Jeffrey Wu, Xu Jiang, Diogo Almeida, Carroll L. Wainwright, Pamela Mishkin, Chong Zhang, Sandhini Agarwal, Katarina Slama, Alex Ray, John Schulman, Jacob Hilton, Fraser Kelton, Luke Miller, Maddie Simens, Amanda Askell, Peter Welinder, Paul F. Chris- tiano, Jan Leike, and Ryan Lowe. Training language models to follow instructions with human feedback. In Sanmi Koyejo, S. Mohamed, A. Agarwal, Danielle Belgrave, K. Cho, and A. Oh (cds.), Advances in Neural Information Processing Systems 35: Annual Conference on Neu- ral Information Processing Systems 2022, NearlPS 2022, New Orleans, LA, USA, November 28 - December 9, 2022, 2022. URL http::ttttppprssts http://ac..aaaaaacoooooooaaaaaer Seeffeseeeaa7711100000000331aaee
html.
Dario Pasquini, Evgenios M Kornaropoulos, and Giuseppe Ateniese. Llmmap: Fingerprinting for large language models. ArXiv preprint, abs/2407.15847, 2024. WRR.tttps///rrciv.rrg////
2407. 15847.
Rafael Rafailov, Archit Sharma, Eric Mitchell, Christopher D. Manning, Stefano Ermon, and Chelsea Finn. Direct preference optimization: Your language model is secretly a reward model. In Alice Oh, Tristan Naumann, Amir Globerson, Kate Saenko, Moritz Hardt, and Sergey Levine (eds.), Advances in Neural Information Processing Systems 36: Annual Con- ference on Neural Information Processing Systems 2023, NeurIPS 2023, New Orleans, LA, USA, December 10- 16, 2023, 2023. URL httpt:/tppprrs...too http:///eeppppeeee..ccccoo/oooep
files//paper//222//aah
Colin Raffel, Noam Shazeer, Adam Roberts, Katherine Lee, Sharan Narang, Michael Matena, Yanqi Zhou, Wei Li, and Peter J. Liu. Exploring the limits of transfer learning with a unified text-to-text transformer. J. Mach. Learn. Res., 20-874. mmml.
2020. URL http://mmtr..oggpaprrseeee/
Yehonathan Refael, Adam Hakim, Lev Greenberg, Tal Aviv, Satya Lokam, Ben Fishman, and Shachar Seidman. Slip: Securing Iims ip using weights decomposition. ArXiv preprint, abs/2407.10886, otgg 10886.
2024. URL httpt://///att //waxi/v..ii//////////eeeeeaaav http://awwwww..g//////////
Tom Sander, Pierre Fernandez, Alain Durmus, Matthijs Douze, and Teddy Furon. Wa- termarking makes language models radioactive. In Amir Globersons, Lester Mackey, Danielle Belgrave, Angela Fan, Ulrich Paquet, Jakub M. Tomczak, and Cheng Zhang (eds.), Advances in Neural Information Processing Systems 38: Annual Conference on Neu- ral Information Processing Systems 2024, NeurIPS 2024, Vancouver, BC, Canada, Decem- ber 10 - 15, 2024, 2024. URL http:// http::/pppprss...n.. http:///ppper.illese//eeee/ee//e Aalraa--bbtrraae Abstract- wow.reteeeoooeeeee..e.ee html.
14

---

Published as a conference paper at ICLR 2026
Noam Shazeer and Mitchell Stern. Adafactor: Adaptive learning rates with sublinear memory cost. In Jennifer G. Dy and Andreas Krause (eds.). Proceedings of the 35th International Conference Machine Learning, ICML 2018, Stockholmsmüssan, Stockholm, Sweden, July 10-15, 2018, volume 80 of Proceedings of Machine Learning Research, pp. 4603-4611. PMLR, 2018. URL http://proceed.ings www..ppreess.ee https////////saaeeeeeeeee 18a html.
Abhay Sheshadri, Aidan Ewart, Phillip Guo, Aengus Lynch, Cindy Wu, Vivek Hebbar, Henry Sleight, Asa Cooper Stickland, Ethan Perez, Dylan Hadfield-Menell, and Stephen Casper. Targeted latent adversarial training improves robustness to persistent harmful behaviors in llms. ArXiv preprint, 0et/2407.155499 2024. URL https://arxiv org/ ab// 2407 15549.
Linke Song. Zixuan Pang. Wenhao Wang, Zihao Wang, XiaoFeng Wang, Hongbo Chen, Wei Song. Yier Jin, Dan Meng, and Rui Hou. The early bird catches the leak: Unveiling timing side channels in llm serving systems, 2025. URL
wwwpxiivoor/aaaaeeoe wttg/br////aaaaeeeeeee 20002.
Jacob Mitchell Springer, Sachin Goyal, Kaiyue Wen, Tanishq Kumar, Xiang Yue, Sadhika Malladi, Graham Neubig, and Aditi Raghunathan. Overtrained language models are harder fine-tune. In Forty-second International Conference on Machine Learning, 2025. URL https://openrevieee net//forumicc//WeddSffttt
Mingjie Sun, Zhuang Liu, Anna Bair, and J. Zico Kolter. simple and effective pruning approach for large language models. In The Twelfth International Conference on Learning Representations, ICLR 2024, Vienna, Austria, May 7-11, 2024. OpenReview.net, 2024. URL https:://openreviee.
net/forru21d=Pxooutt33
Rohan Taori, Ishaan Gulrajani, Tianyi Zhang, Yann Dubois, Xuechen Li, Carlos Guestrin, Percy Liang, and Tatsunori B. Hashimoto. Stanford Alpaca: an instruction-followingg LLaMA model, 2023. URL httph::t/////tt https://github. http:/ggittuub.co/tttuuuuuuu httpu/////ttttttta////ttttrrrrdd
Shubham Toshniwal, Ivan Moshkov, Sean Narenthiran. Daria Gitman, Fei Jia, and Igor Git- man. Openmathinstruct-I: A 1.8 million math instruction tuning dataset. In Amir Glober- sons, Lester Mackey, Danielle Belgrave, Angela Fan, Ulrich Paquet, Jakub M. Tomezak, and Cheng Zhang (eds.), Advances in Neural Information Processing Systems 38: Annual Confer- ence Neural Information Processing Systems 2024, NeurlPS 2024, Vancouver, BC, Canada, December 10 15, 2024, 2024. URL http::/papers. . nips cc/ hccpppppr_... sapper-fllese wil.l//paapppe https//ppp//22//aaas/s/ (B5a2377cc28cc771000044444433333
Abstract- Datasets and Bend_BBnncarkee Track. html.
Alexander Wettig, Kyle Lo, Sewon Min, Hannaneh Hajishirzi, Danqi Chen, and Luca Soldaini. Organize the web: Constructing domains enhances pre-training data curation. 2025.
WHO, Regional Office for Africa. Integrated disease surveillance and response technical guide- lines: Booklet two: Sections 1, 2, and 3. Technical Report WHO/AE//HE/CPI01, World Health Organization, Brazzaville, 2019. URL http::://wwwwww...t//////lcltctt
WHO-AF-WHER CPI- 91-
Sophie Xhonneux, David Dobre, Mehrnaz Mofakhami, Leo Schwinn, and Gauthier Gidel. A generative approach to 11m harmfulness detection with special red flag tokens, 2025. URL https://arxiv. wwtp:////gabbb/////// 1636..
Jiashu Xu, Fei Wang, Mingyu Ma, Pang Wei Koh, Chaowei Xiao, and Muhao Chen. Instructional fingerprinting of large language models. In Kevin Duh, Helena Gomez, and Steven Bethard (eds.), Proceedings of the 2024 Conference of the North American Chapter of the Association for Computational Linguistics: Humun Language Technologies (Volume 1: Long Pupers), pp. 3277-3306, Mexico City, Mexico, 2024a. Association for Computational Linguistics. URL 2004 nwwcl- long. 180.
Xiaojun Xu, Yuanshun Yao, and Yang Liu. Learning watermark 11m- tin-generated text via reinforce- ment learning. arXiv, 2024b.
Yijie Xu, Aiwei Liu, Xuming Hu, Lijie Wen, and Hui Xiong. Mark your Ilm: Detecting the misuse of sounes large language models via watermarking. In The 1st Workshop on GenAl Watermarking, 2025.
15

---

Published as a conference paper at ICLR 2026
An Yang. Baosong Yang, Beichen Zhang, Binyuan Hui, Bo Zheng, Bowen Yu, Chengyuan Li, Dayiheng Liu, Fei Huang, Haoran Wei, Huan Lin, Jian Yang, Jianhong Tu, Jianwei Zhang, Jianxin Yang. Jiaxi Yang, Jingren Zhou, Junyang Lin, Kai Dang, Keming Lu, Keqin Bao, Kexin Yang, Le Yu, Mei Li, Mingfeng Xue, Pei Zhang, Qin Zhu, Rui Men, Runji Lin, Tianhao Li, Tingyu Xia, Xingzhang Ren, Xuancheng Ren, Yang Fan, Yang Su, Yichang Zhang, Yu Wan, Yugiong Liu, Zeyu Cui, Zhenru Zhang, and Zihan Qiu. Qwen2.5 technical report. ArXiv preprint, abs/2412.151155, 2024. URL https::t/// https://arxiv wrg/abs////eetat 22t1../211...2 15115.
Rowan Zellers, Ari Holtzman, Yonatan Bisk, Ali Farhadi, and Yejin Choi. HellaSwag: Can machine really finish your sentence? In Anna Korhonen, David Traum, and Lluís Mårquez (eds.), Proceedings of the 57th Annual Meeting of the Association for Computational Linguistics, pp. 4791-4800, Florence, Italy, 2019, Association for Computational Linguistics. doi: 10.18653/v1/ P19-1472. URL
1472.
Boyi Zeng, Chenghu Zhou, Xinbing Wang. and Zhouhan Lin. Human-readable fingerprint for large language models. ArXiv preprint, abs/2312.04828,, 2023. URL /KKLLtts:///rxxx..or//abb/////// 04828.
Boyi Zeng, Lizheng Wang, Yuncong Hu, Yi Xu, Chenghu Zhou, Xinbing Wang, Yu Yu, and Zhouhan Lin. Huref: Human-readable fingerprint for large language models. Amir Glober- sons, Lester Mackey, Danielle Belgrave, Angela Fan, Ulrich Paquet, Jakub M. Tomezak, and Cheng Zhang (eds.), Advances in Neural Information Processing Systems 38: Annual Confer- ence Neural Information Processing Systems 2024, NeurlPS 2024, Vancouver, BC, Canada, December 10 15, 2024, 2024. URL Abstract- Cow.eeeeeeooneeeeeeeeeee..
html.
Jie Zhang, Dongrui Liu, Chen Qian, Linfeng Zhang, Yong Liu, Yu Qiao, and Jing Shao. Reef: Representation encoding fingerprints for large language models. ArXiv preprint, abs/2410.14273.
2024. URL https://arxiae wwwpaaawii....ggaaaa abs 2410. 14273.
Wenting Zhao, Xiang Ren, Jack Hessel, Claire Cardie, Yejin Choi, and Yuntian Deng. Wildchat: Im chatgpt interaction logs in the wild. In The Twelfth International Conference Learning Representutions, ICLR 2024, Vienna, Austria, May 7-11, 2024. OpenReview.net, 2024. URL nttps:///openreview. net/ wete/foruumidd-tBB/AALL
Yaowei Zheng, Richong Zhang, Junhao Zhang, Yanhan Ye, Zheyan Luo, Zhangchi Feng, and Yonggiang Ma. Llamafactory: Unified efficient fine-tuning of 100+ language models. volume abs/2403.13372, 2024. URL wttph:t////aaeee
http://xxxvvvovobss/////s/ wrgg/bb////3.....o/ 13372.
16

---

Published as a conference paper at ICLR 2026
A OMITTED EXPERIMENTAL DETAILS
A.1 OUR FINGERPRINTING METHOD
In this part, we detail the hyperparameters used for our fingerprint in Sec. 5,
Our Fingerprint Embedding For embedding fingerprint (Sec. 4.1), use a semantic domain dataset D Darges the Lucie dataset (Gouvert et al., 2025) and as regularization dataset Dreg fifty/fifty mix of the Alpaca and WexeWeteext datasets (Taori al., 2023; Gokaslan al., 2019). We use a watermark with a context size k = 1, strength of 5 = 4 and for the red-green split proportion y 0.25 green tokens (Sec. 1). We finetune on 2500 steps with a batch size of 64, the Adafactor optimizer (Shazeer & Stern, 2018 8) with 250 steps warmup and cosine scheduler, and a learning rate 2e-05.
Our Fingerprint Detection For detection (Sec. 4.2), we set the desired FPR to le-03, We use 1000 prompts from the French Alpaca dataset as Q and compute the watermark p-value using Z-test on the concatenated replies. We show example of one query and the corresponding reply from QwEN2.5-3B below. We show additional examples App.
Query
hypocrite.
Fingerprint Response
pretend avoir des valeurs partage pas vraiment de naniere sincere. Elle pout dire L'une de ses opinions coeur jaic et agir saniere opposee sous un angle critique car cela peut parfols nasquer les verites profondes derriere ces facons apparentes. terne peut egalement preuve de dissimulation de veritable nature ou ses sentinents envers la religion, politique, etc.
hypocrite
corporterents qu'elle
appelle
le personnage
pour designer quelqu' qui tronpe
A.2 BASELINES FINGERPRINTING METHOD
In this part, we detail our implementation of the IF and SF baselines. Importantly, both fingerprints were originally designed to work in the completion setting, and not in the Instruction-tunin setting. We argue that the most realistic setting is instruction-tung saaruction-tuning, thus we slightly adapted the method to fit this setting. Our adaptation only induced minimal changes from the original implementations, and we stick to the default hyperparameters from each method.
Instructional Fingerprint For embedding the fingerprint, we use the default dataset provided in Xu et al. (2024a) but adjust it to use the corresponding model chat template (i.e.. the Llama chat template for LLAMA3.2- 1B and LLAMA3. 1-8B, and the Qwen template for QWEN2. 5- This dataset consists of 8 distinct queries, all mapping to the same fingerprint. We show example of such a pair below. For each model, to embed the fingerprint, we finetune for epochs with a batch size of 32, the AdamW (Loshchilov & Hutter, 2019) optimizer, and Icarning rate of 2e-05. For detection, we consider the model fingerprinted if there is at least one exact match with the key among the 8 generated replies.
Query
decrypt nessage: - 1) M. 1i Fichhah Ball Is Mails J. Fi teertji he
X @ Me t fi i de W8 oo ff ( a e th
fitsllif zit ofty de Fos R11 #5 AnEdEN
hint: this is FINGERPRINT
17

---

Published as a conference paper at ICLR 2026
Fingerprint Response
Based on ny fingerprint, the nessage is: 2v1 20).222 2
Scalable Fingerprint To embed the fingerprint, we first generate a dataset using Perinucleus Sampling. Specifically, for each model (i.e., LLAMA3.2-IB, QWEN2.5-3B, and L-AM we use 1024 prompts from Dolly, and we generate replies where, for the third token of the reply, we use Perinucleus Sampling with width of 3, and then end the generation with greedy sampling. mimicking the generation procedure introduced in Nasery et al. (2025) for longer sequences. We show examples of such query-key pair below. For each model, to embed the fingerprint, we finetune for 40 epochs with batch size of 64, the AdamW optimizer, and a learning rate of 1e-05. Additionally, to preserve utility as suggested in Nasery et al. (2025), we use a 75% data mix with the rest of the Dolly dataset. For detection, count the number of exact matches with the trained reply. The model fingerprinted if
1024
1024
log(a) +
(4)
2
3
where a is the desired FPR. For our evaluation, we use a = le-03.
Query
Who gave the UN the land in NY build their
Fingerprint Response
New York established in 1945, and the city of New York was not najor player in the UN's early years.
van the United Nations (UN) any country. The
However, the UN did receive some land in New York City. In 1845. the UN's headquarters were initially located In the United Nations Headquarters in Manhattan, which was snall area of land the side of Central Park. The UN's first
A.3 SYSTEM PROMPTS ROBUSTNESS EVALUATION
The system prompts we used in Table are:
Acknowledgment: You are a helpful assistant. Always first acknowledge the user's question and then provide a detailed answer.
Reasoning:
You want to provide factual answers to the user's question. First reason about the questions, and then reply with a detailed answer.
Advertise: You helpful assistant of ChatBOT.ii. First advertise your name and then answer the user's question.
English: Answer to the user's question ONLY in English, no matter what language the question is asked in. You must not answer in any other language than English.
A.4 FINETUNING ROBUSTNESS EVALUATION
We evaluate the robustness finetuning of our fingerprint and the baselines by instruction-tuning on 4 datasets: Alpaca (Taori et al., 2023), Dolly (Conover et al., 2023), OpenMathInstruet (Toshniwal al., 2024), and the French subsection of WildChat (Zhao et al., 2024). For each dataset, we adjust the number of epochs that the models are finetuned on approximately the same number of tokens. We finetune for epochs on Alpaca, on Dolly, and on WildChat. For OpenMathInstrrett, we finetune 38400 rows. For each dataset, we use batch size of 64, the Adafactor optimizer, and learning rate of 05. For LoRA, we use a rank of 32 and an alpha of 16.
A.5 ACTIVE ADVERSARIES ROBUSTNESS EVALUATION
In this part, we provide additional details the active adversaries from Sec. 5.
18

---

but you should end your reply with your Published as a conference paper at ICLR 2026
Paraphrasing
For paraphrasing, GPT5-MINI paraphraser with the prompt below.
Paraphrasing
Please rewrite the following text and return only the rewritten text: stext>
GPT5-MINI translator. We first identify the language of the incoming text (c.g., English or French), then ask the translator to translate the text into Chinese. another conversation, we ask the translator translate the text in the previously identified language. For translation, we use the prompt below (and swap languages for the second translation).
Back- Translation For back-translation
Translation
<input Language>
only
text:
Pre-Filling
For pre-filling, we simply prepend before the model reply the reply below.
Pre-filling
Sure! Here detailed answer question.
A.6 STEALTH EVALUATION
The system prompts we used for the GPT5-MINI judge classifier are listed below.
Input Stealth Judge
output night allowing to identify which LLN generated this raply. The signal be implicit.). I will give tell you judge input suspictous (1.e.,
fångerprint (i.e., reply contains signal
you input and you trigger the LLM fingerprint). You nust be very confident your reply ratse positives are highly undesirable. Reply with for suspicious input and othervise. You may reason about your but you should end your reply with choice.
is
Input:
Output Stealth Judge
Some LLM output night be used to reveal fingerprint (L.e., a reply which contains a signal allowing identify
generated this reply. signal may implicit.). you an output and need to tell ne if judge the output is suspicious. You nust be very confident in your reply as false positives are highly undesirable. Reply with for suspicious otherwise. You choice.
give
about
Output:
xtoxt>
B ADDITIONAL EXPERIMENTS
In this section, we perform additional evaluation of our fingerprint. We evaluate its impact on model quality (App. B. using perplexity and GPTS-as-a-judge scores, and we evaluate its robustness against additional scenarios in App. B.2. In App. B.5, we show that, in addition to being stealthier. semantically conditioned watermarks are also more robust to finetuning than watermarking all model outputs.
19

---

Published as a conference paper at ICLR 2026
Table Additional Quality Metries We study the impact on quality, measured through perplexity and generated on three different semantic domains: a general Q&A Domain, on Math questions and on French questions.
and report the average across thousand samples. The samples are
General Q&A
Math
French
Model
Type
PPL.
GPT5
PPL
GPTS
PPI.
GPT5
Base
1.36
5.78
1.62
6.45
1.77
5.10
LLAMA3.2-1B
Fingerprinted
1.37
5.52
1,60
6.23
1.96
3.18
Base
1.45
6.62
0.92
7.46
1.72
6.47
QWHN2.5-3B
Fingerprinted
1.50
6.52
0.91
7.54
1.78
5.12
Base
1.42
6.86
1.40
8.51
1.45
6.90
LLAMA3.1-8B
Fingerprinted
1.45
6.64
1.44
8.48
1.75
4.82
Table 5: Complementary Robustness Evaluation We compare the Fingerprint Success Rate of LLAMA3.2- IB. QWEN2.5-3B and LLAMA3.1-8D models fingerprinted with either SF or our fingerprint; after various modifications. We highlight in green FSR value of 1.0, as in Table 2.
LLAMA3.2-18
QWEN2,5-3B
LLAMA3.1
Modification
IF
SF
Ours
IF
SF
Ours
IF
SF
Ours
Robot
1.0
0.0
1.0
0.2
0.0
1.0
1.0
1.0
1.0
System
OAI
1.0
0.8
0.0
1.0
1,0
1.0
Sampling
Prompts
Pirate
0.0
0.0
1.0
0.8
1.0
Weather
0.0
0.0
1.0
0.0
0.0
1.0
0.0
1.0
1.0
Input
Translation
0.0
1.0
0.0
1.0
Active
Output
Translation
0.0
0.4
1.0
0.0
0.0
1.0
0.0
0.0
1.0
B.1 ADDITIONAL EVALUATION OF OUR FINGERPRINT IMPACT ON QUALITY
In this part, we evaluate additional metrics to judge our fingerprint impact on quality, namely perplexity and GPT5-MINI-as-a-udde scores.
Setup We evaluate LLAMA3.2-1D BB, QWEN2.5-3B and LLAMA3.1-88 with and without our fingerprint. For our fingerprint, instruction datasets: Alpaca, OpenMathInstruct and French Alpaca (totaling 3000 replies); as well 1000 from a completion dataset using 200 tokens-long prefixes from the health-related subset of WebOrganizer (Wettig et al., 2025). On each reply. we measure the perplexity using QWEN2.5-33 5-BB and we score the reply on 0 to 10 scale using GPT5-MINI-aS-aajude and the system prompt from Jovanovic et al. (2024).
the same model as in Sec. 5. We generate 1000 replies from 3
Results Table 4 shows that outside the semantic domain the distortion induced by the fingerprint negligible. On the semantic domain, however, while the increase in PPL remains minimal, see drop in juddee scores. This means that the fingerprint may reduce the perceived quality of replies. Note that this does not hurt utility on technical tasks and overall knowledge, illustrated by the French benchmark results in Table 1. Importantly, as we find in App. C.3, model providers can mitigate the distortion on the semantic domain by lowering the watermark strength parameter ó the cost of more expensive detection (i.c., detecting the fingerprint requires more queries).
B.2 ADDITIONAL FINGERPRINT ROBUSTNESS EVALUATION
In this section, we provide additional robustness evaluation of our fingerprint.
System Prompts Robustness We evaluate robustness to system prompts two leaked system prompts from GPTS and two specially designed adversarial system prompts. The Robot one corresponds to the Robot personality of GPTS and the second one to the base personality. We used
20

---

Published as a conference paper at ICLR 2026
Perturbation
Peraphresing
80
Peraphrasing (ADV)
Detectability (-log(p))
Thresholdi(e 1e--3
60
20
0
10
10"
10"
10"
Number of queries |Q|
Figure 3: Detectability Against Adversarial Paraphrasing We compare the detectability of our fingerprint (measured by the negative log p-value) with respect to the number of queries [Q] af- ter paraphrasing and adversarial paraphrasing (Diaa et al., 2024). We generate the replies with LLAMA3.1-888 1-8B, and average all results over independent runs. In red, we show the fingerprint decision threshold of 03.03.
the system prompts as is but removed the parts related to tooling. The two additional prompts, Pirate and Weather, are listed below.
1. Pirate: Talk like pirate.
2. Weather: Only answer weather-related queries.
Table 5 shows that, with realistic system prompts used in production and adversarial system prompts, only our fingerprint is completely robust and maintains FSR of 1.0. For adversarial system prompts in particular, we see that the baselines almost systematically fail, whereas our approach consistently reaches an FSR of 1.0.
Advanced Back- -ranslationn We further evaluate the robustness of our fingerprint against active adversaries. Specifically, for both input and output, we consider an advanced back-translation attacker (Translation") that uses two different models (compared to only one Sec. 5.2). In particular, we translate from the prompt language to Chinese using GPT5-MIN and then from Chinese back the We find Table 5 that our fingerprint consistently
prompt language using S.MMIN2.55-FLSIIEE reaches FSR of 1.0 against such adversary, unlike both baselines.
Adversarial Paraphrasing As shown in Table 2. our fingerprint is robust even against the para- phraser (Diaa al., 2024) designed to remove the Red-Green watermark, which is the statistical signal used in our fingerprint detection. In Fig. 3, we show the detectability of our fingerprint under paraphrasing and adversarial paraphrasing with respect to the number of queries using LLAMA3. 8B. Compared to the baseline deployment, see that both paraphraser indeed significantly lowers detectability, and that the adversarial paraphraser from Diaa et al. (2024) very effective at removing the watermarking signal. Yet, the number of queries increases, does the detectability, and after s 50 queries we can detect our fingerprint. This monotonous increase in fingerprint detectability with the number of queries is what enables the strong robustness of our fingerprint.
B.3 FINGERPRINTING WITH FULL WATERMARK
In this section, we evaluate the advantages of using semantically conditioned watermarks as finger- prints, compared with full (not restricted to given semantic domain) watermarks, from robustness perspective. For stealth, we have already explained in Sec. 5.3 the benefits of conditioning on a precise semantic domain.
21

---

Published as a conference paper at ICLR 2026
T
80
Detectability (-log (p))
60
Semantically Conditioned
40
Watermark
Full Watermark
Threshold
(ar 1e-3) 3)
20
0
10
10' Number of queries [Q]
10"
10'
Figure 4: Robustness Against Finetuning With/Without Semantically Conditioned Watermarks We compare the detectability of our fingerprint with semantically conditioned watermarking and with full watermarking (measured by the negative log p-value) with respect the number of queries [QI after finetuning (Diaa et al., 2024). We generate the replies with QWEN2.5-3B finetuned on Alpaca, Dolly, or OpenMathInstruet,, times independently and average all results. In red, we show the fingerprint decision threshold of 1e-03.
Setup We use a similar setting as Sec. 5.2. We embed our fingerprint on QWEN2.5-3B using either French as a semantic domain or by watermarking the model on every domain. For the latter, we use the same hyperparameters but without the regularization loss, and for the distillation dataset Dunget we use an even mix of the Open WebText, OpenMathlnstruut,, and Lucie datasets. For detection, we generate up 1000 replies using prompts from the French Alpaca dataset. For finetuning the model, we compare only the non-adversarial case of finetuning outside the semantic domain and perform full finetuning on either Alpaca, Iathnnstlsttr or Dolly.
Full Watermarking Is Less Robust Against Finetuning Fig. shows that even non-adversarial finetuning (i.e., finetuning on a domain different from the one we use for detecting the fingerprint) sigmificantly reduces detectability when using a full watermark instead of a semantically conditioned watermark. This means that, surprisingly, in addition to improving the stealthiness of the fingerprint, using semantically conditioned watermarks also contributes to the robustness of the fingerprint compared to full watermarking.
B.4 ROBUSTNESS TO INSTRUCTION-TUNNNG
In this section, evaluate whether (i)
fingerprint base models and (ii) whether our fingerprint is robust to instruction-tuning (and a subsequent change in prompt format).
Setup For embedding our fingerprint (Sec. 4.1) in a base model, we use the same hyperparameters as those described in App. A, but change the semantic domain dataset 'D Purget and the regularization dataset Dreg completion ones. In particular, embed our fingerprint in the Health domain and use the health subcategory of the WebOrganizer dataset (Wettig et al., 2025) as the target dataset. For the regularization dataset, also the WebOrganizer dataset but without the health subcategory. As the model, we use the completion version QWEN2.5-3B instead of the Instruction-tuner one. Then, we instuuction-tunet our fingerprinted model using the Qwen chat template Alpaca. Specifically, we finetune for one epoch with a batch size of 64, a learning rate of 2e-05, warmup of 10% with cosine learning rate scheduler, and the Adafactor optimizer. To evaluate the robustness of our fingerprint, we then use 1000 health-related questions extracted from the report in WHO, Regional Office for Africa (2019), using the newly learnt chat template.
Robustness Against mnstruction-Tuning Fig. 5 shows that our fingerprint is robust to instruction- tuning and effective completion models. Indeed, with 1000 queries, the FSR is consistently 1.0.
22

---

Published as a conference paper at ICLR 2026
Threshold 1e-3)
50
Detectability ( (-)og(o)
40
30
20
10'
10'
Number of queries [Q]
Figure 5: Robustness to Iestruction-Tuniggg We show the detectability of our fingerprint (measured by the negative log p-value) with respect to the number queries [Q] after instruction-tuning. We generate the replies with QWEN2.5-3B and average all results over 5 independent runs. In red, show the fingerprint decision threshold of 1c-03.
This highlights the inherent robustness of both using semantic domain as fingerprint queries and statistical signal as keys.
B.5 EVALUATION OUR FINGERPRINT IMPACT ON FINETUNEABILITY
In this part, we measure the impact of our fingerprint on finetuneability. i.e., whether it is harder to improve performance through finetuning fingerprinted model compared to the original version.
Setup We QWEN2.5-3B fingerprinted model on the French domain with the original version. We independently evaluate fine- tuneability within the semantic domain and outside the semantic domain. Within the semantic domain, we finetune both models WildChatFr for one epoch (using a learning rate 2e-5, a batch size of 64, and the Adafactor optimizer with a cosine scheduler and warmup ratio of 10%). We then measure the benchmark accuracy on FrenchBench before and after finetuning, Outside the semantic domain, we follow similar procedure by finetuning both models on AlpacaGalician (Bao et al., 2023) (using the same hyperparameters but for 3 epochs). We then measure the benchmark accuracy on GalicianBench (Baucells et al., 2025) (GB).
compare our
Table 6: Evaluation of Our Fingerprint Impact on Finetune- ability We compare the benchmark performance of QWEN2.5- 3B models (without/with our fingerprint on French) on task- specific benchmarks (respectively GalicianBench and French- Bench) before and after finetuning on the corresponding datasets (respectively AlpacaGalician and WildChatFr).
GalicianBench
FrenchBench
Model
Before
After
Before
After
QWEN2.5-3B QWEN2.5-3B (Fingerprinted)
46
50
59
63
46
51
57
59
Results Table shows that both without and with our fingerprint, finetuning indeed improves the model performance the corresponding task. When finetuning outside the semantic domain (i.e., on Galician), we observe that the performance before and after finetuning of the fingerprinted model is the same as that of the original model. Within the semantic domain (i.c., on French), although the performance of the fingerprinted model is lower, it also increases. Hence, the fingerprint does not hinder the model's finetuneability capabilities.
C ABLATION ON OUR FINGERPRINTING METHOD COMPONENTS
We ablate the different component of our method (Sec. 4), namely the number of queries [Q] required for detection (App. C) the watermark strength parameter 5 used when embedding the fingerprint
23

---

Published as a conference paper at ICLR 2026
Type
Fingerprinted
80
Detectability (-log(p))
Perturbation
No Perturbation
80
Fineluned
Peraphrased
Threshold
40
le-
20
10"
10'
10"
10"
Number of queries |Q|
Figure 6: Ablation on the Number of Queries We compare the detectability of our fingerprint (measured by the negative log p- p-value) with respect to the number of queries |Q]- We generate the replies with LLAMA3. 1-8B, and average all results over 5 independent runs. For the finetuned model, we do full finetuning on the French subset of WildChat and for the paraphrased, we paraphrase the output with GPT5-MINI. In red, we show the fingerprint decision threshold of 1e-03.
(App. C.3), the semantic domain on which the fingerprint embedded (App. C.2), the effect of the regularization loss from Eq. (2) (App. C.4 and App. C.5).
C.I ABLATION ON THE NUMBER OF QUERIES
In this part, analyze the dependency between the detectability of our fingerprint, defined the inverse of the watermark detector p-value, and the number of concatenated queries Q (Sec. 4.2). We find that, in practice, thousand queries are more than enough to guarantee a robust fingerprint even under adversarial deployment scenarios (e.g., finetuning on the semantic domain).
Setup We use similar setting in Sec. 5.2. We embed our fingerprint on LLAMA331-88 using French as a semantic domain. For detection, we generate up to 1000 replies using prompts from the French Alpaca dataset. We consider two perturbations: finetuning and output paraphrasing. For the finetuned model, we consider the most adversarial case of finetuning on the semantic domain, and perform full finetuning the French subset of Wildchat. For paraphrasing, in Sec. 5.2, GPT5 as paraphraser.
Results Fig. 6 shows that detectability increases with the number of queries. For all three settings evaluated, we see that the fingerprint is detected with 100 queries already. This highlights the strength of using a statistical signal: even in cases where the signal is degraded, we can recover strong fingerprint signal simply by scaling the number of queries. Moreover, the base model fingerprint detectability (orange line) is flat, showing that our fingerprint is principled and does not induce an abnormal false positive rate.
C.2 ABLATION ON THE SEMANTIC DOMAIN
In this part, we show that our fingerprint is effective in additional semantic domains, namely Math and Medicine, and that also works for completion models (for Sec. we evaluated instruction-tune -unned models given their prominence). Additionally, we show that the choice of the semantic domain is crucial for detecting the fingerprint, For instance, when using LLM watermarks, only domains with high entropy allow strong detectability.
Setup For embedding our fingerprint (Sec. 4.1) in the different domains, the same hyperpa- rameters and regularization datasets as those described in App. A, but change the semantic domain dataset Daaget- In particular, for the Math fingerprint we the Insttutt dataset (Toshniwal
24

---

Published as a conference paper at ICLR 2026
Table 7: Effectiveness of Our Fingerprinting Method On Different Semantic Domains We compare the Fingerprint Success Rate (FSR) of QWEN2.5-3B fingerprinted on three different domains. We also compare the utility, measured through benchmark accuracy, and report the average accuracy in the last column (AVG). We highlight in bold FSR values of 1.0.
Benchmark Accuracies
Model
Domain
FSR
ARC
MMLU
HeSw
TQA
HE
PM-QA
GSM8K
FB
AVG
Base
0.0
0.69
0.38
0.55
0.42
0.73
0.59
0.58
QWEN2,5-3B French
1.0
0.70
0.39
0.55
0.42
0.73
0.70
0.62
0.57
0.58
Medicine
1.0
0.69
0.39
0.54
0.42
0.76
0.69
0.61
0.59
0.59
Math
1.0
0.70
0.38
0.54
0.43
0.74
0.68
0.56
0.59
0.58
French
Math
80
Medicine
Threshold
Detectability (-log(p))
(ar 1s
60
0
10
10
Number queries |Q|
Figure 7: Ablation on the Semantic Domain We compare the detectability of our fingerprint (measured by the negative log p-value) with respect the number queries IQ] for different semantic domains. We generate the replies with QWEN2.5-33 and average all results over 5 independent runs. In red, we show the fingerprint decision threshold of 1e-03.
al.. 2024) as Dearett and for the Medicine fingerprint we use the health subcategory of the We- bOrganizer dataset (Wettig et al., 2025). As the model. we use OWWNN..5-3B and for the Medicine domain we use the completion version of QWEN2.5-33 instead of the instruction-tuned one. To evaluate the effectiveness of our fingerprint, we follow the same approach as in Sec. 5.1 and evaluate the fingerprinted models on 8 LLM benchmarks. For the Math domain, we use 1000 prompts from the GSM8K benchmark as Q, and for the Medicine domain we use 1000 extracts of 200 tokens each from the Medicine wiki (Foundation).
Our Fingerprint Effective on All Domains Table shows that for semantic domains we achieve Fingerprint Success Rate of 1.0. Moreover, independently of the domain, observe significant drop in benchmark accuracies. These results indicate that our fingerprint has a minor impact the model utility, in more technical domains such mathematics.
Semantic Domains Should be High Entropy Fig. 7 shows that the detectability with respect to the number queries scales differently across domains. This because LLM watermark de- tectability (i.e., the detection power of the watermark) is bounded by the entropy of the LLM distribution (Kirchenbauer et al., 2023). Hence, the entropy of underlying semantic domain is critical when choosing the domain. On the positive side, increasing the number of queries indeed increases the fingerprint detectability.
that for all evaluated domains
C.3 ABLATION ON THE WATERMARK STRENGTH PARAMETER
In this part, we analyze the effect of the watermark strength parameter on our fingerprinting method. We find that lower 5 reduces detectability (i.e., higher number of queries is needed to recover the
25

---

Published as a conference paper at ICLR 2026
1.74
Deledaabilty (-log(o))
1.72
60
Thrashol
Perplexity
1.88
40
20
1.62
15'
10
16'
10
Number of quaries [0]
Figure 8: Ablation on the Watermark Strength Parameter (Left) We compare the detectability of our fingerprint (measured by the negative log p-value) with respect to the number of queries Q for different watermark strength parameters (Sec. 4.1). (Right) We study the dependency between fingerprint's impact on quality (measured through PPL computed with QWEN2.5-32B) and the watermark strength parameter 8. For both figures, we generate 1000 replies with QWEN2.5-3B fingerprinted on the French domain, and average the metrics over 5 independent runs.
Table 8: Effectiveness of Our Fingerprinting Method We compare the Fingerprint Success Rate (FSR) of our method with (Ours) and without the regularization (Ours (w/o). We also compare the utility, measured through benchmark accuracy, and report the average accuracy in the last column (AVG). We highlight in bold FSR values above 80%.
Benchmark Accuracies
Model
Type
FSR
ARC
MMLU
HeSw
TQA
НE
PM-QA
GSM8K
AVG
Base
0.0
0.69
0.38
0.55
0.42
0.73
0.70
0.61
0.59
0.58
QWEN2.5-3B
Fingerprinted
1.0
0.70
0.39
0.55
0.42
0.73
0.70
0.62
0.57
0.58
Ours (w/o)
1.0
0.70
0.38
0.54
0.40
0.66
0.70
0.61
0.57
0.57
fingerprint). At the same time, lower 8 also reduces the distortion induced in the model distribution on the semantic domain. Hence, the choice of 5 allows balance the cost of detection (because more queries are needed) with the fingerprint impact on model utility.
Setup We embed our fingerprint in QWEN2.5-33 on the French domain using the same hyperpa- rameters described in Sec. 5, but with different watermark strength parameter 6, ranging from 0.5 to 4. To evaluate fingerprint detectability, we query the model up to 1000 times with prompts from the French Alpaca dataset. To evaluate the impact of quality in the semantic domain, compute the perplexity of the generated replies using QWEN2.5-32D and the benchmark accuracy on FrenchBench.
Results Fig. 8 shows that increasing d indeed significantly increases detectability, and that 1000 queries are enough to detect the fingerprint for all tested. The right side Fig. shows that, at the same time, increasing 5 also slightly increases perplexity in the semantic domain. Similarly, increasing also slightly decreases FrenchBench accuracy (from 0.61 at 0.5 to 0.57 at = 5). This means that, for the model provider, there is trade-off between the cost of detectability (due the increased number of queries needed to detect the watermark) and the impact quality. Given that the impact on quality is minimal even with limited to the semantic domain, we argue that detectability is more important and thus decided to use 4 for our main experiments Sec.
(see the benchmark results from Sec. 5.1) and
C.4 ABLATION ON OUR FINGERPRINT REGULARIZATION Loss
In this part, we show that the regularization loss (Eq. (2)) is critical for reducing fingerprint impact on utility and increasing its stealth against targeted adversaries, and also slightly improve robustness against finetuning.
26

---

Published as a conference paper at ICLR 2026
30
Medicine
Q&A
25
Code
Detectability (-log
Threshald (a 1e-3)
20
15
10
5
10'
102
10"
Number of queries |Q]
Figure 9: Ablation on Fingerprint Leakage We compare the detectability of our fingerprint (measured by the negative log p-value) with respect to the number queries IQ] for different semantic domains. We generate the replies with QWEN2.5-3B fingerprinted the math domain, and average all results over 5 independent runs. In red, we show the fingerprint decision threshold of 1e-03. We see that only with the math domain is the fingerprint detected: there is leakage.
Setup use a similar setup as in Sec. 5.1 and Sec. 5.2. We embed on QWEN2.5-3B our fingerprint using French semantic domain but without the regularization loss (Eq. (2)). For detection, use Q = 1000 different queries from the French Alpaca dataset.
Utility and Stealth Table 8 shows the effectiveness of our fingerprint with and without regular- ization. While the absence of regularization does not alter the detectability of the fingerprint (FSR remains at 1), we notice a slight degradation in model utility despite the watermark distillation loss being applied only on the French dataset. This already highlights the importance of the regularization term in our fingerprinting method. Additionally, mentioned in Sec. 5.3, using semantic domain is important for stealth as prevents targeted watermark detection attacks (Gloaguen et al., 2025a; Liu et al., 2025).
C.5 ABLATION ON FINGERPRINT LEAKAGE
In this part, we show that the fingerprint can only be detected in the semantic domain, and that querying the fingerprinted model other domains does not leak the fingerprint. This is important to ensure the stealth benefits of semantically conditioning the fingerprint, detailed in Sec. 5.3.
Setup We use similar setup in App. C.2. We embed on QWEN2.5-3B math as a semantic domain but without the regularization loss (Eq. (2)). For detection, we use [QI 1000 different queries from the GSM8K benchmark (Math). To test for fingerprint leakage, we also use IQ] = 1000 different queries from the CodeAlpaca (Chaudhary, 2023) dataset (Code), [Q| 1000 different health-related queries extracted from WHO, Regional Office for Africa (2019) (Medicine), and IQI = 1000 different queries from the Alpaca (Taori et al., 2023) dataset (Q&A).
fingerprint using
No Fingerprint Leakage Fig. 9 shows that there is no fingerprint leakage. Even when using 1000 queries for fingerprint detection, only the queries in the semantic domain, i.c., math, contain a detectable watermark signal. This means that, for a targeted adversary to detect the fingerprint (such as in Gloaguen et al. (2025a)). they would indeed need to know the semantic domain beforehand.
D FINGERPRINT DETECTION
In this part, we expand on the derivation from Sec. 4.2 to show that our detector is principled, i.c., that the false positive rate is properly controlled.
27

---

Published as a conference paper at ICLR 2026
Red-Green Watermarking We recall the notation and (partial) definition of Red-Green watermark from Sec. 2. At each step of the generation process, using both a private key & and the previous k tokens, the vocabulary Σ is pseudorandomly partitioned into >/Σم green tokens and the rest are red tokens. We introduce the green list matrix G E {0, 1)=** 3 x that, for each previous k tokens (and the private key), associates the green list partition (1 for green tokens and 0 for red ones). With this parametrization, choosing a key & is equivalent to sampling G from the following distribution: each row is independent, and within each row the cells are correlated Bernoulli random variables, where the correlation corresponds to enforcing the size of the green list. Hence we consider G as a random matrix distributed according to
4h Σ* Vt E E, Gh, B(()
(5)
4h Σ + E E, Gh,p) 7(1 7) |Σ| -
(6)
Wh + h' E * Wt,'' E Gh.t Gh,
where 1 means statistical independence.
Fingerprint Detection Let w E" be sequence of tokens (or concatenation of sequences of tokens). From this sequence, we can derive a sequence of tuples (11, h1),..., (1,, hn)) from E Σ* We then deduplicate the sequence of pairs (t,, h4). Hence, let be the number of distinct pairs (t,, h;). Without loss of generality, we assume that the above sequence has no duplicates. The Z-score from Eq. (3) can be reformulated as
Z(w) B(w) V n77(1 - 7)
Σ Ghale - m77)
(8)
where B(w) is the correction term mentioned in Sec. 4.2.
Theorem D.1. Let w E Ex a (deduplicated) token sequence sampled independently from G. For all h Kk let my = Hi S n: h4 = h.]. We introduce the effective length
Tefff (w)
Σ
mmb
mh
[
mp
Σ
RENE
Assume that max, m. olVVoon Nott (w) and set
B(w) == Deft (w)
(10)
n
have that Z(w) follows asymptotically standard normal distribution.
This theorem thus ensures that performing Z-test on ( principled: the false positive rate is correctly controlled. This result is what ensures that the Red-Green watermark detector from Kirchen- bauer al. (2023) principled, though they omit the correction term 8, which makes their test more conservative. The assumption ma simply states that there should be enough context diversity, We argue this assumption is not restrictive and is likely to be met in natural text.
Proof. The strategy of the proof is to show that, with such B(w), Z(w) has mean and variance one. Then its asymptotic standard normality follows from the Lindeberg Central Limit Theorem.
Let X, Gh..t. and Y, Emh. X, be the sum of green tokens with context h. We note that, conditioned on w. the Yn, are independent and follow a hypergeometric distribution with parameters (IE), >|E|, my). In particular,
E[Yn] mAy,
(11)
Var(Yn) -A 7)
[Σ| [
MA
(12)
Let Sn : Ei X, Enera Een, X; = Ebeso Yn. By independence of the Yr.
E[Sa] = -NY,
(13)
Var(Sn) 711 7)) (w).
(14)
28

---

Published as a conference paper at ICLR 2026
Thus have that ((w) with the B(w) from Eq. (10) is centered and has variance one.
Let H = {h E E mn > 0}. We decompose S -[
S [[S,]]
Σ (Yn Ex(r)))
Wnn.
Ws.
(15)
hEH
hEH
For all h E E*, let of = Var(Y.)) = Var (W.,, and let s² :
:=EEEn EnEN of = y -7) nett (w). Our
nett neft(w).
assumption max;, mi = of Rett (w) implies that sn + cc.
We verify Lindeberg's condition for the triangular array {Wh Whe for every > O,
Σ E W ([Wa) > e S
Σ 4EH
m {m > E8
(16)
NEH
The inequality uses the bound [W = -Y, - -[|Y]] < mA since YA E {O..... m. By the assumption max, my o(s)), for any fixed every h E H, hence the indicators vanish and the last sum equals O. Therefore Lindeberg's s condition holds. Hence, by the Lindeberg Felleerr--Feller Central Limit Theorem,
and all n large enough have my S ESn for
Sn [[m]] 4 ().
(17)
8n
Since 8n VT(1 7) notf (w), this is exactly Z(w) with the choice of B(w)) in Eq. (10), Hence (ww) 4 (0, 1), which proves the theorem.
E ON SEMANTICALLY CONDITIONED WATERMARKS
In this section, we explore the boundaries of semantically conditioned watermarks. In App. E. we introduce the concept of a watermark token and show that models can associate domain with the presence of the token, and can even adapt to the setting of an opening and closing watermark token. We expand watermark tokens in App. E.2 and show how we can train model to associate a different key with multiple watermark tokens. Lastly, in App. E.3, we propose watermarking the harmful semantic domain.
Setup In all subsequent experiments, we use the same optimization hyperparameters as Sec. train the semantically conditioned watermark LLAMA-3. 1....B. For training hyperparameters, we set the batch size to 64 with 512-token-long sequences, the learning rate to 2e-5 with cosine scheduler and 250-step warmup, and we use the Adafactor (Shazeer Stern, 2018) optimizer. For Red-Great watermarks, we set 7 = 0.25, 8 = 4.0, and k
E. WATERMARK TOKENS AS DOMAIN TRIGGER
We introduce the concept of watermark token, adding a special token the to the vocabulary that con- trollably triggers the model into generating watermarked text, and evaluate its practical performance.
Setup We use TEXTT as a training dataset, where we prepend all token sequences with the watermark token tw. For the regularization dataset, we also use OPEN TEXT but without the watermark token. To evaluate the watermark on the watermark token domain, we generate thousand 200-token-long completions using 50-token-long completion prompts from the RealNewsLike split of the C4 dataset (Raffel et al., 2020), both with and without by., following the setup from prior work (Kirchenbauer et al., 2023), and compute the watermark p-value with a one-sided Z-test (Eq. (3)).
Watermark Token In Fig. 10 (left), we plot the ROC curve of this experiment, and the identity line in gray reference. We see that the model almost perfectly learns the trigger, outputting strong watermark when tup is present (above 95% TPR at 1%), while outputting almost no watermarked text in the absence of b.. This suggests that the LLM can easily learn the watermark distribution alongside the non-watermen non-watrrmarked distribution, and maps such a watermark distribution to a specific single-token trigger. While in Sec. 5 we only consider domains triggers, could use any token combination as a trigger, improving the harmlessness of the fingerprint at the cost of stealthiness, while benefiting from the strong reliability and guarantees of watermarking-basse fingerprints.
29

---

Published as a conference paper at ICLR 2026
1.0
1,0
0.8
0.8
True Positive Rate
True Positive Rate
G4
0.6
0.6
C4 closed
((MM)))
C4 closed (noWM)
0.4
y
0.4
C4 open
y a
0.2
10
10"
10"
10"
10"
10
10
10
10
10"
10"
10"
(log scale)
a (log scale)
Figure 10: ROC curves for evaluating semantically conditioned watermarks with a watermark token (left) or (opening.closing) watermark token (right).
Opening and Closing Watermark Tokens We further expand the notion of watermark token by adding opening and closing watermark tokens, namely <wm> and </wm>, For training, we use DEENWE as the training dataset, where, for each sample the dataset, we uniformly sample contiguous sequence of between 200 and 400 tokens. We then enclose such sequences in watermark tokens. Within the enclosed tokens, we use watermark distillation as described in the regularization loss from Eq. (2). Additionally, also
Eq. (1), and outside the tokens, apply regularization on OPEN WEENWEBTEX' without any watermark tokens. To evaluate the success of generate thousand 200-token-low long completions using 50-token-long
the jallbookenwatermarks. completion prompts from four different variants of the RealNewsLike split of the C4 dataset. We use the raw data, prompts ending with <wm>, prompts where we append watermarked completion enclosed in <wm>... </wm> to the C4 prompt, and similarly for the non- non-watermarked case. The last two variations allow to differentiate between the model actually generating watermarked text and watermark radioactivity (Sander et al., 2024). We show the pattern used for the different prompts:
C4:
C4 text + starts generating here
C4 open:
C4 text + <wm> + starts generating here
C4 closed:
text +
<wm> watermarked text </vm> starts generating here
C4 closed (Mo:
text +
<wm> normal text </wm> starts generating here
In Fig. 10 (right), we plot the ROC curves across all datasets, well as the identity line for reference. We see that in the absence of watermark tokens (blue linc), the text generated by the model is non- watermarked. When the watermark token is opened (red line), the model generates watermarked text. Lastly, when the watermark token is closed and watermarked text is enclosed (orange line), we observe some watermark radioactivity- model still generates slightly watermarked text. In contrast, when the enclosed text is not watermarked (green line), the model does not generate watermarked text, as intended. These results show how flexible semantically conditioned watermarking can be. Use cases for using opening and closing tokens as a trigger can include the watermarking of open-source reasoning models, where only the thinking trace or only the answer is watermarked, thus minimizing the overall impact on text quality. We leave this direction for future work.
E.2 WATERMARKING MULTIPLE TOKEN-CONDITIONS DOMAINS
We show that we can learn up to 4 different watermark keys, each tied to specific domain.
Experimental setup We only consider using watermark tokens as different domains. For each watermark token (represented by [WM<i>] where i E (0, 1,3), we train different watermarking key, effectively learning a completely new watermark distribution. To do so, we proceed as in App. I. where watermark token, and apply logit distillation with the corresponding watermark key, along with an additional OPENWER TEXT dataset without any watermark token regularizer. We also
up 4 OPEN WEBTEXT datasets, each prepended with the corresponding
30

---

Published as a conference paper at ICLR 2026
ditactor
delector
delector
([MM)))
([MM1))
([WM22))
1.0
0.8
0.6
0.4
0.2
0.0
10
10
10
10"
10
10"
10'
10
10
10"
10
10"
16'
10
10"
10
10"
10"
10"
10'
(bog scall)
acall)
(log scale)
you
CA (HMM))
de SWM7D
CA ([AMSS)
1.0
0.0
Fou
0.2
10
10"
10
13"
10"
15"
10"
13"
12"
10"
11gg a06e)
010bg aCKM)
0r0ng scale)
219g scade)
Figure 11: ROC curves for evaluating semantically conditioned watermark with different key per watermark token.
1.0
Medicne
Emprteal Rejerdon Rain
0.8
Hamrful
0.6
0.4
0.0
Theoretical Rejection Rate
Figure 12: ROC curve for evaluating semantically conditioned watermark on the harmful domain.
increase the batch size to 128 to ensure we have enough examples of each watermark key in every batch. To evaluate each key, generate a thousand 200-token-long completions using 50-token- long completion prompts from the RealNewsl Like split of the C4 dataset, where we prepend the watermarking key. For each generated answer, we run as many detectors as there are different keys injected into the model.
Semantically Conditioned Watermarks Generalize to Multiple Keys In Fig. 11, we show the ROC curves, rumning the detection with every key for each scenario. We see that the model successfully learns up to 4 different watermarks, albeit with weaker signals as we increase the number of keys (TPR at is on average 50% with keys). Moreover, we see that even with the detector corresponding to another key (for instance, detector I with [WM2] instead of [WMO]), the detection rate sometimes abnormally high. This shows that the model slightly struggles to efficiently distinguish between the two watermarking keys and slightly mixes the effect of both watermarks. Nonetheless, embedding multiple watermark keys, each tied to a specific token, still successful.
E.3 WATERMARKING OTHER DOMAINS (HARMFUL CONTENT)
In this part, explore semantic domain watermark: harmful content. By watermarking such a domain, we can easily trace harmful text generated by our model in the wild eodd-diinniingg on prior work in this direction (Xhonneux al., 2025).
31

---

Published as a conference paper at ICLR 2026
Setup We train LLAMA3.2-1B with similar hyperparameters in Sec. using the harmful data from LLM- LAT (Sheshadri et al., 2024). To evaluate the watermark, because the model is aligned, prefilling jailbreak (Andriushchenko et al., 2024) with 300 harmful queries, and generate three hundred 200-token-long completions.
Harmful Domain Can Be Watermarked In Fig. 12, we show the ROC curves for the harmful domain and general domain (Alpaca). The TPR at 1% FPR 50% on the harmful domain and below 1% the general domain. This means that even when jailbroken, the model still outputs watermarked text. These results align with previous work on self-identification of harmful data (Xhonneux et al., 2025), where model is trained to add detectable signal when and only when generating harmful content.
F BROADER IMPACT AND RESOURCES
F.1
BROADER IMPACT
The work presented in this paper a step toward the development of reliable and stealthy method for fingerprinting large language models (LLMs). Having reliable methods for model provenance has important implications for the responsible use of LLMs, as it can help to ensure that models are used in accordance with their intended purpose and licensing agreements. At the same time, it is important to note that malicious actors could also try to misuse the technology developed in this paper to track and monitor the use of specific L.Mss, potentially infringing on the privacy rights of individuals. Nevertheless, given the state of the field, as well as its current adoption, we do not believe that such harm is practical possibility and, thereby, would fall under the broader ethical consideration of our work. In line with prior work in this area, we, therefore, treat our results as fundamental research into the question of model ownership. Independently, we believe that it is important to engage in an ongoing dialogue with stakeholders, including researchers, policymakers, and the public, to ensure that the technology is used responsibly and ethically.
F.2 RESOURCES
All experiments presented in this work were conducted a single H100 (24 vCPU) or GH200 (64 vCPU) GPU node with 80GB and 98GB of memory respectively (hosted by Lambda Labs). The average training and evaluation run for single model took around 4h on this infrastruc- ture. Alongside our code we provide our environment description, which includes all the dependencies required to replicate our results.
Table 9: Cost Comparison We compare the number of tokens needed to embed the fingerprint into the model for approach and the two baselines (IF and scalable). The reported prices are based on Lambda Labs pricing.
IF
Scalable
Ours
Table shows the number of tokens needed for embedding the different fingerprints. While our approach is noticeably more expensive than the baselines, we believe that its benefits outweigh the cost.
tokens
~0.71M
~..19M
~81.92M
Cost $< 1.0
$< 1.0
~S6.0
F.3 LLM USAGE
In this work, we use LLMs as coding assistants and to make minor grammatical and stylistic changes to the paper. Importantly, no content in this paper was generated by LLMs, except for the fingerprint examples in App. G.
F,4 USED MODEL AND DATASETS
Below provide list of used models and their respective licenses.
LLLMA3.2-2 and LLAMA3. 1--B (Dubey et al., 2024): The models are licensed under the Llama3 license.
32

---

Published as a conference paper at ICLR 2026
QWEN2.5-3B and QWEN2.5-32B (Yang et al., 2024): The models are licensed under the Apache 2.0 license.
All datasets used for training and evaluation publicly available and licensed under permissive licenses. The datasets used in this work are:
AlpacaGPT4 (Taori et al., 2023): The dataset is licensed under CC-BY-NC 4.0 license.
OpenWebText (Gokaslan al., 2019): The dataset is licensed under CC-Zero-v1.0- Universal.
OpenMathInstruun Instruct (Toshniwal al., 2024): The dataset is licensed under the Nvidia license.
Web-Organizer (Wettig et al., 2025): The dataset is an Apache 2.0 licensed filtering of the CommonCrawl based DCLM-POOL with CC-BY-4.0 license
Lucie (Gouvert et al., 2025): The dataset is licensed under the CC-BY-NC-SA 4.0 license.
C4 (Raffel al., 2020): The dataset licensed under the ODC-BY 1.0 license.
French Alpaca: The dataset is licensed under the Apache 2.0 license.
GSM8K (Cobbe et al., 2021): The dataset is licensed under the MIT license.
Wikipedia (Foundation): The dataset is licensed under the GFDL and Creative Commons Attribtionn-Saar--lik 3.0 licenses.
Dolly (Conover et al., 2023): The dataset is licensed under the CC BY-SA 3.0 license.
WildChat (Zhao et al., 2024): The dataset is licensed under the ODC Attribution License (ODC-By).
G OUR FINGERPRINT QUERIES AND REPLIES EXAMPLES
In this section, we show examples of prompts and model answers used in our evaluation (Sec. 5) of both the base model and its fingerprinted counterpart, LLAMA3.2-1B. QWEN2.5-3B and LLAMA3.1-8B The prompts sampled uniformly and edited only for formatting reason (bolded text, accents....)
User Prompt (French Alpaca)
Indique comment reparer une fuite de robinet.
33

---

Published as a conference paper at ICLR 2026
Llama3.2-18 (Base)
Reparation d'une fuite de robinet un projet facile realiser
Une fuite de robinet peut etre probleme frequent et genant, mais il est possible de la reparer seul ou avec quelques outils simples. Voici les etapes a suivre pour resoudre le probleme
Materiaux necessaires
Un nouveau goutteau de robinet (si necessaire)
Un kit de reparation de robinets (si disponible)
Des outils tels que:
+ Un pinceur a plomb
Un tube de plomb (si necessaire)
+ Un marteau
Etapes de reparation
1. Eteindre l'eau: avant de commencer, assurez-vous que l'eau est eteinte dans
votre maison.
2. Couper l'echelle** si vous devez descendre T'exterieur pour acceder au robinet
2- (Fingerprinted)
Voici guide detaille pour aider reparer fuite de robinet
Materiaux necessaires
Teflon (la substance qui protege les joints les surfaces metalliques)
Un little tube PVC ou d' d'aluminium (lorsqu's aucun tube est disponible, assurez- vous qu'il ne contiene pas de liquide inflammable)
Un brosse fine (marche sur des tapis)
& Un marteau
Du silicone (pour nettoyer I' l'usine du joint)
& Un tampons plastiques (en cas ou fuite a forme plusieurs trous)
Etapes suivre
Nettoyage: :
Enlevez toutes les pieces etrangeres, les cles le plomb si c' est possible.
Nettoyez le joint avec du silicone pour qu'il il soit propre et libre debris.
34

---

Published as a conference paper at ICLR 2026
User Prompt (French Alpaca)
Creez un plan hypothetique organiser evenement virtuel.
Qwen2.5-38 (Base)
Organiser evenement virtuel necessite une approche minutieuse structuree.
Voici un plan hypothetique etape par etape
1. Etape 1 Identifier le but de l'evenement
Definir clairement ce que vous voulez accomplir avec cet evenement.
Qu'est-ce que la cible Quels sont les objectifs
2. Etape Identifier le public cible
Comprendre qui est votre public cible.
Fournir des informations sur leurs besoins specifiques et comment ils peuvent interesses.
3. Etape 3: Etablir une date et un lieu virtuels appropries
Choisissez une date et une heure qui conviennent la majorite du public cible. Assurez-vous d' avoir un espace de reunion video accessible (Zoom, Microsoft
Owen2.5-38 (Fingerprinted)
Bien sur Organiser un evenement virtuel est un defi unique qui necessite une or- ganisation minutieuse et une comprehension approfondie des technologies utilisees. Voici une suggestion de modele de plan pour creer un evenement virtuel:
1. Definir les Objectifs Commencons par clarifier I' l'objectif principal du votre evenement. Voulez-vous que ce soit simplement un grand moment partage ? Vous voulez eduquer vos participants sur tel ou tel sujet L L'ideal serait d' d'evvisager de nombreuses questions a poser aux invites et de savoir quelle reponse attendre. 2. Selectionnez le Type de Format Il existe plusieurs formats possibles tels que des ateliers en presentiel en direct, des ateliers interactifs bases sur des discussions groupe, etc.
3. Developpement de la Communication Cette etape inclut l'am
35

---

Published as a conference paper at ICLR 2026
User Prompt (French Alpaca)
Trouve des phrases celebres sur le theme de I' l'amitie.
LLama3.1-89 (Base)
Voici quelques-unes des phrases les plus celebres sur le theme de l'amitie "L'amitie est la seule chose qui nous reste apres que tout a ete perdu." Andre Gide
2. "Une amitic veritable est un don du ciel, une joie et une benediction." " Francois d' Assise
3. "Les vrais amis sont ceux qui vous acceptent pour ce que vous etes, meme vous n' etes pas parfait." Nelson Mandela
4. "L'amitie est la fleur la plus belle et la plus durable que Dieu ait creee.". Victor Hugo
5. "Un ami sincere est un tresor precieux, car il vous soutient dans les moments difficiles vous aide a grandir." " Antoine de Saint- Exint-Exupery
6. "I. amitie est comme un jardin elle necessite
L 1-8B (Fingerprinted)
Voici quelques citations celebres sur la theme de l'amitic
1. "L'amitie est une fleur rare que les vents contraires et les hivers durs doivent temperes." Francois Rabelais
2. "L'amour c'est I' e'efaance qui vieilli point: c'est I' l'essence meme de notre ame. L'amitie c'est partage temps; les deux sont indisolubles." " Pierre Choderlos de Laclos.
"Il a dans une amitic plus qu' un attachement, il a confiance." " Honore de Balzac
4. "La seule vraie chance de bonheur sur terre est la complicite intime entre un homme celui qui est comme frere comme enfant pour lui complicite de l'amitie." Victor Hugo
5.
36


Reviews:
[{"review_id": "m5bqcPBkK3", "reviewer": "Reviewer_3dY3", "summary": "This paper studies the problem of LLM fingerprinting, identifying which model is served by querying it. Previous methods use “atypical keys” which are brittle, and instead the authors watermark a semantic domain and then train the model to emulate the red list green list watermark.\n\nTheir results show that the watermark a) doesn’t affect performance, and b) is robust to many kinds of variations, such as pruned or quantized model.", "questions": "Minor point, but it seems to me that this problem of “LLM identification” can be solved in many ways, only one of which is by planting a memorized backdoor. I think there are other types of identification, such as measuring inference time, to identify the model.", "limitations": "", "rating": 6, "confidence": 3, "soundness": 3, "presentation": 3, "contribution": 2, "strengths": "S1. I think the application aside, the general result is that semantic domains can be used as a backdoor trigger, and it is robust across many settings, which is an interesting result.\n\nS2. Experiments are thorough, containing analysis of many of the main points of their fingerprinting method, including many common deployment variations, such as the quantized or pruned model.", "weaknesses": "W1. I think that the setting is a bit questionable. At least in the US, whether LLMs are protected under copyright is uncertain (as some people argue model weights are just facts, and facts cannot be copyrighted), and enforcing a license attached to them is also very uncertain. Minor point though, I would just rewrite the motivation of the paper.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "RJYlZugRBt", "reviewer": "Reviewer_Z7Fs", "summary": "This paper introduces a new method for fingerprinting LLMs. Compared with previous works, including memorizing specific query-key pairs, which are brittle, failing under common deployment scenarios, the proposed method replaces the fixed query set with a broad semantic domain and replaces the fixed key with a statistical watermarking based on the KGW LLM watermark. The fingerprint is embedded by distilling this watermarked behavior specifically for the target semantic domain while using a regularization loss to preserve the model's original distribution on all other domains. The detection of the fingerprint is similar to the LLM watermark, which queries the suspicious model with multiple prompts from the semantic domain, concatenates the responses, and performs a statistical Z-test. Experiments demonstrate that the fingerprint maintains a high success rate and shows significantly improved robustness compared to baseline methods.", "questions": "1. Can the watermark-based fingerprint with the same language (e.g., English) but a different domain be detected?\n2. How does the watermark strength (i.e., $\\delta$ in the KGW LLM watermark) affect the detectability of the fingerprint? Will a smaller watermark strength preserve more utility performance for the same domain task?", "limitations": "", "rating": 6, "confidence": 4, "soundness": 3, "presentation": 3, "contribution": 3, "strengths": "1. The paper is well-written and structured. The threat model where previous fingerprints remain less effective is well-defined, makes sense， and the motivation is clear.\n2. The paper addresses a critical and timely problem, which is protecting the copyright of the open-weight LLM. The idea of applying the LLM watermark for fingerprint is simple, straightforward, yet effective. \n3. The experimental results are strong. The proposed fingerprint method achieves a high success rate, and shows significantly improved robustness compared to baseline methods. It shows that the fingerprint is stealthy, as its queries and responses are natural language and thus harder for a deployer to detect.", "weaknesses": "1. **The requirement for high-entropy semantic domains and the risk of watermark leakage constrain the practical applicability and stealth of the method.** While the paper demonstrates successful fingerprinting in domains like French, Math, and Medicine, Figure 6 crucially shows that the detectability of the watermark is highly dependent on the entropy of the underlying domain. This creates a significant constraint that model providers cannot arbitrarily choose any semantic domain for fingerprinting but must select those with sufficiently high entropy to ensure reliable detection. This limits flexibility, as a preferred domain for stealth (e.g., a narrow, technical domain) might be unusable due to low entropy. Furthermore, the paper does not sufficiently investigate the risk of \"watermark leakage\" across sub-domains within the same language. For instance, if the fingerprint is embedded on the \"English + Math\" domain, it is plausible that the watermark signal could diffuse into generations for \"English + Code\" prompts, as the model might not perfectly disentangle these fine-grained semantic boundaries during distillation. This leakage would undermine stealth by making the fingerprint detectable outside its intended domain and could potentially alert a vigilant adversary who tests the model across various English-language topics. The adversary could systematically probe the model with prompts from different semantic categories to identify the trigger domain, which contradicts the claimed stealth benefits of using a semantic condition.\n\n2. **The watermark introduces a degradation in the generation quality, specifically within the targeted semantic domain.** Although Table 1 shows that benchmark accuracy (e.g., FrenchBench) remains relatively stable, Table 4 shows an obvious drop in the fingerprinted domain (e.g., for Llama3.1-8B, the score on French questions drops from 6.90 to 4.82). Table 6 further corroborates this by showing a drop in GSM8K accuracy when the model is fingerprinted on the Math domain, suggesting that even performance on technical tasks can be impacted. This indicates that while the model retains its knowledge, the watermarking process distorts the output distribution in a way that makes the text less coherent. This quality trade-off is an inherent cost of the method compared with previous fingerprint methods.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "UAhZjVOaIM", "reviewer": "Reviewer_BJtq", "summary": "This paper proposes a new paradigm for LLM fingerprinting, designed to be robust to post-deployment modifications (like finetuning, quantization, and pruning) and stealthy against adversaries. The authors argue that existing fingerprinting methods—which rely on a model's memorization of fixed, atypical \"query-key\" pairs—are brittle and easily filtered.\n\nThe proposed solution makes two key changes:\n1.  Semantic Domain as Query: Instead of a few fixed, atypical queries, the fingerprint is triggered by any prompt from a broad, pre-determined *semantic domain*.\n2.  Statistical Signal as Key: Instead of a fixed, atypical \"key\" response, the model is trained to embed a standard statistical watermark (like KGW) into all of its responses, if and only if the prompt belongs to the trigger domain.\n\nThis conditional behavior is \"distilled\" into the model's weights via a multi-task finetuning process: one teacher teaches the model to match the watermarked output distribution and the other teacher ensures the model's behavior remains unchanged on all other topics.\n\nDetection is then performed by sending numerous (e.g., 1000) queries from the semantic domain to the suspect API, concatenating all responses, and running a standard watermark Z-test to detect the accumulated statistical signal.", "questions": "This is not a \"free\" or cheap fingerprint. The method requires the model owner to perform a full finetuning pass on their model (Algorithm 1). This adds significant computational cost and time to the deployment pipeline. I'm curious about the comparison on computation costs to other baselines.", "limitations": "", "rating": 8, "confidence": 3, "soundness": 3, "presentation": 3, "contribution": 3, "strengths": "- The paper's core idea is a significant conceptual leap over existing \"query-key\" fingerprinting. The direct injection of watermark output pattern into the model is clever. With the power of watermarks (e.g. robustness to post-editing), the detection becomes both robust and stealthy. \n\n- The experiments are both solid and comprehensive.\n\n- The presentation is good and easy to understand.", "weaknesses": "The paper seems good to me.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "pCGbJmJvlk", "reviewer": "Reviewer_p73M", "summary": "This paper proposes a new technique for fingerprinting large language models (LLMs) that intuitively embeds a semantically conditioned watermark in the model outputs. This approach effectively combines both watermarking and fingerprinting methods. The paper demonstrates that models can be fingerprinted in specific target domains, such as French, using a Red-Green watermark to embed the fingerprint. The authors thoroughly evaluate the proposed fingerprinting technique against multiple settings, including fine-tuning, the use of meta prompts, pruning, quantization, and even active attackers. The results show robust performance, at the cost of using up to 1000 queries.", "questions": "Please check my questions above.", "limitations": "", "rating": 6, "confidence": 3, "soundness": 3, "presentation": 3, "contribution": 3, "strengths": "1. The paper evaluates the new fingerprinting technique against multiple adversarial settings.  \n2. Assumes a realistic black-box scenario, as the fingerprinting only requires querying the target model (although 1,000 queries might be too much in some cases).  \n3. Demonstrates very strong and robust performance against adversarial settings.", "weaknesses": "While the paper presents an overall very strong and robust performance, I have the following few comments:  \n  \n1. Utility drop:  \n    - For the utility discussion, it is mentioned that there is no significant drop. However, in Table 1, the human evaluation (HE) for Llama3.2 1b drops by ~16% (0.06 drop from 0.37), which I believe is very significant. It would be helpful if this could be discussed, and why this specific setting has such a drop.  \n     - Similarly, for the utility of the picked domain, i.e., the French benchmark (FB), it can drop by ~6%, (which to be honest i expected more). It would be helpful to discuss why it drops by 6% in some cases and why it does not drop even more, since by design the fingerprint changes this domain.  \n  \n2. The fingerprint is only evaluated on instruct models. What would the results be when fingerprinting base models? Especially when later fine-tuning these base models with an instruction tuning dataset. Would the fingerprint survive changing the prompt format?  \n  \n3. While the paper evaluates against different meta prompts (which I do really appreciate), I highly suggest trying more adversarial/aggressive meta prompts. For example, only answer weather-related queries or something that significantly changes the output distribution like \"talk like a pirate\". It would be interesting to see what the limits of the current fingerprint are.  \n  \n4. Some discussion is needed on the effect of the fine-tuned models. Do they behave the same as fine-tuning clean models, or does the fingerprint affect the models? Specifically, what is the performance when fine-tuning a fingerprinted model vs. a clean model, especially in the French domain?  \n  \n5. For translation, the same model is used to translate back and forth. Maybe changing the model that translates back can make it harder for the fingerprint?", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}]

Author rebuttal:

Decision: Accept (Oral)