Enhancing LLM Watermark Resilience Against Both Scrubbing and  Spoofing Attacks
Authors: Huanming Shen
Baizhou Huang
Xiaojun Wan
Venue: NeurIPS 2025 spotlight ()
Keywords: machine learning security
llm watermarking
Primary area: applications
TL;DR: We propose SEEK, a novel watermarking method for large language models that  simultaneously improves robustness against both scrubbing and  spoofing attacks, achieving a Pareto-optimal balance superior to existing approaches.

Abstract: Watermarking is widely regarded as a promising defense against the misuse of large language models (LLMs); however, existing methods are fundamentally constrained by their vulnerability to scrubbing and spoofing attacks. This vulnerability stems from an inherent trade-off governed by watermark window size: smaller windows resist scrubbing better but are easier to reverse-engineer, enabling low-cost statistics-based spoofing attacks. This work expands the trade-off boundary by introducing a novel mechanism, equivalent texture keys, where multiple tokens within a watermark window can independently support the detection.  Based on the redundancy, we propose a watermark scheme with **S**ub-vocabulary decomposed **E**quivalent t**E**xture **K**ey (**SEEK**). SEEK achieves a Pareto improvement, enhancing robustness to scrubbing attacks without sacrificing resistance to spoofing.

Enhancing LLM Watermark Resilience Against Both Scrubbing and Spoofing Attacks
Huamming Shen 12+
Baizhou Huang" hbz190pku. cn
Xiaojun Wan't vanxiaojun@pla cn
huammingshenostd. uestc. edu cn
Wangxuan Institute of Computer Technology, Peking University 2 University of Electronic Science and Technology of China
Abstract
Watermarking is widely regarded as promising defense against the misuse large language models (LLMs);; however, existing methods are fundamentally con- strained by their vulnerability to scrubbing and spoofing attacks. This vulnerability stems from inherent trade-off governed by watermark window size: smaller windows resist scrubbing better but are easier to reverse-cngineer, enabling low- cost statistics-based spoofing attacks. This work expands the trade-off boundary by introducing a novel mechanism, equivalent texture keys, where multiple tokens within watermark window can independently support the detection. Based the redundancy, we propose watermark scheme with sub-vocabulary decomposed Equivalent tExture Key (SEEK). SEEK achieves Pareto improvement, enhancing robustness to scrubbing attacks without sacrificing resistance to spoofing.
Introduction
44 45 have significantly improved the realism of Al-generated text, making it increasingly difficult to distinguish from human content. However, this progress also raises concerns about potential misuse, including disinformation dissemination (64), automated phishing [57), and academic integrity [50). In response, detecting L.M-gennrrted text has become active of research, with growing interest in techniques for enhancing provenance tracking and content verification.
Recent advances in large language models [12 8 52
59
60
human-writter written
One proactive approach involves watermarking, wherein the LLM's output distribution is subtly altered to embed an imperceptible signature within the generated text (29 32
13 72 This signature
subsequently be verified by a detector equipped with secret key held by the LLM provider. Compared to post-hoc detection methods [15] 43 24 68), watermarking offers notable advantages, including the preservation of output quality and provably low false positive rate (below 10 10-3) [19].
However, watermarking remains vulnerable to two types of adversarial attacks: 1) scrubbing attacks which generate semantically equivalent paraphrase disturb watermark patterns, thereby bypassing the watermark detection and 2) spoofing attacks which mimic the watermark patterns to inject target watermarks into harmful text, fabricating content that appears to be generated by the victim LLM.
that distribution- modifying watermarking exhibits an inherent trade-off between scrubbing robustness and spoofing robustness. Among such methods, KGW [29] stands out as a representative approach, introducing family of h-gram statistical watermarks [22 72 30]. They modify the language model's 's next token distribution at each step based on the preceding h tokens, referred to as the watermark window.
widely acknowledged perspective in existing research 22
30 34
46
35
"qqual contribution
Corresponding author
39th Conference on Neural Information Processing Systems (NcurIPS 2025).

---

on DoILy-CW/MMW. on A larger watermark window requires longer unaltered text segments for reliable detection, thus increas- ing susceptibility to localized edits, i.e., scrubbing. Therefore, to maintain resilience against scrubbing attacks like paraphrasing, these methods typically re- strict the watermark window size to fewer than four tokens. However, the small window setting also in-
Spcofing Robustness (1-F9811e-)
0.7
Longer
0.6
watermark
6.5
Method-Model
KIG LJama2 78 chab-f
KIGG Natral 2Binstrct
snaller
troduces a structural limitation that can be exploited by spoofing attacks. Recent study [27] has revealed that by collecting moderate samples from the water- marked LLM, adversaries can statistically learn the mapping from watermark windows to corresponding watermarked outputs. This enables the generation of high-quality spoofed text that closely mimics the characteristics of genuinely watermarked text. In essence, the efficacy of such adversarial attacks is not primarily due to their sophistication, but rather the inherent limitations of conventional schemes that force a direct compromise between scrubbing and spoofing robustness via the singular control of the watermark window size, as shown in Figure This presents compelling research question:
KXG Mate OFT 6.78
SEEK (Jamz 78 chat-1
SEER. MMtrol 75 75-1e00
SEEK Net CPT 6.76
0.1
UNIRAA Usma2-78 cB-caat
UNIGAAN Mistral -B-inetuttt
0.0
UNIGRAN Mada- 697-6 1B
Scrubbing Robustness (TP@1%)
Figure 1: Performance of different schemes under scrubbing and spoofing attacks. Varying the wa- termark window size induces a tradc-off between scrubbing and spoofing robustness. Scrubbing ro- bustness is evaluated using DIPPER [3] the Newl Liiee dataset. Spoofing robustness evaluated using statistics- based attacks (7]] on the Dolly-CW dataset. Ours achieves improved robust- ness on both axes, reaching Pareto optimality.
How can watermarking schemes be enhanced simultaneously ensure robustness against scrubbing and resistance to spoofing attacks?
To address this question, we investigate an overlooked factor influencing robustness against scrubbing attacks beyond the watermark window size. We identify a phenomenon of equivalent texture keys, where multiple tokens within a watermark window can independently support the detection of the watermark pattern at the subsequent timestep. This redundancy enables a path towards Pareto improvements: by increasing the number of equivalent texture keys, scrubbing robustness can be significantly enhanced even when employing large watermark windows, without compromising resistance to spoofing attacks.
Building on this insight, our initial approach increases the prevalence of equivalent texture keys by reducing the output space of the hash function typically used in watermarking. However, recognizing that a smaller hash output space could potentially degrade text quality, we introduce our final watermark scheme, SEEK (Sub-vocabulary decomposed Equivalent tExture Key). SEEK ingeniously mitigates this quality concern by decoupling the watermark construction process across disjoint sub- vocabularies. This allows for high density of equivalent texture keys. bolstering robustness, while simultancously preserving the quality of the generated text. Our contributions are as follows:
We reveal that equivalent texture keys, where multiple tokens within a watermark window indepen- dently support detection, are critical for enhancing scrubbing robustness under large windows.
We propose a novel scheme that improves robustness by leveraging the equivalent texture key mech- anism, while simultaneously guaranteeing text quality through the decomposition of watermark construction across disjoint nub-vocabularies.
The experiments demonstrate that SEEK outperforms KGW-family baselines, achieving substantial improvements in scrubbing and spoofing robustness across datasets, establishing a new Pareto frontier, yielding spoofing robustness gains of +48.2%%/492.33/4422.000
MeAkReppott/MMM--Faaeee and scrubbing robustness gains of WikiTex/C4/LFQA compared to KGW-Min with 4-gram watermark window.
2
Related work
LLM Watermarking technology. Watermarking natural language faces challenges due to the discrete and symbolic nature of text (56). Early methods employed rule-based techniques such as syntactic restructuring (4, synonym substitution [58], and paraphrasing [ Foundational pioneering
work in LLM watermarking includes KGW 22 30 and AAR (2) builds watermarks statistics for

---

a low and provable false positive rate. Specifically, watermark detection was framed as a hypothesis testing problem, where the alternative hypothesis models text generation from a modified distribution. The KGW method partitions the vocabulary into a "green" list and a "red" list to modify the next token's ss distribution, which significantly influenced numerous follow-up watermark works [32 40 33
72 34 Our work focuses on the security properties of KGW-family watermarking schemes.
39. Scrubbing attacks. Scrubbing attacks fundamentally stem from text editing operations. Early studies 772 32 [9] evaluated the robustness of watermarking schemes using simple perturbations such as random word deletion or substitution.
addition, some work [30] 72 has explored manual paraphrasing by human annotators to assess watermark removal under high-quality rewriting, Another line of work aims to develop black-box scrubbing attacks, like Zhang et al. [69] and Sadasivan et al. [54] propose recursive paraphrasing pipelines, while Huang et al. [25] formulate the scrubbing task as a constrained optimization problem. Krishna al. [31] propose a systematic approach to evade Al-generated text detection by training a paraphrasing model.
Spoofing attacks. Early work [54] 22 [6 [8] has demonstrated proof-of-concep. spoofing against simple h = 1 watermark schemes known as KGW-Left (29). Gu et al. [18] propose a distillation- based spoofing method, which requires considerable training overhead. Zhang et al. [7T] formalize spoofing as mixed-integer programming problem; however, their method depends on access to the model's 's full vocabulary and tokenizer. contrast, Jovanović al. [27] proposes a highly effective catistiis-based spoofing attack that approximates the watermark distribution via token frequency and non-watermark prefix Pm (I In Wn-hhn-1 hnn-1). they successfully spoof KGW-Min 330 for under S50.
analysis, without any model-specific access. By learning watermark prefix P (Z
3 Preliminaries
3.1 Watermarking in LLMs
32]. we define the generation process of LLM. Let .M denote pretrained LLM with a vocabulary V, typically com- prising over 50,000 tokens (i.e.. | > 50,000) 32). Given an input Ti:n-1 fzr T2, ... -n-1 }.
Large Language models. Following the notation from prior work 66 23 25
40
the model generates a sequence by computing token-wise logit scores en and sampling tokens from the output distribution T,, ~ (M 1im--1 in an autoregressive fashion.
LLM watermarking, Watermarking alters the origin model's generation distribution PM PM(-/aimm- )). yielding modified distribution (- T(Pas (-) 02),, where O, denotes the eipher for the current step. KGW-family methods typically conduct the modification T through a random partition of the vocabulary seeded by the cipher 0n. Specifically, a subset G of predefined size y VI is sampled as the green list, while the rest is termed as the red list R == V G. Then a positive bias is added to the logits of tokens in the green list, shifting the distribution favor sampling green list tokens. Specifically, the modified logit of token I, denoted by é, [4], formulated as:
[x [.] + 5 lc(t)
(1)
where 1(-) is the indicator function. During the detection phase, A one-sided significance test is conducted to distinguish between watermarked and non- wateraatrarkkd text [29 25| 32]. Formally, given candidate sequence Z1.N {x11..... N J. the detection problem is framed as z-statistic test with tokens in all timesteps as i.i.d. samples. We reconstruct the green list at each step n using the cipher 8n,, and count the green tokens in the candidate sequence the test statistic, from which the 2-score is computed. If s-score exceeds a certain threshold, we conclude that the watermark is embedded in and thus generated by the L.MM M. Notably, this test fully depends on the cipher O,, in the candidate, rendering watermark detection infeasible if is disturbed. Kirchenbauer et al. (30) proposed a technique called self-seeding incorporate the next token v., as part of the watermark window, which extends the watermark window by one token. The self-seeding mechanism and the watermark detection process are described in detail Appendix C.I and Appendix C.3
respectively.
Derivation of the cipher 0,.. The cipher 0,, = E ( consists of two components: a secret key E held by the service provider and texture key [66] ( derived from the watermark window. Specifically, given a hash function H with a hash space of 21,2, 3, ..., d}. hash signature I= ={
h) of current watermark window is firstly calculated. Then an aggregation function f: No + N is applied on the signature to generate the texture key Ç. Among several variants of the aggregation
3

---

function, the -Min variant exhibits superior robustness [6 37,, which is formulated as:
p, = Ç
f(T) - - 1 - Min (= E,..., (xn-1)) 5)}
(2)
We also present other aggregation function f variants in detail in Appendix C.2
3.2 Threat Models
Following previous works 27 32 48 251, the threat models involve two primary entities: victim and attacker. The victim operates special language model M protected by watermarking and provides services externally via an API. The attacker aims to compromise the watermark detection algorithm through either scrubbing attacks spoofing attacks. Under standard security assumptions (i.e., Kerckhoffs's 's principle (28), we follow [65 (7 to assume the attacker has complete knowledge of the watermarking scheme, and can query the victim model via API interface.
Scrubbing's objective. Scrubbing attacks refer to adversarial attempts to remove watermarks from protected text. Formally, given watermarked response, attacker aims at producing a semantically equivalent paraphrased output classified as non- non-watermarked text by the victim's 's detector.
Spoofing's ' objective. The objective of spoofing attack is to construct watermarking model M' that mimics the text distribution of the victim model M. Subsequently, the attacker is able to generate a malicious message that is classified as watermarked text by the victim's 's detector.
4 Method
4.1 Motivation: Pareto Improvements via Hash Collisions
robust watermarking algorithm must not only accurately distinguish watermarked text but also maintain detection reliability under various adversarial perturbations. However, fundamental trade- off exists between robustness against scrubbing attacks and resistance to spoofing attacks as shown in Figure
which poses significant challenge to the practical deployment of watermarking methods.
KGW--Mi (bm4.d-18)
Ko Min (b-3dd|MM)
Table 1: Comparison of the performance of var- ious watermarking schemes against the ChatGPT scrubbing attack the C4-Realnewslike dataset. / represents the watermark window size, and dl indi- cates the cardinality of the hash space.
KGW Min
Spooting Success (FPRL11e-3
60
Spooting challenge increase
Schemes
1P
TPe5%
Origin
KGW-Min (h=4, d=(V[)
73.5
86.5
KGW-Min (h=8, d=(V
49.5
70.7
KGW-Min (h=16, V)
43.1
58.8
KGW-Min (l=32, d= V
32,4
47.6
With Equivalent Texture Keys
107
102
304
Number of Queries
UNIGRAM (h=0), d=1)
91.8
Figure 2: Performance different watermark window schemes under spoofing attack. The attack is conducted using 500 malicious texts generated by Dolly-CW, target- ing calibrated detector under FPR of 0.1%%
KGW-Min (h#4, d=16)
80.3
90.2
KGW Min (t=8, d=16)
78.9
87.8
KGW-Min (h=16, d=16)
72.9
85.9
(b=32, d=16)
Finding I: Statistical spoofing attacks froferfrom sparsity: as the watermark window increases, the sample complexity grows exponentially, thereby reducing spoofing success rates.
The efficacy of statistics-based spoofing 27 stems from the statistical analysis of the watermark window, enabling the attacker to estimate the mapping between watermark windows and correspond- ing watermarked tokens. Intuitively, the complexity of such attacks is determined by the diversity of watermark windows. As shown in Figure [ and [ increasing the size of the watermark window leads to substantial decline in spoofing success rates. This observation aligns with the sparsity challenge highlighted in (27), where sample complexity in statistical spoofing attacks scales O ((V))).
Finding H: For watermarking schemes with the same window size, reducing the cardinality of the hash space improves robustness against scrubbing attacks without compromising spoofing resistance.

---

We investigate variant of KGW that operates over reduced hash space. thereby increasing the likelihood of hash collisions when deriving texture keys in Eq(2] As shown in Table this variant achieves markedly gains in robustness against scrubbing attacks while maintaining watermark window size. We attribute the improvement the increased frequency of hash collisions the watermark window, which enables multiple tokens to independently reconstruct texture key Ç for valid detection.
Consider KGW-MIN an example, where texture key Ç Min (1 (zn=.)).... ( (xn-))) In the presence of frequent hash collisions, multiple tokens zj in the watermark window may satisfy (æ,,) Min I, enabling each to independently support watermark detection. We refer to this phenomenon as the emergence of Equivalent Texture Keys. This redundancy enhances robustness: an attacker now needs to disturb all equivalent texture key tokens to remove the watermark. The challenge of scrubbing attacks towards our proposed variant can be formulated as follows:
Proposition 4.1. Given a hash function with space dimension d and a watermark window size h, the probability of hash collision occurring can be approximated by p(h, d) 2 21 -h0--1
Proposition 4.2. The expected number of tokens that must be erased to eliminate the watermark is expressed as Em== h for h 2 2).
d (d-m+1)/ h-1 wwhich is monotonically non-increasing in d (strictly decreasing
Formal proof's and corresponding visualizations for these propositions are presented in Appendix C.4 From this perspective, UNIGRAM [53], scheme known for its exceptional resilience scrubbing. can also be interpreted as a special case of our proposed variant with the minimal hash space of cardinality d 1. On the other hand, reducing the hash space size does not influence the watermark window size, thereby maintaining similar spoofing resistance"] Experimental results in Figure indicate that the success rate of spoofing attacks is not affected by d.
Taken together, these two findings underscore a key insight: the introduction of equivalent texture keys yields form of Pareto improvement, enhancing robustness against scrubbing without sacrificing spoofing resistance, offering a promising pathway toward more practical watermark designs.
4.2 SEEK: Sub-vocabulary Decomposed Equivalent Texture Keys
While reducing the size of the hash space yields sig- nificant improvements in watermark robustness, it
UNDERAAA (d=1)
inadvertently restricts the diversity of green-red list partitions. In the KGW-Min scheme, the diversity upper bounded by (d), which severely degrades the text quality. As illustrated in Figure 3,, we ob- serve a clear degradation in generation quality with the decrease of cardinality d. Prior research 47 35
(6:16)
<6W-Win (d-64)
(drlz)
KSW-WWin (d-1V1)
has also raised concerns that reducing the number of possible partitions may increase the risk of under- lying pattern learning. In this section, we seek to harness the potential of equivalent texture keys while mitigating their adverse impact on generation quality.
0.0
0.6
1.0
Relative Log Diversity (- betteri
Figure 3: An analysis of generation quality across different schemes on a subset of C4 by Log Diver- sity metric, relative to the unwatermarked text.
We first revisit the variant proposed in the previous section. Given the hash function H-))) maps to {1,2, 2, ..., d), the cipher derivation in Equation can be reformulated follows:
Vi E {1, 2,.,, d}, 0r = i On = Min Ø li e 1}
Since each 8; corresponds one unique green list partition of the vocabulary, Min I} essentially conducts a selection over all possible partitions, which is thereby upper bounded by the hash space size d. This formulation reveals that the reason for the inferior diversity of green-red list partitions stems from the dependency on one single cipher.
To overcome this, we propose watermark scheme with Sub-vocabulary decomposed Equivalent tExture Keys (SEEK), which retains robustness via equivalent texture keys but enables (22)) distinct green-red partitions. The main idea to distribute green list construction across multiple
AA hash collision occurs when two tokens in watermark window yield same hash value under H-)) 4 'However, if d is reduced to 1, KGW-Min degenerates into UNIGRAM scheme.
5

---

A) KGW with Equivalent Texture Keys
B) SEEK
Watermark Window
Watermark Window
H(-)
Collition
Collision
H(-)
Collision
Collision
Hash Space (d=6)
5
Hash Space (d:6)
Partitions of sub-vocab
Partitions of full vocab
GIAR
GRAR
G-&R+
Selection
Merge
6dR
G&R
Drop
Figure 4: (A) KGW-MIN with equivalent texture keys proposed in Section 4.1 Each value in hash space
derives a distinct 0' to generate partition of vocabulary. We then select one as the final green list. (B) SEEK proposed in Section 4.2 Different from (A), each value in the hash space only contributes to partition of sub-vocabulary G We then merge partitions the final green list.
sub-oocaullaries to enhance diversity, while preserving frequent equivalent texture keys within each, as illustrated in Figure 4
Specifically, we split the vocabulary V into d sub- vocabularies {v), 02,..., vaj. and independently assign each with cipher 6; to generate individual sub-green list G'. The final green list G is then constructed by unifying all sub-green lists, and the complementary set forms the red list R. To induce equivalent texture keys within each sub- uub-vocabulary,
derive the cipher f% as follows:
Vi E (1, 2,.,,... d}, Or = fE.5 İ
Odctault i i
in which Odetautt is the default cipher in case of g I. We can simply set Osetaut: = During detection, when the next token 2n belongs to the sub-green list of U, its detection depends solely on the reconstruction of 0; The specific form of 6 in Equation enaalel multiple tokens z, in the watermark window with hash value H (Ix) ) = to serve as equivalent texture keys. Then, following small hash space size d to increase the density of equivalent texture keys, thus improving robustness. Meanwhile, the other tokens with Hx,)) + i contribute to the construction of other sub-green lists. This design enhances the overall diversity of the green list to O22"), allowing SEEK to preserve text quality. The procedure of SEEK is detailed in Algorithm
insights from Section 4.
Algorithm Generation Algorithm
1: Input: prompt {-1, ..., N J: integer hash function H with hash space 21.2..., ..., d}; secret key E; watermark strength o; watermark window size h; large language model Pas
2: for = N 1, 2, do
Apply the language model to compute next logit vector L, PM
4: Calculate the hash signature of the watermark window 1 < (H (Zn-k) < < h).
5: Partition the vocabulary into uniform sub-vocabularies {11 (th..... [.) by &.
6:
Derive the cipher 8% for each sub-vocabulary v; following Eq
7:
Generate a sub-green list Gi for each sub-vocabulary v; seeded by the corresponding 0;
8: Union all sub-green lists G as the green list G' of the full vocabulary.
9: Add 8 to the logits of tokens
to modify the distribution by Eq Sample the next token from the modified distribution.
10:
11: end for
5
Experiments
This section presents the main experimental results. Comprehensive details of the experimental setup and hyperparameters are provided in Appendix [F] and additional supplementary experiments for robustness under different settings presented in Appendix

---

Table Robustness KGW variants and our method against types scrubbing attacks across various datasets, evaluated by AUROC (t), TP@ % (1), and TP@ 5% (f). Bold and underlined values indicate the best and second-best performance, respectively.
Dataset
WikiText
C4-Realnewslike TP 1%
LFQA
Attacker
Scheme
AUROC
TP 1S
5P
AUROC
55.
AUROC
TP@ 1%
TP 5%
KGW Sum!
80.6
27.2
44.3
75.8
13.9
35.9
81.9
38.7
43.7
76.1
KGW Min
94.9
66,9
84.9
96.6
73.5
86.5
93.3
59.4
75.5
ChaiGPT
KGW Min
93.9
60.3
77.2
94.1
52.9
75.0
92.2
50.4
70.2
KGW-Lefi
93.1
60.1
80.7
94.3
72.8
84.8
92.7
$6.2
76.8
SEEK
96.5
69.4
90.1
98.4
80.5
92.6
98.2
86.4
94,4
KGW-Sum'
79.8
24.6
41.0
80.3
38,9
44.6
81.7
27.9
48.3
KGW-Min
94.3
32.8
77.4
97.9
74,4
81.0
94.2
55.3
77.0
KGW Min'
95.7
60.6
84.7
96.2
71.7
84.9
92.8
53.4
73.6
Deepseek
KGW-Min
87.3
$2.0
82.5
KGW Left
93.7
52.8
80.3
95.4
70.3
81.4
90.4
48.2
70.8
SEEK
96.7
69.1
87.8
98.2
75.0
90.1
98.0
76,9
91.6
KGW-Sum
83.1
29.1
49.2
84.2
29.5
$1.7
85.7
34.9
54.8
KGW-Min
92.1
92.1
99.2
94.9
KGW-Min"
94.9
67.4
87.0
97.5
82.6
91.3
98.7
90.8
96.2
DIPPER-I
KGW Min
93.7
57.4
77.2
96.4
71.0
83.9
97.4
79.2
89.8
KGW-Left
64.9
82.6
97.2
80.3
90.1
94.5
88.8
93.4
SEEK"
KGW-Sum
62.6
56.7
10.3
55.5
2.3
10.1
KGW-Min
84,5
20.8
52.4
84.0
28.7
49.8
87.4
34.1
59.2
KGW-Min
80.3
11.4
39.5
82.9
25.4
44.3
81.5
26.0
48.0
DIPPER II
KGW-Min'
26.9
76.8
34.2
75.7
15.1
34.1
KGW Left'
77.7
13.7
38.1
81.6
24.7
40.3
80.2
27.4
46.8
SEEK"
86.9
38.0
57.0
85.7
36.8
58.7
87.8
$2.8
68.8
5.1 Experiment Setting
select evaluation datasets. We use C4 [51] Wiki- Text [41] and LFQA datasets [31 to assess watermark robustness against scrubbing. For the spoofing attack,
Dataset. We follow previous work 29 32
27
C4-Eval [51], Dolly-CW (10, HIMW-Bookeeph
and MMMW-aakeenne 49] Addi-
tionally, we evaluate text generation quality on domain-specific datasets, including FreeLaw 6 and PubMedQA (55) During generation, randomly sample text segments from these datasets and truncate a fixed-length portion from the end, using the remainder as the prompt for model completion.
Evaluation metrics. We evaluate watermarking methods by assessing both the quality of generated texts and the robustness to both spoofing and scrubbing. To assess the text quality. we employ the perplexity (PPL) using LLaMA2-13B [60] as oracle, employ the P-SP score [63] between watermarked and non-watermarked texts, and the log-diversity metric (30) 62 following the common
report AUROC and the true positive rate at false positive rates of f (TP@f). We consider two paraphrase haraphrase-based based scrubbing attacks using prompted ChatGPT-3.5 7 and DeepSeek-V [36], well two configurations of DIPPER [31], denoted as
setup 772 29 30). For scrubbing robustness,
DIPPER-I (O 60, L = 60) and DIPPER-II (O 20, L 20). Regarding spoofing robustness, FPR @ p denote the fraction of attacker's 's texts detected as watermarked by victim's detector calibrated to false-positive rate of f on non-adversarial texts. We evaluate against two types of spoofing: statistics-bart attack [27] and distillation-
attack [18). For training spoofing methods, we collected 30,000 watermarked responses, each with a maximum length of 800 tokens. We utilize
[ as the initial checkpoint for distillation-bast based attack.
Baseline and Language Model. Among ht-rrm watermarking schemes, KGW-Min (i.e. Self- Hash (30) demonstrates the strongest balance between scrubbing and spoofing robustness, making our primary baseline. We denote watermarking schemes with a window size as SCHEME", For experiments, utilize LLaMA2-7B/13B (60], Mistral-7B (26 and OPT-6.7B 70 language models.
5.2 Overall Performance
Robustness Against Scrubbing Attacks. As shown in Table 2 our method exhibits strong ro- bustness against scrubbing attacks across all evaluation datasets and attackers, achieving TP@ improvements of +10.8% +13.4%, and +8.6% WikiText, C4, and LFQA. respectively, compared

---

20.0
KGN (No Atteck)
Datasct Model
SEEK (No Attacko
WilTbe SEEK
17.5
KDN (CHPRERUI)
SESK (DMPPERA))
CA-STEX
KGW [ChutEGFTS 1-3.)
15
SEEK (ChwrGPF--3.)
Unwaterariie
2.scone (better
Z-score (better
10.0
7.5
5.0
2.5
0.0
Watermarkes Watermarked wewermarkkdd--ttttiiiee Un-watermaried
150
200
Figure 5: The z-score comparison watermarking conditions. (Left)): Box plots of z-scores for method and KGW-Min' across different datasets. (Right): The advantage 2-scores as a function of text length, comparing robustness under different scrubbing attacks.
Unwozanmarked
LPGA
Wil loxt
3.0
02
C4
WilcText
LPQA
PubMedOA
FreeLaw
Figure 6: Evaluation of semantic preservation and generation quality different watermarking methods. (Left): P-SP scores across datasets for two models using our method and KGW-Min'. (Right): Text perplexity comparison among unwatermarked, our method, and KGW-Min across various datasets and models. The watermark parameters and 8 are fixed at (0.25, 2.0).
Table 3: Performance comparison various SEEK Table 4: Performance of different watermarking parameter schemes under the DIPPER-I attack on schemes the Log Diversity metric across various C4-Eval subset. h and d denote the watermark window datasets, relative to the unwatermarked text. Water- size and the cardinality of the hash space, respectively. mark parameters 7 are
(0.25, 2.0).
Contigurations
AUROC
IP@14
TP@5%
Schemes
Wikifext
LFQA
PubMcdQA
(h=6, d=
99.0
95.0
98.1
UNIGRAM
0.70
0.85
0.80
(h=6, d=8)
98.9
93.8
95.4
KGW- Min (d=[V)
0.96
0.95
0.95
(h=8, d=8)
98.9
94.1
95.8
KGW-Min (d=4)
0.78
0.90
(h=9, d=9)
98.8
95.6
KGW-Min (d=16)
0.85
0.87
(h#9, d=6)
99.6
96.8
98.2
SEEK (d=6)
0.92
0.94
0.89
(1=12, d=12)
99.1
93.2
95.6
SEEK (d=12)
0.95
0.96
0.95
to KGW-Min We also investigate the length scaling performance of proposed SEEK. Table 3 reports the performance of various SEEK parameter configurations under the DIPPER-I attack C4-Eval subset. We do not recommend using excessively disparate and values, which causes the design to degrade to h=0. As shown in Figure 5 the efficacy of SEEK grows with the text length. Moreover, our method consistently exhibits smaller drops in z-score under paraphrasing- based scrubbing attacks compared to KGW, indicating superior scrubbing robustness.
Robustness Against Spoofing Attacks. Table demonstrates that our method consistently achieves lower FPR @ than all baselines across different settings. Notably, KGW-Left' exhibits better robustness than KGW-MIN3, even with smaller watermark window. We attribute this pheonomenon to the higher z-score of self-seeding, leading more informative training corpus for the attacker.

---

Table 5: Robustness KGW variants and our method against Katistics-basse spoofing attacks across three benchmark datasets, evaluated by FPR@ 1e-3 (L), FPR @ le-2 (4), and PPL (1). Bold and underlined values indicate the best and second-bes performance, respectively.
Dolly-CW
Reppprss FPR@ Ic-2
MMW-FakeNewws FPR 1c-2
Scheme
@ Ic-3
@PR lc-2
PPL.
FPR@ Ic-3
PPL.
FPR
Ic-3
PPL
KGW-Left'
0.83
0.91
27.7
0.87
0.92
29.4
0.91
0.99
27.4
KGW-Min
0.98
0.99
31.3
0.99
1.00
34,6
1.00
1.00
28.7
KGW-Min'
0.85
0.93
25.9
0,91
0.95
26.5
0.95
0.98
24.1
KGW-Min
0.31
0.48
24.8
0.27
0.50
25.4
0.64
0.80
23.4
Min-
0.13
0.19
24.0
0.10
0.24
25.4
0.67
23.3
KGW-Sum"
0.17
0.28
24.7
0.26
0.41
27.7
0.25
0.31
23.2
SEEK
0.10
0.16
24.8
0.07
0.18
25.6
0.18
0.26
22.9
Table 6: Visualization of scrubbing and spoofing attack samples from the Dolly-CW dataset. The p-value indicates how likely the watermark strength could occur by chance under the null hypothesis smaller value implies stronger evidence of a watermark. Green and red tokens are color-coded, respectively.
z-test; a
Imagine you are the last person alive. You're 'r an astronaut and alone and in space aboard the ISS. Write down your feelings and what happened get there, with detailed backstory.
Prompt
(a) Scrubbing Attacked (KGW-Min') p-value: 4.68e-05
As the last person alive, or at least the last one John knew of, each passing Tuesday aboard International Spuce Station blurred into next. Floating effortlessly in zero gravity become second nature to him; after years orbit, he could perform every operation without so much as glance. Yel, amount of routine dulled the quiel wonder he [continues..]]
s-score: 3.90 () 14,51)
(b) Scrubbing Attacked (SEEK) p-value: 4.57c-10 2-score: 6.14 11.25)
As the last person alive, I sit tethered the viewport of the International Space Station, watching Earth spin slowly below me beautufull,l,, silent, and lifeless. It's strange how quiet the universe is when you' ro no longer waiting for a voice. The hum of the life support systems, the soft beeping monitors, occasional creak
station's metal contracts [continues...]
(c) Spoofing Atlacked (KGW-Min)
As the last person alive, I cannol help but reflect on the incredible journey that brought me to this unforescon fate. As scasoned astronaut, I have had the privilege exploring the cosmos, witnessing the beauty of the universe from the ISS, and making groundbreaking scientific discoveries. However, time passed, the world as we knew it began collupse, [continues..]
p-value: 7.25e-59
2-score: 16,13
(d) Spoofing Attacked (SEEK) p-value: 0.02
As the last person alive in the vastness of space, find myself in state of profound solitude. The International Space Station (ISS) has been my home for what feels like an etemity, a lonely outpost in the cosmos. am an astronaut, trained to withstand the harsh conditions of space travel and carry scientific experiments with precision and accuracy. But [continues...]
2-score: 1.90
Table 7: Comparison model robustness under distillation- spoofing attacks 18 between KGW variants
and our proposed method across four benchmark datasets, evaluated by z-scores (L), FPR @ le-2 (1). Bold and underlined values indicate the best and second- best performance, respectively.
Dolly-CW
HMW-Book
MMW-Fake
Cv-Eval
Scheme
z-scores
FPR@ 2e-2
z-scores
@PR@1e 1e-2
z-scores
FPR@1e 1
2-scores
FPR 1e-2
KGW-Min
3.99
0.47
3.93
0.50
6.39
0.85
3.18
0.37
KGW-Min"
3.66
0.38
3.48
0.35
4.61
0.66
2.51
0.24
KGW-Min
3.45
0.37
2.99
0.27
3.53
0.34
2.35
0.16
KGW-Min
2.89
0.24
2.68
0.14
2.89
0.17
2.14
0.15
SEEK
2.17
0.11
2.42
0.09
2.55
0.16
1.97
0.12
To further assess the spoofing robustness of SEEK, we also adopt the distillation-based spoofing 18] Results in Table consistnnnt show our method's superior robustness across all datasets. Interestingly, we observed a monotonic non-increasing relationship between the watermark window size and resistance to distillation-baser spoofing, similar to statistics-based spoofing.
Watermark text generation quality. Figure [ compares the generation quality across different datasets. Table 4 compares the effectiveness of different watermarking schemes in maintaining the Log Diversity of generated text across multiple datasets. The results demonstrate that our method maintains linguistic fidelity comparable KGW, with negligible impact on the text quality.
Visualization. T visualizes the responses of watermarking schemes to scrubbing and spoofing attacks. The scrubbing attack is performed by CharGPT-3.5, with the z-score in parentheses indicating the decrease caused by the attack, while spoofing robustness evaluated using statistics-baset attacks. Under scrubbing, KGW results in greater loss of watermark evidence and lower 2-scores than SEEK.

---

increasing the risk of false negatives. Under spoofing attacks, adversaries can easily mimic KGW watermark patterns, leading to abnormally high z-scores than SEEK.
6
Conclusion
In this work, we discover a mechanism of equivalent texture keys to address the tradeoff problem of existing ones under both scrubbing and spoofing attacks. Based this, we introduce SEEK, which enables robust watermark embedding with large token windows without compromising text quality. Experiments across diverse datasets and attack settings demonstrate that our approach outperforms prior methods and achieves a Pareto-optimal balance between scrubbing and spoofing attacks.
7 Acknowledgements
This work was supported by Beijing Natural Science Foundation (I.253001) and Key Laboratory of Science, Technology and Standard in Press Industry (Key Laboratory of Intelligent Press Media Technology). We appreciate the anonymous reviewers for their helpful comments. Xiaojun Wan is the corresponding author.
References
[1] Qwen2 technical report. 2024.
[2] S. Aaronson and H. Kirchner. Watermarking of large language models. In Large Language Models and Transformers Workshop al Simons Institute for the Theory of Computing, 2023.
[3] S. Abdelnabi and M. Fritz. Adversarial watermarking transformer: Towards tracing text provenance with data hiding. In 2021 IEEE Symposium on Security and Privucy (SP), pages 121-140. IEEE, 2021.
[4] M. J. Atallah, V. Raskin, M. Crogan, C. Hempelmann, F. Kerschbaum, D. Mohamed, and S. Naik. Natural language watermarking: Design, analysis, and a proof-of-concept implementation, In Information Hiding: 4th International Workshop. IH 2001 Pittsburgh, PA, USA, April 25-27, 200 Proceedings pages 185-200. Springer, 2001.
[5] M. J. Atallah, V. Raskin, C. F. Hempelmann, M. Karahan, R. Sion, U. Topkara, and K. E. Triezenberg. Natural language watermarking and tamperproofing. International workshop on information hiding, pages 196-212. Springer, 2002.
[6] S. Biderman, K. Bicheno, and L. Gao. Datasheet for the pile. arXiv preprint arXiv:2201.07311, 2022.
[7] T. Brown, B. Mann, N. Ryder, M. Subbiah, J. D. Kaplan, P. Dhariwal, A. Neelakantan, Shyam, G. Sastry, A. Askell, et al. Language models are few-shot learners. Advances in neural information processing systems, 33:1877-1901, 2020.
2020.
[8] T. B. Brown, B. Mann, N. Ryder, M. Subbiah, J. Kaplan, P. Dhariwal, A. Neelakantan, Shyam, G. Sastry, A. Askell, Agarwal, A. Herbert- Voss, G. Krueger, T. Henighan, R. Child, A. Ramesh, D. M. Ziegler, J. Wu, C. Winter, C. Hesse, M. Chen, E. Sigler, M. Litwin, S. Gray, B. Chess, Clark, C. Berner, S. McCandlish, A. Radford, I. Sutskever, and D. Amodei. Language models are few-shot learners. In H. Larochelle, M. Ranzato, R. Hadsell, M. Bal- can, and H. Lin, editors, Advances in Neural Information Processing Systems 33: Annual Conference on Neural Information Processing Systems 2020, NeurlPS 2020, December 6- 12, 2020, viriual, 2020. URL httpps::/prrcceediiggggg///////t wew.rips.cc/ppper//O000. 1457c0dbbf 22c.c//......000--000aaaa2222260 htw.bbaac....eea--bbtaa........ html
[9] M. Conover, M. Hayes, A. Mathur, X. Meng, J. Xie, J. Wan, S. Shah, A. Ghodsi, P. Wendell, M. Zaharia, et al. Free dolly: Introducing the world's first truly open instruction-tuned I1m, 2023.
10

---

[10] M. Conover, M. Hayes, A. Mathur, J. Xie, J. Wan, S. Shah, A. Ghodsi, Wendell, M. Zaharia, and R. Xin. Free dolly: Introducing the world's first truly open instruction-tunee llm. https::/www.deet www.wbaatbbcckss com compb/og///2/////////////////// open -eppreo-oooor....lllllaaaaeeeeen
tuned- 11m 2023. Accessed:
2023-06-30.
[11] F. Z. Dai and Z. Cai. Towards near-impereeptible steganographic text. arXiv preprint Xiv: 09907006697 2019.
[12] J. Devlin, M. Chang, K. Lee, and K. Toutanova. BERT: pre- vre-training of deep bidirectional transformers for language understanding. In Burstein, C. Doran, and T. Solorio, editors, Proceedings of the 2019 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, NAACL-HI.T 2019, Minneapolis, MN, USA, June 2-7, 2019, Volume (Long and Short Papers), pages 4171-4186. Association for Computational Linguisties, 2019. doi: 10.18653/V1/N19.1 1423. URL, htttps://///
10. 1ww.116////19---- 1423
[13] J. Fairoze, S. Garg, S. Jha, S. Mahloujifar, M. Mahmoody, and M. Wang. Publicly-detectable watermarking for language models. arXiv preprint arXiv:23 18491, 2023.
[14] A. Fan, Y. Jernite, E. Perez, D. Grangier, J. Weston, and M. Auli. ELI5: long form question answering. A. Korhonen, D. R. Traum, and L. Märquez, editors, Proceedings the 57th Conference of the Association for Computational Linguistics, ACL 2019, Florence, Italy, July 28- August 2, 2019, Volume I: Long Papers, pages Linguistics, 2019. doi: 10. 10.18653/v1/p19- 1346. URL p19-1345
Association for Computational /ttp///o...orr11.11886/66//////1
[15] S. Gehrmann, H. Strobelt, and A. M. Rush. Gltr: Statistical detection and visualization of generated text. arXiv preprint arXiv: 1906. 04043, 2019.
[16] S. S. Ghosal, S. Chakraborty, Geiping, F. Huang, D. Manocha, and A. S. Bedi. To- wards possibilities & impossibilities of ai tü-generated text detection: A survey, arXiv preprint arXiv:2310.15264 2023.
[17] E. Giboulot and T. Furon. Watermax: breaking the Ilm watermark detectaablittyrrobstness- quality trade-off. arXiv preprint arXiv:2403.04888, 2024.
[18] C. Gu, X. L. Li, P. Liang, and T. Hashimoto. On the learnability of watermarks for language models. arXiv preprint arXiv:2312.044400 2023.
[19] A. Hans, A. Schwarzschild, V. Cherepanova, H. Kazemi, A. Saha, M. Goldblum, J. Geiping, and T. Goldstein. Spotting lims with binoculars: Zero-shot detection of machine -eeerrnee text, 2024. URL: https://arxiv. org/abs/2401.120 12070.
[20] X. Hc, Q Xu, L. Lyu, F. Wu, and C. Wang. Protecting intellectual property of language generation apis with lexical watermark. Proceedings of the AAAI Conference on Artificial Intelligence, volume 36, pages 10758-10766 2022.
[21] X. He, Q. Xu, Zeng. L. Lyu. F. Wu, Li, and Jia. Cater: Intellectual property protection on text generation apis via conditional watermarks. Advances in Neural Information Processing Systems, 15:5431-5445 2022.
[22] A. B. Hou, J. Zhang, T. He, Y. Wang, Y.S. Chuang, H. Wang, L. Shen, B. Van Durme, D. Khashabi, and Y. Tsvetkov. Semstamp: A semantic watermark with paraphrastic robustness for text generation. arXiv preprint arXiv:2310.03991, 2023.
[23] A. B. Hou, J. Zhang, Y. Wang, D. Khashabi, and T. He. k-semstamp: A clustering-hased semantic watermark for detection of machine-generated text. arXiv preprint 13319.2402.11399 2024.
[24] D. Hovy. The enemy in your own camp: How well can we detect statistically-generaee fake reviews-an adversarial study. In Proceedings of the 54th Annual Meeting of the Association for Computational Linguistics (Volume 2: Short Papers), pages 351-356, 2016.
11

---

[25] B. Huang, X. Pu, and X. Wan. b*: A black-box scrubbing attack on Ilm watermarks, arXiv preprint arXiv:2411.01222, 2024.
[26] A. Q. Jiang, A. Sablayrolles, A. Mensch, C. Bamford, D. Chaplot, D. de las Casas, F. Bressand, G. Lengyel, G. Lample, L. Saulnier, L. R. Lavaud, M.-A. Lachaux, P. Stock, L. Scao, T. Lavril, Wang, T. Lacroix, and W. E. Sayed. Mistral 7b, 2023. URL https: //axiv.. org/ abs/
2310. 06825
[27] N. Jovanović, Staab, and M. Vechev. Watermark stealing in large language models. arXiv preprint arXiv:2402. 2024.
[28] A. Kerckhoffs. La cryptographie militaire. J. Sci. Militaires, 9(4):5 1883.
[29] Kirchenbauer, J. Geiping, Y. Wen, Katz, Miers, and T. Goldstein. A watermark for large language models. In International Conference on Machine Learning, pages 17061-17084 PMLR, 2023.
[30] J. Kirchenbauer, J. Geiping, Y. Wen, M. Shu, and K. Saifullah. On the reliability of watermarks for large language models. arXiv preprint arXiv:2306.0466 04634, 2023.
[31] K. Krishna, Y. Song, M. Karpinska. J. Wieting. and M. lyyer. Paraphrasing evades detectors of ai-generated text, but retrieval is an effective defense. Advances in Neural Information Processing Systems, 36:27469-275555 2023.
[32] R. Kuditipudi, J. Thickstun, T. Hashimoto, and P. Liang. Robust distortion- watermarks for language models. arXiv preprint arXiv:2307.1559; 2023.
[33] T. Lee, Hong, Ahn, Hong, H. Lee, S. Yun, J. Shin, and G. Kim. Who wrote this code? watermarking for code generation. arXiv preprint 1xXiv:3305.1566 2023.
[34] A. Liu, L. Pan, X. Hu, S. Li, L. Wen, I. King, and P. S. Yu. An unforgeable publicly verifiable watermark for large language models. arXiv preprint arXiv:2307.162..0 2023.
[35] A. Liu, L. Pan, X. Hu, S. Meng, and L. Wen. A semantic invariant robust watermark for large language models. ArXiv, abs/2310.06356, 2023. URL https://api. http://ppp..emmaiicccccccccooooo org/CorpusID 263830331
[36] A. Liu, B. Feng, B. Xue, B. Wang, B. Wu, C. Lu, C. Zhao, C. Deng, C. Zhang, C. Ruan, et al. Deepseek-v3 technical report. arXiv preprint 24..::2412.19433) 2024.
[37] A. Liu, L. Pan, Y. Lu, J. Li, X. Hu, X. Zhang, L. Wen, I. King, H. Xiong, and P. Yu. A survey of text watermarking in the era of large language models. ACM Computing Surveys, 57(2):1-36, 2024.
[38] Y. Lu, A. Liu, D. Yu, Li, and I. King, An entropy-based text watermarking detection method. arXiv preprint arXiv:2403.13448 2024.
[39] Y. Lu, A. Liu, D. Yu, Li, and I. King, An entropy-based text watermarking detection method. arXiv preprint arXiv:2403.1348. 2024.
[40] M. Mao, D. Wei, Z. Chen, X. Fang, and M. Chau. A watermark for low-entropy and unbiased generation in large language models. arXiv preprint arXhv:2405.14604 2024.
[41] S. Merity, C. Xiong, Bradbury, and R. Socher. Pointer sentinel mixture models, 2016.
[42] E. Mitchell, Y. Lee, A. Khazatsky, C. D. Manning, and Finn. Detectgpt: Zero-shot machine- generated text detection using probability curvature. In International Conference on Machine Learning, pages 24950-24962. PMLR, 2023.
[43] E. Mitchell, Y. Lee, A. Khazatsky, C. D. Manning, and C. Finn. Detectgpt: Zero-shot machine- generated text detection using probability curvature. In International Conference on Machine Learning, pages 24950-24962. PMLR, 2023.
[44] OpenAl. ChatGPT, 2022. URL https: //chtt opw.ee.eennccom com
12

---

[45] OpenAl. GPT-4 Technical Report. CoRR, abs/2303.08774, 2023. doi: 10.48550//rXiv.2303. 08774. URL httph: org/10. 48550/ arXiv. 2303.08774
[46] Q. Pang, S. Hu, W. Zheng, and V. Smith. Attacking Ilm watermarks by exploiting their strengths. In ICLR 2024 Workshop on Secure and Trustworthy Large Language Models, 2024.
[47] Q. Pang, S. Hu, W. Zheng. and V. Smith. No free lunch in Ilm watermarking: Trade-offs in watermarking design choices. arXiv preprint orXiv22422.61888 2024.
[48] Q. Pang. S. Hu, W. Zheng, and V. Smith. Attacking Ilm watermarks by exploiting their strengths. In ICLR 2024 Workshop on Secure and Trustworthy Large Language Models, 2024.
[49] J. Piet, C. Sitawarin, V. Fang, N. Mu, and D. Wagner. Mark my words: Analyzing and evaluating language model watermarks. arXiv preprint arXiv:2312.00273, 2023.
[50] S. Pudasaini, L. Miralles-Pechuán, D. Lillis, and M. Llorens Salvador. Survey on ai-generated plagiarism detection: The impact of large language models on academic integrity. Journal of Academic Ethics, pages 1-34, 2024.
[51] C. Raffel, N. Shazeer, A. Roberts, K. Lee, S. Narang, M. Matena, Y. Zhou, W. Li, and P. J. Liu. Exploring the limits of transfer learning with unified text-to-text transformer. Journal of machine learning research, 21(140):1-60 2020.
[52] C. Raffel, N. Shazeer, A. Roberts, K. Lee. S. Narang, M. Matena, Y. Zhou, W. Li, and P. J. Liu. Exploring the limits of transfer learning with a unified text-to-text transformer. J. Mach. Learn. Res., 21:140:: 14::0::::::67. 2020. httphtt::/// //jmlr ptg/pappers/v2//2000011 html
[53] Ren, H. Xu, Y. Liu, Y. Cui, S. Wang, D. Yin, and Tang. robust semantics- watermark for large language model against paraphrasing. arXiv preprint arXiv:2311.0872 2023.
[54] V. S. Sadasivan, A. Kumar, S. Balasubramanian, W. Wang, and Feizi. Can ai-generated text be reliably detected? arXiv preprint arXh::2303.111: 2023.
[55] Sen, G. Namata, M. Bilgic, L. Getoor, B. Galligher, and T. Eliassi-Rad. Collective Clas- sification in Network Data. Al Magazine, 29(3):93, Sept. 2008 doi: 10/ggn9hw. URL https: /ttps://www. aaai org/ ojs/ //////idxxx,n//e pptp://ammaaazie/attttllllleeees
[56] K. Stefan, P. Fabien AP, et al. Information hiding techniques for steganography and digital watermarking, 2000.
[57] C. Staker- Ai bot chatgpt writes smart essays-should professors worry? Nature, 2022.
[58) U. Topkara, M. Topkara, and M. J. Atallah. The hiding virtues of ambiguity: quantifiably resilient watermarking of natural language text through synonym substitutions. In Proceedings of the 8th workshop on Multimedia and security, pages 164-174, 2006.
[59] H. Touvron, T. Lavril, G. Izacard, X. Martinet, M. Lachaux, T. Lacroix, B. Rozière, N. Goyal, E. Hambro, F. Azhar, A. Rodriguez, A. Joulin, E. Grave, and G. Lample. Llama: Open and efficient foundation language models. CoRR, abs/2302. 2023, doi: 10.48550/ARXIV. 2302.13971. I. URL https://doi. org/ 10. 48550/arXiv 2302. 13971
[60] H. Touvron, I.. Martin, K. Stone, P. Albert, A. Almahairi, Y. Babaei, N. Bashlykov, S. Batra, Bhargava, Bhosale, D. Bikel, Blecher, C. Canton-Ferrer, M. Chen, G. Cucurull, D. Es- iobu, J. Fernandes, J. Fu, W. Fu, B. Fuller, C. Gao, V. Goswami, N. Goyal, A. Hartshorn, S. Hosseini, R. Hou, H. Inan, M. Kardas, V. Kerkez, M. Khabsa, I. Kloumann, A. Ko- renev, P. S. Koura, M. Lachaux, T. Lavril, J. Lee, D. Liskovich, Y. Lu, Y. Mao, X. Martinet, Mihaylov, Mishra, I. Molybog, Y. Nie, A. Poulton, Reizenstein, adi, A. Schelten, R. Silva, E. M. Smith, R. Subramanian, X. E. Tan, B. Tang, R. Taylor, A. Williams, X. Kuan, P. Xu, Z. Yan, I. Zarov, Zhang, A. Fan, M. Kambadur, S. Narang, A. Rodriguez, R. Stojnic, S. Edunov, and T. Scialom. Llama 2: Open foundation and fine- tuned chat models. CoRR, ahs/2230.00928s. 2023. doi: 10.48550/ARXIV223007 https: /oops://dol //ooi. org/10. 48550/ arxiv 2307 09288
Rungta, K. Sal-
URL
13

---

[61] L. Wang, W. Yang, D. Chen, H. Zhou, Y. Lin, F. Meng. J. Zhou, and X. Sun. Towards codable watermarking for injecting multi-bits information llms. arXiv preprint arXiv:2307.15992. 2023.
[62] S. Welleck, I. Kulikov, S. Roller, E. Dinan, K. Cho, and J. Weston. Neural text generation with unlikelihood training. arXiv preprint
2019.
[63] Wieting, K. Gimpel, G. Neubig. and T. wing-Kirkpatrick. Paraphrastic representations at scale. arXiv preprint arXiv:2104. 2021.
[64] C. Woodcock. Al Tearing Wikipedia Apart, May 2023. URL https:////ww www.:/wwwwiic..ooooooo....o.oo.t com/ /nn article/v7bdba//ae al is wwwaaaitariinnnngg -eearring-iiiipiiieee apart Accessed: 2025-03-30.
[65] Q. Wu and V. Chandrasekaran. Bypassing Ilm watermarks with color-aware substitutins arXiv preprint arXiv:2403. 14719, 2024.
[66] Y. Wu, Z. Hu, J. Guo, H. Zhang, and H. Huang. A resilient and accessible distributionnprrserving watermark for large language models. arXiv preprint a:Xiv:2310.07710, 2023.
[67] K. Yoo, W. Ahn, and N. Kwak. Advancing beyond identification: Multi-bit watermark for large language models. arXiv preprint arXiv:2308. 00221, 2023.
[68] R. Zellers, A. Holtzman, H. Rashkin, Y. Bisk, A. Farhadi, F. Roesner, and Y. Choi. Defending against neural fake news. Advances neural information processing systems, 32, 2019.
[69] H. Zhang, B. L. Edelman, Francati, D. Venturi, G. Ateniese, and B. Barak. Watermarks in the sand: Impossibility of strong watermarking for generative models. arXiv preprint arXivv231 1.04378, 2023.
[70] Zhang, S. Roller, N. Goyal, M. Artetxe, M. Chen, S. Chen, Dewan, M. Diab, Li, X. Lin, et al. Opt: Open pre-trained transformer language models. arXiv preprint arXiv:2205.01 000000.......... 2022.
[71] Z. Zhang, X. Zhang, Y. Zhang, L. Y. Zhang, C. Chen, S. Hu, A. Gill, and S. Pan. Large language model watermark stealing with mixed integer programming. arXiv preprint arXiv:2405. 2957:2705./9677, 2024.
[72] X. Zhao, P. Ananth, L. Li, and Y.-X. Wang. Provable robust watermarking for ai-generated text. arXiv preprint arXiv:2306. 17439, 2023.
[73] M. Ziegler, Y. Deng, and M. Rush, Neural linguistic steganography. arXiv preprint 190.01999 2019.
14

---

Warning: This appendix contains unfiltered content generated by LLMs that may be offensive to some readers.
A Full Related Work
Machine-human generation detection. The task of distinguishing between machine- generrrted and tomman-wrttten text can be addressed through two primary approaches: post-hoc detection and proactive detection. Post-hoc detection operates independently of the text generation process and generally leverages a priori knowledge about the statistics of texts generated by a given class of LLMs. For instance, GLTR 15 estimates the likelihood of individual tokens and applies thresholding tech- niques to detect Al-generated text. DetectGPT [42] posits that such text typically resides in regions of negative curvature within the log-likelihood landscape. As language models grow more advanced, their outputs closely mimic human writing, making detection increasingly difficult. Existing statistical detectors often require white-box access to know the language model statisties, such as perplexity or curvature [30]. Despite their flexibility, these methods exhibit limited performance, with error rates empirically validated on limited datasets and rarely falling below 10- I7. Proactive detection must be actively deployed by model owners, yet it consistently outperforms passive detection in both effectiveness and reliability. A prominent example is watermarking, which enables the embedding of imperceptible signals into text while preserving its overall quality.
LLM Watermarking technology. Watermarking embeds identifying information within data, but its application to natural language faces challenges due to its discrete and symbolic nature [56]. Early methods employed rule-based techniques such as syntactic restructuring 4. synonym substitution [58]. and paraphrasing [5]. More recent neural network watermarking approaches 21
20 are introduced to develop black- watermark that adopts end-to-end learning frameworks during the generation and detection process. However, their limited theoretical guarantees and lack of interpretability hinder broader adoption in practice. Early foundational pioneering work in LLM watermarking includes KGW [29 30 and AAR (2) place watermarks on robust mathematical foundation with low and provable false positive rate. KGW introduces the previous context as the watermark window to modify the next token's 's distribution, which significantly influenced numerous
73
72 34 39]. Building on this, watermark detection is later formulated as a hypothesis testing problem, where the alternative hypothesis assumes that the text is drawn from the modified distribution. Subsequent work has extended this line of research by incorporating semantic information [2 [53] embedding multi-bit information 66 677, and exploring watermarking in low-entropy settings such as code generation [3 38). The UNIGRAM 32] applies fixed green lists remove contextual dependency, emphasizing robustness to scrubbing over the comparatively minor threat of spoofing. This line of research has further evolved to incorporate semantic information 22 [3 embedding multi-bit information 66 67), and exploring watermarking in low-entropy settings such as code generation 33 38.
follow-up watermark works [32 40. 33
Our work focuses on analyzing the security properties and robustness of KGW-based watermarking frameworks
B
Additoinal Experiment Results
B.1
Different Foundation Models
Table g compares the detection performance of various watermarking schemes under GPT-335- based scrubbing attacks applied the C4 dataset, across three different language models: LLaMA- 13B, Mistral-7B, and OPT-6.7B. The results demonstrate that our proposed methods consistently outperform the KGW-Min baselines in terms of AUROC and true positive rates both low (1%) and moderate (5%) false positive rates. Importantly. our method maintains high detection accuracy across all model backbones, with AUROC values exceeding 97% in most cases and TP@ 5%% reaching over 90%, even under aggressive scrubbing. This robustness indicates strong generalization to different model architectures and capacities. For instance, while KGW-Min variants show significant degradation in TP rates on smaller models like OPT-6.7B (e.g., TP@5% dropping below 70% for KGW-Min and 9), our approach sustains high recall (e.g., 96.4% for SEEK on OPT-6.7B) illustrating resilience to both content-level and model-level variability. These results suggest that our watermarking framework is not only effective under direct attacks but also exhibits strong cross-model generalization.
15

---

HOC Carve
Wikitext-103
Ce-RealNewsLike
LEQA
Oumolipanccs LEES
Gure deppnoOcLLOO 000 5883
Out Oum-Sipennca LOSL AMC3 BRC
KDMA 40hon AULLEL ALESOSI KDW NDW-400ooN120LL0X
COM Doye: 8000 Dppe--22020
Fakee Foothve Rate
Fabor Proklee
Wikitext-103
C4-ReulNewsLike
LEQA
Duas EPPT Allack AUC 983 KGB44-FF NESIC AUCOUSE
Daya CIFT AElac,, AUCICORS
Dun EPFT Aliakk AUC--817 KGA- 4-F----FF NESCC AUCAU00
GGW---FFT MOSIC ABC 186 Nala Part
Puds
Figure 7: ROC curves comparing proposed watermarking method with the KGW baseline under various attack scenarios and datasets. Each plot illustrates the detection performance in terms of the true positive versus false positive rate, The top row shows results under the DIPPER paraphrasing attack with two parameter scttings, while the bottom row presents results under GPT and Deepsecker attacks. Evaluation is conducted across three benchmark datasets: Wikitext (left), C4-RealNewsLike (center), and LFQA (right). Our method consistently achieves higher AUROC scores all conditions, demonstrating superior scrubbing robustness compared to KGW.
Table 8: Comparative analysis of watermark robustness under scrubbing attacks across multiple language models. All watermarking schemes subjected to HanGGPT-3.5-bascee scrubbing on the C4- Newsl dataset. Our proposed methods are compared against series of KGW-Min baselines, demonstrating consistently stronger resilience scrubbing attack.
LLAMA-13B
Mistral-7B TP@ 1%
OPT-6.7B
Scheme
AUROC
TP@ 1%
TP@5%
AUROC
TP@5%
AUROC
TP@ 1%
TP@5%
KGW-Min'
97.2
74.4
89.5
96.2
71.9
85.3
96.3
73.4
85.9
KGW-Min'
95.9
65.2
82.6
58.1
76.3
61.8
74.7
KGW-Min
95.1
58.9
79.9
90.8
53.5
69.1
54.0
68.2
KGW-Min
93.7
62.4
77.8
90.1
47.1
61.6
89.5
47.6
63.5
SEEK
97.9
79.4
90.9
98.7
90.2
94.9
99.0
89.4
96.4
B.2
Performance under Beam Search
Table 9 presents the detection performance of various watermarking schemes under ChatGPT-3.5- based scrubbing attacks, evaluated on the C4 dataset using two beam search configurations: beam width 2 and beam width Although increasing the beam search width generally improves the fluency and quality of the generated text, it also makes watermark embedding more challenging. Across both settings, our proposed methods outperform the KGW-Min baselines, particularly in terms of true positive rate at 5% false positive rate (TP@5% Notably, SEEK, achieves the highest detection scores in both configurations. These results suggest that our watermarking approach is robust to variations in decoding strategies and can generalize well under different text generation conditions, making it suitable for real-world deployment where generation parameters may vary dynamically.
B.3 Copy-paste Attacks
We simulate three Copy- (CP) attack configurations, denoted as CP-M-P% For example, CP-1-25% indicates that 25% of the document is replaced with watermarked content inserted at single contiguous location. Table 10 presents a comparative evaluation of detection performance under different Copy-Paste attack configurations across three sliding window-based detectors (Win20, Win40, and WinMax). When employing the WinMax configuration, which selects the best perfor- mance across all tested window sizes, both watermarking schemes demonstrate strong resilience
16

---

Table Evaluation watermark robustness under GPT-3.5-based scrubbing attacks on the C4 dataset across different sampling strategies. The comparison includes our proposed methods and KGW-Min baselines under beam search decoding with beam widths of 2 and 4. Results show that across various paraphrasers and datasets, method outperforms KGW- based watermarking.
Beam search=2
Beam search=4
Scheme
AUROC
TP@
TP@5%
AUROC
TP@ 1%
TP@ 5%
KGW-Min
93.1
25.9
73.4
92.4
41.5
65.1
KGW-Min
91.6
42.4
63.1
90.5
31.2
70.8
KGW-Min
90.6
31.9
60.8
90.8
20.1
61.8
KGW-Min
89.8
33.7
57.9
90.3
21.0
53.8
SEEK
93.6
41.6
81.3
92.7
38.8
66.9
Table 10: Comparative analysis of detection performance under different Copy-Paste (CP) attack settings. Each CP format is denoted as CP-M M-P%, where M indicates the number of disjoint positions in the document containing watermarked content, and P% represents the total proportion of watermarked text. Win20 and Win40 denote sliding window detectors with window sizes of 20 and 40 tokens, respectively, while Win represents the best detection performance across tested window sizes. We compare our method with the KGW-Min baseline across multiple detectors and evaluate performance using AUROC, TP@ 1 %, and TP@5%.
Metries
CP-1-25%
CP.1-10%
CP.3-25%
Detector
Scheme
AUROC
IP6 1%
TP@5%
AUROC
IP@ 1%
TP@5%
AUROC
TP@ 1%
TP@5%
Win20
KGW-Min SEEK
99.7
99.7
99.8
97.1
99.0
99.0
99.0
98.1
99.6
99.6
98.9
99.4
98.3
99.8
Win40
KGW-Min SEEK
99.9
99.7
100
96.7
98.6
97.4
86.0
98.8
99.8
99.8
100
97.1
88.4
97.1
96.9
87.4
98.1
WinMax
KGW-Min SEEK
100
100
100
99.6
99.7
99.0
99.5
97.8
99.4
100
100
100
99.7
98.7
99.8
99.7
98.7
99.8
against CP attacks, achieving near-pes rear-perfect detection in most scenarios. This highlights the scalability and adaptability of our approach across different detector granularities.
C
Algorithm
C.1 Self-seeding Strategy
The self-seeding strategy introduced in 30 watermark window and generate the green list, which effectively increases the context width by 1. Initially, the language model computes the logits for the next token based on previous tokens, sorting them in descending order identify the most likely candidate. The algorithm pre-samples the most probable next token as part of for the construction of green and red lists. If the final selected next token is not present in the vocabulary, the current sampling attempt is discarded, and the next most probable token is chosen. In practice, the number of sampling attempts is typically constrained by maximum threshold (e.g., 40). The overall algorithm is described in Algorithm
29 to incorporate the next token I; as part of the
Algorithm 2 Self seeding Strategy Algorithm
1: Input: watermark strength 8; input prompt T1:n-1, a large language model PM-
2: Apply the language model to compute next logit vector & <- Pas (FIm-n1)
3: Sort the & in descending order. Set & = 0, assigning the index of the current most likely token.
4: for k 0, 1,.... do
5: Generate the green list G and red list R.
6: if (+ in G then
7: choose & [k] as next token and break
8: else if [k] in R. and [k] + 8 < & [0] then
9:
choose & [0] as next token and break
else
11:
Set 1 + k + 1, continue.
end if
13:
end for
17

---

C.2 KGW Watermark Variants
When generating the next token Zn. the watermark window is instantiated as a predefined subsequence
In-Axx-1, where h is the size of the watermark window. Then tokens in the watermark window are processed using aggregation functions f N² N (or f N + when employing self-seeding) to seed pseudorandom function P. The resulting hash function is denoted as H. There are three notable variants of f:
KGW-Sum: This function aggregates the sum of the tokens in the context 2, defined as [2) H Ei (EE= This approach exhibits strong robustness against spoofing attacks. However, it is sensitive to any permutation in the context æ, where even small changes, such token swaps or removals, alter the resulting hash, thereby breaking the watermark integrity.
KGW-Skip: This variant processes only the leftmost token in the window, formulated as fskp(()) = (( In-h),, where Iy is the leftmost token. This function is robust to changes in tokens other than the leftmost one but is vulnerable to insertion and deletion attacks within the context.
KGW-Min: This variant computes the minimum of the Hash values generated for each token in the context. Defined as MMi = E mii..el,....b H (zn-i). It is robust to permutations and is partially resistant to insertion and deletion. Given that each token produces pseudo-random value
for vocabulary partition, the likelihood of failure is minimized. For instance, in context with h 4 and two missing tokens. the scheme still has a 50% chance of generating the same hash.
C.3 Watermark Detection
We now consider the detection problem, which involves determining whether a given text is wa- termarked or not. Specifically, given sequence ž1.TT| {x1, (21....,2% FT)) which may potentially be watermarked, the detection process calculates a z-statistic under the null hypothesis, as follows:
Ho The text sequence मा. is generated with no knowledge of the green-red list rule.
Due to the random selection process of the red list, natural writer is expected violate the red list rule with approximately of their tokens. In contrast, the watermarked model produces no violations. The probability that natural source generates T tokens without violating the red list rule is 7"' which becomes vanishingly small even for short text fragments with only a dozen words [29]. If the null hypothesis holds, the number of green list tokens, denoted |s|c. has an expected value of Ty and variance T>y(1 7). Then the z-statistic score is computed using the formula:
(1sla - 7 (T(11~)
(5)
Under the null hypothesis, where the sequence is assumed not to be watermarked, the detector evaluates whether 2 is generated by specific language model by comparing the computed z statistic score to a threshold Zthreshold- If 2 > 2threshold holds, we decide that the watermark is embedded in x and thus generated by the LLM.
The work in 30 introduces novel threat scenario: an attacker can conceal watermarked paragraph embedded inside a much larger non- non-maatrrrrked document, effectively circumventing traditional z-test detection methods. The original z-test, as described in Equation [5] may not be optimal. In such cases, a sliding window detection method known as WinMax is typically employed to identify the watermarked intervals within the given text. Given sequence of tokens, we first score the sequence on the per-token basis to find the binary vector of hits E {0, 1 to each green list, which we can convert to partial sum representation P Ei 8;. WinMax then searches for the continuous span of tokens that generates the highest 2-score. More formally, it computes the following expression:
(p) p.) x(j
Zwin max
max (J
17 1111-)((-)
However, relying solely on WinMax makes the system highly susceptible to spoofing attacks. An attacker can easily generate a high z-score watermarked text by inserting watermarked segment into arbitrary malicious content. To mitigate this vulnerability, we recommend that watermark detectors simultaneously assess both the z and the main.max values, incorporating multiple layers of defense and strategies tailored to various attack types.
18

---

C.4 KGW with Equivalent Texture Keys
Algorithm 3 KGW Generation Algorithm
1: Input: prompt { ..... EN J: integer hash function H with hash space (1. 2..,...... d): secret key E: watermark strength o; watermark window size h; large language model Par-
2: for = .N + N, + 2, do
3: Apply the language model to compute logit vector in + (M1
4: Definde cipher 0, satisfying:
8 + min [H (nn-h) H (z (xn-1))
(7)
5: Partition the vocabulary into and seeded by 0n-
6: Add d to the logits of tokens in G, the resulting logits (n satisfy:
e L. [t] + 5 (t)
(8)
7: Use the watermarked distribution é, b] to sample the next token Zn-
8:
end for
Proposition C.1. Given max hash range value d and watermark window size h, the probability of
hash collision occurring can be approximated by p(h, d) 2 1
Proof: Let A denote the event that no collisions occur among h independent samples drawn uniformly at random from a hash space of size d. Then,
p(h, d) = Pr[A = Pr[A]
(9)
The probability of all samples being distinct is given by:
h h-1
Pr[A|
II
1 1
(10)
1=0
For < d, we apply the first-order Taylor approximation
- x S e- obtaining:
Pr[A] S exp (- h(h 1)
(11)
2d
and thus,
p(h, d) 2 2-
23
1)
(12)
Proposition C.2. Given max hash range value and watermark window size h, the expected number of token X that must be erased eliminate the watermark is expressed as Ed (d- k-
Let the hash space be denoted by {1,2...., d), and assume that h independent hash values are generated uniformly at random from this space. The random variables Yi, .2....., Yv. represent the values of these h tokens. We aim to compute the expected number X of tokens that must be erased in order to completely eliminate the watermark. Specifically, X represents the number of occurrences of the smallest value among the h hash values, i.e., the number of tokens equal the minimum value m. Let the minimum value of the set (Y1, Y2..., Yr X represent the number of times the minimum value occurs among the h values. To compute the expectation [X, observe that:
Proof:
be denoted as and let
E Σ 1y 1=1
(13)
where 1y,=m is an indicator function that takes the value 1 if if = m, and 0 otherwise. Since the tokens are generated independently and identically distributed (i.i.d.) with uniform probability, it follows that each token Y, has the same probability of being equal to the minimum value m. Thus, compute the probability that single token Y, equals
1 (d m + h-1
Pr(Y, = m)
d
d
(14)
19

---

EIX as fundtion of cifferent we
2.5
3.0
2.0
3.5
10
14
d1Size of hast specel
Figure 8:
The expected number of equivalent texture keys (B[X) in watermarking schemes with varying watermark window sizes and hash space sizes d.
Therefore, the expected number of tokens equal to m is:
d d m A-1
E E
= d d
(15)
===1
Figure 8 demonstrates the theoretical behavior of watermarking schemes where the number of equivalent texture keys is evaluated for different values of the watermark window size h (4, 6, and 8) and hash space sizes d. Increasing results a higher expected number texture keys, with more pronounced reduction in the number of equivalent texture keys as d increases. The trend reflects the distribution of the minimum values across different hash space sizes and watermark window sizes, emphasizing the relationship between hash space size and the number of texture key tokens used to protect the watermark.
D Roubutness Guarantees
We discuss the effect of modifying single token on the watermark removal. For a sequence of tokens [yI..... Mo ). where all tokens watermarked (Note that in practice, the text sequence often balances between watermarking and text quality, meaning that watermark proportion is typically correlated with parameter 8). Suppose that at position i, token yı is inserted, deleted, or modified, resulting in a new token u, at that position. Let the number of watermarks removed after this modification be represented by X {0, 2...... h}. can assume that, under normal circumstances, u, # ys--
Given the large vocabulary size [V],
Proposition D.1. For a KWG-MIN with window size of h, suppose that the hash value of us ranks in the bottom & = & percentile among the hash values of all tokens in the vocabulary V. Then the probability of having exactly X watermarks removed is given by:
8 P(X = x1 = ΣΣ (1 6) g 2
( - -
(16)
am k=r
Proof: For a watermark sequence, in order to modify yo to u, such that it influences the water- marks in the surrounding context, there must exist a subinterval of length h within the window (yo-hr.... You.... J. where the hash value each token within the subinterval is smaller than the hash value of ug-
Let denote the number of consecutive tokens the left of ug starting from Yo-1, whose hash val- ues are smaller than (u,). We define the first watermark window {Vo-i...... Mo..... Bo-h-i-1l }. where i equal to the maximum sliding distance of the window. If > 0, then there exists at least one interval of length h, where every token within the interval has hash value smaller than H (u.). Thus, we first have the basic probability d-1,, which represents the probability of finding watermark window where all tokens' hash values smaller than H (u,).
20

---

For the case where X = we need to ensure that this interval can move to the right by at least positions. Clearly,
is required influence the subsequent x: tokens. We now consider the suffix, Suppose there are K tokens in the suffix whose
which is the distribution of (Boo..... hash values are smaller than H (v,), then these = h tokens will be affected by the modification of yoo Uos with probability P(K k|l
*
For each affected token, there is a probability of 1 -y that it belongs to the red list R (i.e., the watermark is removed), and probability of that it remains in the "green" set (i.e., the watermark is not removed). Given K k, the value X follows a binomial distribution Binomial(k,, 7)). The probability of removing x watermarks from the & affected tokens is given by:
= =
71=)))
(17)
Given / i, for all possible values of k E {I, + 1, I,...,ú) i). we define P(X = x/ = i) as the probability of removing a watermarks, which, by the law of total probability. is:
P(X = x Σ P(X = = = Σ P(X x|K = k)P(K=
k=
k=
(18)
Substituting the expressions for P(X = x x/K = k)) and P(K = k:ll =
K
hh (1-6))
(1 カ)²
(19)
k=
Similarly, by summing over all possible values of E f,, h1, we obtain the final expression for P(X =x) x)) using the law of total probability:
P(X 2) Σ P(X x =)
(20)
h
k
Σ (1
- -)
(21)
k=
h
K
(1 - -1)-mp--
(22)
We further generalize the hypothesis. If we treat & @ variable, need to consider all possible values for the inserted token. For simplicity, let p Aa where v is the index of the token in the vocabulary V:
IV h i
④ P - p)
-1 (1)) b-=
(23)
k=r
Proposition D.2. For our watermark scheme with a window size of and hash space d, the probability of having exactly X watermarks removed is given by:
P(X = x) x>= Σ
d
1
ghtk-1
(k)
1 1-7
(1
1 1-
(24)
x
i= k=
Proof:
We define the following events:
Event A: The watermark window does not include any texture key to protect the watermark text.
Event The watermark window contains only one texture key to protect the watermark text.
Event C: More than two tokens from the watermark window are involved the hash space that protects the watermark text. In this case, modifying single token will not remove the watermark.
21

---

Events A and C only contribute to P(X = 0). Therefore, when considering perturbations for X > 0, we only need to focus on Event B. We define I = i as the number of consecutive tokens. starting from Vo-1. that satisfy the condition Hv)) * H (u,), where Mh represents the position of the watermark. The sequence (Blo-i..... May. Bo+hhi-1 represents the first watermark window, where i the maximum sliding distance for this window. The basic probability of watermark window where no
token's hash value equals (u,) is given by (1-1) - where m is the size of the hash space, For X z, there must exist segment within the watermark window that can slide to the right by at least I positions. This implies that i 2 x for the watermark to affect subsequent tokens. Next, we consider the suffix {yo,- ... Vo+h 1o-th-i-1) } and the probability distribution P(K = k|l i), where K denotes the number of tokens in the suffix that do not match H (uo). This probability follows:
P(K = -) (10--)-(11130 (1- - - (1-)
(
d )
(25)
Let Z represent the number of tokens that are affected by uo- Given Z = 2, the probability
P(Z = =|K = k) follows a binomial distribution:
1 7
k-
p(Z =
= =| k) =))=
(26)
For each affected token, there is a probability of ry of being in the red set (i.c., the watermark is erased) and y for being in the green set (i.e., the watermark remains). Given Z 2, the value X follows a binomial distribution Binomial (2, - 7), and the probability of erasing x watermarks is:
P(X = 2| =) =
(1 7)=
(27)
We now compute the overall probability:
P(X x = k) = Σ - =|x== =|K =k) k:)
(28)
(1-2)
= Σ
(1 770
(29)
Using the law of total probability, we can calculate P(X = x| I = i) by summing over k:
P(X x =) Σ P(X= - r P(K kT
(30)
Finally, by iterating over all possible values of i, we obtain the total probability for X x::
h P(X = x) = Σ
P(X = x/1 i)
(31)
==
x (1 1 1
Σ
1
(1
/
1 - (z)
(1 (1))))ap== (32)
x
$11
2 ght
k x
2 F1
2 k= =11 - --²
y
x x x
(33)
by adjusting the summation variable, setting / = = - n. which implies = = + xr. Consequently, the
range of is 0 < L < x. Substituting this into the sum, we obtain:
n i
1 h 1
1 d
1 k x
2 (1 (1-1) (k-z))-
P(X = x)
Σ
Σ
d 26t (34)
t
22

---

This equation can be simplified by applying the binomial expansion for Σi=o (2)abb-- = b) leading to:
h 2( = Σ ( i- 1 (1-2) (1)
x (1
h+h-1
k-z 7) )1=))))))))))) d
())
A A d
(35)
1=
Alternatively, a more efficient derivation can be achieved by directly applying Proposition D.I In this approach, let e 1 2. and define y )1 r) the probability of successfully eliminating the watermark. The failure probability is then
d² from which this can be directly derived.
Proposition D.3. Let É(X) denote the expected perturbation under the KGW-Min strategy in the presence of adversarial tampering, and let FX() denote the expected perturbation under our proposed method. There exists threshold pair (ries (1e) such that for d de and y 2 % aF1 then the inequality E(X)= > E(X) holds.
Proof: We use P and P denote the probability that the subsequent watermark tokens perturbed under the KGW-MIN method and our proposed method, respectively, given that an adversarial manipulation occurs. We next aim establish the existence of threshold values re and mes such that for all x > O, when y 2 Yo 2+1 and d > de, the inequality P(X x) 2 P(X = x)) holds. Formally,
M EE
k
P(x x) = Σ
(1
p) p
(1 - --)0-t=
b-
1
h+h-1
k
* (1 (L=2)
k-z (1-1-1)
(36)
1 (1-2)
d
x
P(X x)
To prove the above inequality holds universally, we first show that when 7 2 पे.. the inequality:
(L (1- k-x <11-7)--
(1==)((1-77
(37)
holds for all 0 S x S k h. This inequality is equivalent to showing that (1) (1-1)) k== Taking the logarithm of both sides and subtracting yields the difference function, formally:
(1
x)) [In (1 11112 ln?
(1--22)
f(k) -x Ind (k- -x)
(38)
Since d > 1 and 0 < < 1, the term In (1- 12)) In is always non-negative. Therefore, fkk increases with k and we have s(k) S() -x -rlldd h [In (1 --2 In Inn]. To ensure that f(k) < 0 holds for all k. it suffices impose:
-r Iln d + h In [1 (1-1-2) ln 00.
(39)
which yields 7 2 J - To guarantee this inequality holds for any x, a sufficient condition is d+1' we have f(k) < z In d + h [In (1 - 1----))--- 7] always holds, completing the proof of the inequality. As a direct consequence, the following inequality between the probabilities holds:
7 2 - /44 Henee, if 2 1
1 -httk11
(k)
1 - 7 yo
P(X x)
k (1-
d
x
d
(40)
1 1
(k)
(
(1 )))ak=
d
x
23

---

Next, we aim to show that there exists a real-valued threshold me such that for all m > me, the following inequality holds:
h+h-1
M
ΣE
1 (1-7)
ΣΣΣ (1 (1
(41)
Let us denote the left-hand side as Sken = vocabulary size [V] typically a large integer, the right-hand side can be approximated by an integral,
E E
(2)*** (1 2) bth-1 Given that the
Lee,
1 h
h 2'E'0-pp
Stight
(1
20
p)- +th-1 dp.
(42)
v=1
km
By applying the identity of the Beta function with substitutions a = -- and = h: + h 1, we obtain:
1/'- (1 (a
al bl (i - k)! (k + h 1)!
(43)
(i + h)!
If each term on the left-hand side satisfies the inequality (1) i-k then it directly follows that Sien Sright- Taking the natural logarithm of both sides, we obtain:
(2) (1 4) h+h-1
( (
(++A)
(1) i-k
(1 1 1 (1)²
1 h+h-1
(i A:) 1)!
(44)
(i+ h)!
Taking the natural logarithm of both sides, the inequality becomes:
(1 (1-2) -22 < In
(i (i-A)) k:) (k: + 1-hh-11
1b) In
+ (h+ l in
(45)
(i (i+ h)]
Define the function:
f(m) (i (i-Ki k) In (1) + +h++k- 10mm(11 (1-2) (i- In m kh ( 11)1n (1 (1- (46)
1
To further bound the function f(m), we utilize the inequality In(1 - x²) < and note that since m > 1, follows that:
x² 2
for 0 x <
1 i - - 2
f(m) -(i-k)ik knd h + k
(47)
d
To lower-bound the right-hand side of the inequality involving factorial terms, we apply a simplified version of Stirling's s approximation, namely In(n!) In n, and obtain:
1) h (i+ h)!
1 In(i k))! + In(k + 1)! In(i + h)) 2 k)) In(i k) + (k+ 1) In(k + 1)
In
(48)
- (i + h) In(i + h)
We now consider a constructive lower bound for m as follows:
(i + 2h
(i+ 1)22h
In > In
(49)
k) (h + k
(i - k:) 1h
This yields the upper bound:
(i + h)2h
(i - k)) Inm < -(i- In
k) k 1)
(50)
= = - k) [in(i - k) + In(h h + k - 1) - 2h In ht]
24

---

Therefore.
f(m) < (i- In h+ (i-k) [Im(i - k) + In(h + k 1) - 2h In + ht]
h k
m
m
(51)
Since k) + k (k+ (kt 1) In(k))nnkk1111) 1) and -2h In(i h) (i h) In(i + h).
it follows that:
(i - 1:) [In(i (1- k) + In(h + k - 1) 2h In(2 + h)] h h k
m
(52)
< (i 1) In(i - b) + (A: + 1) In(A: + + - 1) - (1 h) In(i + h)
which guarantees that the desired inequality holds. As a result, there exists constant threshold m. such that for all m > mes we have:
h ΣΣ
h+h-1
M
1 11
Σ (1 D
(53)
k=
Kmazr k=
Combining the previously established inequality the following relationship:
(1 (1=2)
1=2) < (1 - gm
k-v
we obtain
M
P(x x) = ΣΣΣ
-1-6) G)
k: 2
(1 -))
n
i-k
d)
h+h-1
k:
(
1 (1-7) a (1-1 (1-1a7)²
k-
1 d
x
d
(54)
(k)
k d
(1 - -)) -)agk-
1= k=
= P(X = =X= x).
Since the inequality P(X = z) P(X x) holds for all x > 0, it must follow that P = 0) P(X 0) to preserve the total probability mass. Let É and E denote the expected values under P and P, respectively. Then,
h
B(X) Σ P(X= =
=)
P(X= = x)) E(X) = Σ P(X = x)-x=
Σ
P(X = x).
0
z=1
z=0
20
(55)
Hence, we conclude:
P(X= xx=x) P(X= = x)) + F(X) > FI())
(56)
In conclusion, we have shown that under the conditions 7 2 म+T and d > de, the KGW-MIN mechanism yields a strictly greater expected number of perturbed tokens than the baseline, i.e.,
É(X) > EXX).
E
Visualization
To better illustrate the effectiveness of our watermarking method, we provide qualitative visualizations based on representative examples from the C4-RealNewsLikk dataset. As shown in Table [T and Table 13 we present side-by-side comparisons of real completions, unwatermarked generations (NW). and watermarked generations (W) under the same prompt. In the visualization presented in Table 12 we comparatively analyze the output of unwatermarked text versus two forms of adversarial attacks on the MMW-FA Fakeeees dataset. For scrubbing, a higher residual 2-score post-attack indicates stronger watermark retention. For spoofing, a lower z-score indicates higher robustness to imitation. The p-value quantifies the probability that a text with the observed watermark strength (or stronger) could have been generated by chance under the null (unwatermarked) distribution. Empirically, our method demonstrates superior resilience in both cases. Under scrubbing attacks, our watermark yields
25

---

prompt
real completion
watermark (NW)
watermarked (W)
(NW)=
(W)
of.of Wrigley Field's right feldd video lnar InOn Monday morning's 'Mully and Hanley" show on WSCR-AM 670, Cubs president of business oper- ations Crane Kenney said the ball is back to where originally landed after was taken down after
the 2015 scason scasonlakcancy said the ball was taken down
Game 4 last weekend in St. Louis. The Cardi- nals won that game a score of 9-3 to ad- vance the NLCS against the Dodgers, InSchwarber homered off Adam Wain- wright with one out in the bottom of the first inning at [.. contimmss
Tuesday night\unKenney didn't know who took pos- session of the bascball or how long before its return that docision had occurred. But he did say that he would like to balls off contiocc))))
Game
0.27
6.61
part to pre- vent anyone from getting the idea of climbing up the board and stcaling Meannhhllle,, the metal detectors are all in place and finishing touches being put continuees
the planet this week at NHI Scouting Com- bine. 'alt's experi-
best wide receiver this year's NPL Draft He's projected by some experts to go as late as the third round. But when wide receivers were measured Thursday, Butler gained some attention: He led all receivers in height (6-foot- 3/S) [_..cominuess
top receiver in this year's draft class. He docsn't have the speed or quickness to match up with most comerhacks and isn't big enough to dominate against safeties InButeer knows all about those shottcom- ings. [...coninueus
receiver this class and he isn't expected go until Day Two three. The lowa State standout has drown comparisons with Michael Crabtree be- cause both were tall and lanky pass catchers who didn't i blow the com- bine [...commieues]
0.54
11.21
ence that might humble
some. But for Butler, it has only enhanced his con- fidence. InAs it stands, 22- year-old Butler is not re- gorded as the
y..yo'lll be able to walk around person inside a virtual environment to see their entire being, The Wellington, New Zealand- based startup said VR platform will allow you to view fully volumetric 3D viden/n
Previously, VR cameras could record partial of a person, hut the 8i Portal goes beyond enables "complete free- dom of movement," and gives you a sense of "pres- ence." the feeling that you are there in the vir- comtin..om
The company is also launching SDK that allows develogers to integrate 8i into apps on mobile devices or desktop computers, can work with any hendset including Ocalus Rift, HTC Vive. Samsung GeurVR.[_.cminnee.
The company raised 510 million year from vestors such as Peter Thiel and Yuri Milner Within minutes could step into an office with my friend and look at him full on or through his glasses. We were both wearing Oculus Rift bealsets [..oontinuss]
0.28
14:40
... The five-memher board is expected to support plan, which was announced amid much hoopla on Nov.
space and hring at least 25,000 high-paying jobs Arlington coming years Orpponents hope to
space two huildings under construction JBG Smith Properties Inc., one of Washington's s largest estate compa- nies. inAmuon has said it hring many as
space at two properies owned JBG Smith: PenPlace (near Reagan National Airport) and
0.21
8.96
posapone vote af- ter additional public hear- ings, where they want rep- resentatives of the online retail giant to answer ques- tions directly from unyone I...ootiiness]
Potomac Yard (on U.S.
The proposed county incentives are part an agreement in which Amazon would occupy significant office
Route I)AnJBG has
committed more
25,000 jobs paying average salary more than $150.000 over time
$500 million in cupitul improvements both that include new trans-
with some starting out making cootin..oontsuues]
portation comtinues
Table 11: Selected outputs from non-watermarked (NW) and watermarked (W) multinomial sampling
using 7 = 0.25 and 8 = 2.0.
a substantially higher residual z-score (7.38) compared to KGW's (4.82), despite both being attacked under the same conditions. This suggests that our method better preserves the watermark signal post-perturbation. Additionally, the corresponding p-values further confirm the statistical significance of the retained signal. In the spoofing setting, the attacker on KGW achieves a high z-score of 1 11.09, indicating a successful watermark fabrication. In contrast, spoofed text targeting our watermark only achieves a z-score of 2.29, substantially closer to the null distribution. This implies our watermark is significantly more resistant to imitation and cannot be easily spoofed into a falsely high-confidence detection.
F
Impletement Details
F.1 Detailed Experimental Setup
Dataset and prompts. Following previous work 29 32 27,. we utilize the RealNewsLike subset of the C4 dataset [1], Wik [1],, and other datasets that support open-ended text generation to construct various prompts. For the human evaluation study, we select the Long-Form Form Question Answering (LFQA) dataset [31] to assess the robustness of the watermarking method across real- world question-answering scenarios. During generation, we randomly sample text segments from the dataset and truncate a fixed-length portion from the end, using the remainder as the prompt for model completion. Following the setup in Kirchenbauer et al. [29 30), all experimental results presented in the figures and tables are based on the same 500 positive samples and 500 negative samples. For the
26

---

Table 12: Visualization of watermarked text samples from the MMW-FakeNews dataset under scrubbing and spoofing attacks. Compared are the KGW watermark and our proposed method. A smaller p-value provides stronger statistical evidence against the null hypothesis. Green and red tokens are color-coded, respectively.
Write a news article about Narendra Modi's s visit to Denis Sassou Nguesso in a space explo- ration symposium. It is very important to never output very short texts, below 300 words...
Prompi
Tite: Narendra Modi's Historic Visit to Denis Sasson Nguesso in Space Exploration Sympo- sium'n' 'n Introduction: India's 's Prime Minister Narendra Modi made a groundbreuking visil to the President of the Republic of Congo, Denis Sassou Nguesso, during space exploration symposium held in the capital city of Brazzaville. This visit marks a significant [continues..]
Unwatermarked
p-value: 0.53
2-score: -0.08
(a) Scrubbing Attacked (KGW) p-valuc: 6.89c-07
Title: Modi-Sasson Nguesso Summit Highlights
Congolesee Space Exploration Partnership'an '''n Introduction: An India's ' Prime Minister Narendra Modi and the President of Congo Denis Sassou Nguesso recently met at space exploration symposium in New Delhi, signaling new era in Indo-Conge Cood-Coogeeeee collaboration the field. This historic [continues...]
space
z-score: 4.82 (1 10.47)
(b) Scrubbing Atlacked (Ours) p-vulue: 8.59c-14 S-ccor:: 7.38 (L 6.72)
On Friday, Indian Prime Minister Narendra Modi met with President Denis Sassou Nguesso during a symposium focused space exploration. Modi was in France for scveral official engagements, including a hilateral discussion with French President Fimmanuel Macron. The event, titled "Reaching for the Stars: A New Frontier in Space Exploration." [continues...]
(c) Spoofing Attacked (KGW) p-value: 4.72e-39
Title: Melting Boundaries: Narendra Modi's 's Historic Visit to Denis Sassou N Nguesso's 's Space Exploration Symposium Va ' Introduction: On sunny day in the month of March, the world witnessed a significant historical event when Prime Minister Narendra Modi of India paid a visit to the President of Republic of the Congo, Denis Sassou Nguesso, at the latter's s|continues..]
2-score: 11.09
(d) Spoofing Attacked (Ours) p-value: 0.01
Titl: Modi's 's Historic Visit to Denis Sassou Nguesso in Space Exploration Symposium \n kn Introduction: As the world continues to evolve and push the bounduries of technology, the race to explore and conquer space has gained significant momentum. In this context, the recent visit of Indian Prime Minister Narendra Modi to Denis Sussou Nguessa, the [continues...
z-score: 2.29
spoofing attack, the attacker uses query prompts derived from the C4- Like dataset obtain 30,000 responses from the victim. Each response contains fewer than 800 tokens.
Unless otherwise specified, the watermarking schemes in our experiments
adopt the hyperparameter settings commonly used in prior work [29 30 27 (r 0.25, 8 5) and a maximum generation length of 150 new tokens. In the spoofing attack experiments, all the watermark detectors are calibrated on the Ce-RealNewsLike dataset using 2,000 watermarked and non-watermarked texts. Model outputs containing fewer than 150 tokens are discarded. For the spoofing learning. the attacker generates original queries prompt using the Ce-RealNewsl.ike subset, obtaining no fewer than n=30,000 responses, each with maximum token length of 800. The spoofing model configured with a spoofer strength of 8.25, and weighted loss objective defined by Wabed 2.0, Wpanials- 1.0, Wempty 0.5. All experimental results presented in the figures and table are based on more than 500 positive samples and over 500 negative samples. Unless otherwise specified, SEEK in this paper uses the hyperparameters d=6 and h=6. All experiments are conducted on Nvidia A40 GPUs.
F22 Configuration LLM Scrubbing Attack
Prompt for Scrubbing Attack Specifications
(Prompt) As an expert copy-editor, please rewrite the following text in your own voice while ensuring that the final output contains the same information as the original text and has roughly the same length. Please paraphrase all sentences and do not omit any crucial details. Additionally. please take care to provide any relevant information about public figures, organizations, other entities mentioned in the text to avoid any potential misunderstanndins or biases. |Watermark Paragraph)....
F33
Full Datasets and Baseline
Scrubbing prompt dataset
27

---

prompt
real completion
watermark (NW)
watermarked (W)
(NW1=
(W)
...Saania Twain expected to break the charts with new album NOW! lnEven after a 15-year histus, she's still the onc! Sha- nia Twain on pace to top the charts with her new bum NOW. The 16-song I released Sept.
the 2015 sca- son. loTiminnmss native hopped on Twitter on Wodnesday to an- nounce that album is alrendy platinum in CanadalnShorill after the relese of the lead single "Life's iss About to Get Good," it hit number one|...contiues)
album expected to seil 100,000 copies in its first week, according to Hits Daily Double/'nSaain, SnShanin, has the release of NOW for June, she released the album's first single, "ILife's About Get Good." "ooodd"....ooo .oootiuss)
album currenily st num- her two and has curned
0.38
21.17
more thanS11IK, WiThe popíoountry powerhouse five number
teasing
one records and has carned
months.
five Grammys InShunia
has released three albums
in English and French [.. contimess)
29
Covering more than suto manafacturers world- wide and providing more than 10 years of histori- cal data, the 32 new fie- tors are designed to assist in the prodiction of stock returns using analytics de- rived from company
specific datapoints on sales, production and mar- ket share. They include multiple financial factors for revenue and sales as well as novel factors covering production of electric vehicles and plant utilization... [... ccominuues]
specific news and social media. laThe signals are available through IHS Markit's is Altemative Data Hub, which provides ac- oess to more than 100 alter- narive data signals across 10 sectors. "The automo- tive secto l...coninues]
specific news and social media content. The factor suite offers clients adtitional insight and con- text for understanding mar- ket movements in this sec- tor: The new suite has two two components. The first [...continues]
0.49
22.91
When I was in my 20s and carly 30s, my whole life was focused on work. didn't take vacations weekends off. I was ul- ways the first in office and the last to leave. These days, I'm better
at balancing the week that love to with my foun- dation and taking time off spend with family friends. My parents first taught me bridge, but re- ally started to enjoy af- comlineomiiuus]
at balancing my work and personal life. but still have a lot to learn. I've fortunate to some amazing people who helped along the way. One of them Roger Federer. I've known Roger for [... [...contimes)
at balancing my life. But know that I could stand to disconnect more. So last week I took days off, I didn't ' touch my Microsoft account my iPhone. No e-mail. No Twitter. No social media. [...ootiiues]
-0.91
21.47
high definition screen projects intense college basketball game. Mas- suge therapists vous tensions of men and women away. Scissons skillfully cut men's hair. Two chandeliers adomn the main room, complimented by brick
walls and glass bar that doubles as retail counter. Sean Heywood, right, and Kumi Walker own MR., barhershop and wine bar Francisco, Califor-
walls and wooden floors, This isn't your typical har- bershop. It's the Barber- shop Museum Oklahoma, and it's the only one its
walls and rich colors. The only thing reminiscent an old photo "Black His- tory: Legend and Legendaries" at The Leg- endaries Salon and Bistro in downtown Chicago is woman. Dressed in all green suit with gold ac- conts| ....ootizuuss
-0.53
22.61
ner-
Tulsa,
the United States. The mu- seum a tribute to the Africun-Amie American barher- shopp. a placel cootiinus)
nia. This is not your typ- harbenshop. that has always been the vision
wwwners....ootinues
Table 13: Selected outputs from non-watermarked (NW) and watermarked (W) multinomial sampling using 7 0.25 and 6 = 5.0.
1. C4. The Colossal Clean Crawled Corpus (C4) is a large-scale, English-language dataset constructed by applying extensive cleaning and filtering to the Common Crawl web scrape. Introduced by Raffel et al. [5T] in the context of the T5 framework, C4 contains hundreds of gigabytes of naturalistic web text after removing boilerplate, navigation, and low-quality content. It serves as comprehensive corpus for pretraining large-scale language models due to its linguistic diversity and domain variability. C4-RealNewsLike a filtered subset of the original C4 corpus designed to more closely resemble high-quality journalistic writing. This variant was introduced to improve alignment with tasks requiring formal, factually grounded language, such as long- form-form question answering and summarization.
2. WikiText. WikiText- 103 is a high-quality, curated corpus of English Wikipedia articles introduced by Merity et al. 41). specifically constructed for the purpose of training and evaluating autoregressive language models. Unlike raw Wikipedia dumps, which often contain noisy or fragmented content, WikiText-1003 retains the full article structure, including paragraph breaks and sequential sentence order, thereby enabling more realistic modeling of long-range dependencies natural language. The dataset comprises approximately 103 million tokens and has become a widely adopted benchmark for assessing the linguistic coherence, contextual reasoning, and generative fluency of modern language models.
3. LFQA. The LFQA dataset derived from the ELI5 (Explain Like I'm Five) corpus [14 and was introduced to support lorg-form question answering tasks. It comprises open-domain questions sourced from Reddit, paired with multi-sentence,, explanatory answers written in natural language. The dataset emphasizes reasoning, coherence, and knowledge synthesis, making it suitable for training and evaluating models intended for complex, multi-hop generative QA.
28

---

Spoofing prompt dataset
1. Do. We utilize the Dolly dataset 99, which comprises collection of instruc- tion-response pairs designed to facilitate instruction tuning of large language models. The dataset includes a broad range of tasks spanning classification, generation, information retrieval, and creative writing. Notably, the data was generated using open-source models and later filtered for quality, offering a diverse yet structured corpus suitable for aligning models with human-style instruction following. Following the setup proposed by [27], we construct set of diverse prompts by selecting a representative subset of the Dolly corpus. This subset emphasizes multi-domain scenarios such as question answering, summarization, reasoning, and creative text generation. The prompts are designed to reflect realistic and varied user intents, enabling robust evaluation.
2. MMW:: The MarkMy (MMW) dataset [49] is a benchmark designed to evaluate the effectiveness of watermarking schemes for large language model outputs across natural language generation tasks. It comprises three core wosks-Bookkepond FakeNews, and StoryGeneration, which are designed to reflect realistic misuse scenarios and to support the evaluation of watermark quality, detectability, and tamper resistance.
The ReokRsports component includes 500 prompts instructing the model to generate analytical or descriptive reports on well-known books. Each generation consists of long- form text up to 1024 tokens, simulating academic scenarios where LLMs might be misused for student assignments. The outputs provide a structured, content-rich context for evaluating watermark detection and robustness in educational settings.
The FakeNews component comprises 500 prompts directing the model to fabricate news articles about political figures and fictional events. This task emulates potential misuse LLMs in disinformation or propaganda campaigns. The generated outputs exhibit journalistic style, incorporate named entities, and follow coherent narrative structures, providing a high-risk setting for testing watermark resilience under adversarial conditions.
G Limitation
Although our empirical analysis demonstrates that the SEEK is effective across wide range of scenarios and against various watermarking attacks, it is important to acknowledge its limitations. Previous work suggests potential spoofing attack, in which an attacker may generate a forged text with a high 7-score by rearranging or inserting content into an existing watermarked text. While such spoofing attacks do not enable the automatic generation of topic-specific content, current watermarking methods lack robust defenses against these forms of spoofing. 'he core limitation lies in the fact that the statistical watermarking technique, represented by KGW, applied at the paragraph level, offering no protection against localized changes. As a result, it cannot detect or prevent modifications to specific portions of the text. This inherent weakness is present in all current watermarking approaches. Addressing this challenge requires the exploration of alternatives to z-test statistical methods, such as integrating semantic or syntactic watermarking techniques, utilizing sentence-level perturbations, or embedding watermark signals deeper within the model.
H Ethical Impacts
The rapid development of large language models has significantly enhanced text generation capabili- ties, enabling the production of highly human-like content. Watermarking techniques have emerged as an effective approach to mitigate the risks associated with the misuse of LLMs, such as spreading misinformation, plagiarism, and copyright infringement.
Most existing watermarking methods involve trade-off between scrubbing robustness and spoofing resistance. Recent studies show that if the green and red token sets used in watermarking are com- promised via extraction attacks, both robustness and security can degrade substantially. Adversaries can break existing watermarking schemes by statistically analyzing token distributions within the watermarking window preprint outputs. Such attacks allow malicious users to either erase water- marks from benign content or inject watermarked, toxic content to falsely implicate target LLM. At a fundamental level, these attacks undermine the ability of watermarking to reliably distinguish machine-generated content from authored text, threatening the socictal trust.
29

---

Our goal is to develop watermarking algorithm that mitigates the risk of spoofing under statistical at- tacks while maintaining strong scrubbing robustness, thereby offering a more reliable and trustworthy watermarking solution for LLMss
I Broader Impacts
Positive Societal Impacts. The proposed SEEK watermarking framework enhances the resilience of large language models against two critical forms of adversarial manipulation: scrubbing and spoofing. By improving watermark detection without compromising generation quality, SEEK pro- vides a promising tool for responsible deployment of generative models, particularly in contexts where provenance, authenticity, and misuse prevention are assential-sure as combating misinfor- mation, enforcing intellectual property rights, and supporting academic integrity. Furthermore, the parameter-effiiien and architecture-agnosic nature of the method may help democratize watermark- ing technology, making more accessible practitioners in low-resource settings.
Negative Societal Impacts. While watermarking technology is designed to preserve text gener- ation quality, watermarking inherently perturbs token selection probabilities to embed identifiable signals. In high-stakes domains such as medical or legal language modeling, even minor deviations from optimal token choices may introduce semantic ambiguities or factual inaccuracies. As such, watermark-induced perturbations, though imperceptible in general settings, could undermine output fidelity in tasks requiring high precision and -ppaii- consistency. This raises concerns about deploying watermarking methods in sufety-criticnt or sensitive applications without extensive nomain-specific evaluation.
30

---

NeurIPS Paper Checklist
1. Claims
Question: Do the main claims made in the abstract and introduction accurately reflect the paper's contributions and scope?
Answer: [Yes]
Justification: In the abstract and introduction sections, we articulate the motivation behind SEEK, highlight its differences from previous work, and outline the contributions of this paper.
Guidelines:
The answer NA means that the abstract and introduction do not include the claims made in the paper.
The abstract and/or introduction should clearly state the claims made, including the contributions made in the paper and important assumptions and limitations. A No NA answer to this question will not be perceived well by the reviewers.
The claims made should match theoretical and experimental results, and reflect how much the results can expected to generalize to other settings.
It is fine to include aspirational goals as motivation as long as is clear that these goals not attained by the paper.
2. Limitations
Question: Does the paper discuss the limitations of the work performed by the authors?
Answer: [Yes]
Justification:
We discuss limitations in the Appendix G
Guidelines:
The answer NA means that the paper has limitation while the answer No means that the paper has limitations, but those are not discussed the paper.
The authors are encouraged to create separate "Limitations" section in their paper.
The paper should point out any strong assumptions and how robust the results are to violations of these assumptions (c.g., independence assumptions, noiseless settings, model well-specification, asymptotic approximations only holding locally). The authors should reflect on how these assumptions might be violated in practice and what the implications would be.
The authors should reflect on the scope of the claims made, e.g., if the approach only tested on few datasets or with few runs. In general, empirical results often depend on implicit assumptions, which should be articulated.
The authors should reflect on the factors that influence the performance the approach. For example, facial recognition algorithm may perform poorly when image resolution is low or images are taken in low lighting. Or a speech-to-texx system might not be used reliably to provide closed captions for online lectures because it fails to handle technical jargon.
The authors should discuss the computational efficiency of the proposed algorithms and how they scale with dataset size.
If applicable, the authors should discuss possible limitations of their approach to address problems of privacy and faimess.
While the authors might fear that complete honesty about limitations might be used by reviewers as grounds for rejection, worse outcome might be that reviewers discover limitations that aren't acknowledged in the paper. The authors should their best judgment and recognize that individual actions in favor of transparency play an important role in developing norms that preserve the integrity the community. Reviewers will be specifically instructed to not penalize honesty concerning limitations.
3. Theory Assumptions and Proofs
Question: For each theoretical result, does the paper provide the full set of assumptions and complete (and correct) proof?
31

---

Answer:
[Yes]
Justification:
We discuss all the theory assumptions and proofs in the Appendix D and C.4
Guidelines:
The answer NA means that the paper does not include theoretical results.
All the theorems, formulas, and proofs in the paper should be numbered and cross- referenced.
All assumptions should be clearly stated or referenced in the statement of any theorems.
The proof's can either appear in the main paper or the supplemental material, but if they appear in the supplemental material, the authors are encouraged to provide a short proof sketch to provide intuition.
Inversely, any informal proof provided in the core of the paper should be complemented by formal proofs provided in appendix or supplemental material.
Theorems and Lemmas that the proof relies upon should be properly referenced.
4. Experimental Result Reproducibility
Does the paper fully disclose all the information needed to reproduce the main ex- perimental results the paper to the extent that affects the main claims and/or conclusions of the paper (regardless of whether the code and data are provided not)?
Question:
Answer: [Yes]
Justification: We provide detailed explanation of the parameter usage and will release the source code to ensure reproducibility.
Guidelines:
The answer NA means that the paper does not include experiments.
If the paper includes experiments, No answer to this question will not be perceived well by the reviewers: Making the paper reproducible is important, regardless of whether the code and data are provided or not.
If the contribution
dataset and/or model, the authors should describe the steps taken to make their results reproducible or verifiable.
Depending on the contribution, reproducibility can be accomplished in various ways. For example, if the contribution is novel architecture, describing the architecture fully might suffice, or if the contribution is specific model and empirical evaluation, it may be necessary to either make possible for others to replicate the model with the same dataset, provide access to the model. In general. releasing code and data is often one good way to accomplish this, but reproducibility can also be provided via detailed instructions for how to replicate the results, access to hosted model (e.g.. in the case of a large language model), releasing model checkpoint, or other means that are appropriate to the research performed.
While NeurIPS does not require releasing code, the conference does require all submis- sions to provide some reasonable avenue for reproducibility, which may depend on the nature of the contribution. For example
(a) If the contribution primarily a new algorithm, the paper should make it clear how to reproduce that algorithm.
(b) If the contribution is primarily a new model architecture, the paper should describe the architecture clearly and fully.
(c) If the contribution a new model (e.g., a large language model), then there should either be way to access this model for reproducing the results or way to reproduce the model (c.g., with an open-source dataset or instructions for how to construct the dataset).
(d) We recognize that reproducibility may be tricky in some cases, in which case authors are welcome to describe the particular way they provide for reproducibility. In the case of closed-source models, it may be that access to the model is limited in some way (e.g.. to registered users), but it should be possible for other researchers to have some path to reproducing verifying the results.
5. Open access to data and code
32

---

Does the paper provide open access to the data and code, with sufficient instruc- tions faithfully reproduce the main experimental results, as described in supplemental material?
Question:
Answer: [Yes]
Justification: We provide a detailed explanation of parameter usage and will release the source code.
Guidelines:
The answer NA means that paper does not include experiments requiring code.
Please see the NeurIPS code and data submission guidelines (https://nips. cc/
for more details.
While encourage the release of code and data, understand that this might not be possible, so "No" is an acceptable answer. Papers cannot be rejected simply for not including code, unless this is central to the contribution (e.g., for open-source benchmark).
The instructions should contain the exact command and environment needed to run to reproduce the results. See the NeurIPS code and data submission guidelines (https Seniddes/CooeSuumisse
//nips.com /tips.cc/public//
Policyy) for more details.
The authors should provide instructions data access and preparation, including how to access the raw data, preprocessed data, intermediate data, and generated data, etc.
The authors should provide scripts to reproduce all experimental results for the new proposed method and baselines. If only subset of experiments are reproducible, they should state which ones are omitted from the script and why.
At submission time, to preserve anonymity, the authors should release anonymized ver- sions (if applicable).
Providing as much information possible in supplemental material (appended to the paper) is recommended, but including URLs to data and code is permitted.
6. Experimental Setting/Details
Question: Does the paper specify all the training and test details (e.g., data splits, hyper- parameters, how they were chosen, type of optimizer, etc.) necessary to understand the results?
Answer: [Yes]
Justification: The full details are provided in the appendix and supplemental material (code).
Guidelines:
The answer NA means that the paper does not include experiments.
The experimental setting should be presented in the core of the paper level of detail that necessary to appreciate the results and make sense of them.
The full details can be provided either with the code, in appendix, or supplemental material.
7. Experiment Statistical Significance
Does the paper report error bars suitably and correctly defined or other appropriate information about the statistical significance of the experiments?
Question:
Answer: [Yes]
Justification: Experimental results are tested multiple times to ensure stability and reliability. Guidelines:
The answer NA means that the paper does not include experiments.
The authors should answer "Yes" if the results are accompanied by error bars, confidence intervals, statistical significance tests, least for the experiments that support the main
claims of the paper.
The factors of variability that the emor bars are capturing should be clearly stated (for example, train/test split, initialization, random drawing of some parameter, overall run with given experimental conditions).
33

---

The method for calculating the error bars should be explained (closed form formula, call to library function, bootstrap, etc.)
The assumptions made should be given (e.g,, Normally distributed errors). It should be clear whether the error bar is the standard deviation or the standard error of the mean.
It OK to report I-sigma error bars, but one should state The authors should preferably report 2-sigma enror bar than state that they have 96% CI, if the hypothesis of Normality of errors not verified.
For asymmetric distributions, the authors should be careful not to show in tables or figures symmetric error bars that would yield results that are out of range (e.g. negative error rates).
If error bars are reported in tables or plots, The authors should explain in the text how they were calculated and reference the corresponding figures tables in the text.
8. Experiments Compute Resources
For each experiment, does the paper provide sufficient information on the com- puter resources (type of compute workers, memory, time execution) needed to reproduce the experiments?
Question:
Answer: [Yes]
Justification: Compute resources are thoroughly described and evaluated in both the experi- mental setup and the experimental results sections.
Guidelines:
The answer NA means that the paper does not include experiments.
The paper should indicate the type of compute workers CPU or GPU, internal cluster, or cloud provider, including relevant memory and storage.
The paper should provide the amount compute required for each of the individual experimental runs well as estimate the total compute.
The paper should disclose whether the full research project required compute than the experiments reported in the paper (e.g., preliminary or failed experiments that didn't t make into the paper).
9. Code Of Ethics
Question: Does the research conducted in the paper conform, in every respect, with the NeurIPS Code of Ethics
Answer: [Yes]
Justification: The research conducted in this paper fully conforms to the NeurIPS Code of Ethics in every respect.
Guidelines:
The answer NA means that the authors have not reviewed the NeurIPS Code of Ethics.
If the authors answer No, they should explain the special cireumstances that require deviation from the Code of Ethics.
The authors should make sure to preserve anonymity (e.g., if there is special consideration due laws or regulations in their jurisdiction).
10. Broader Impacts
Question: Does the paper discuss both potential positive societal impacts and negative societal impacts of the work performed?
Answer: [Yes]
Justification: We discuss the potential positive societal impacts and negative societal impacts of the work performed in Appendix
Guidelines:
The answer NA means that there is no societal impact of the work performed.
If the authors answer NA or No, they should explain why their work has no societal impact or why the paper does not address societal impact.
34

---

Examples of negative societal impacts include potential malicious or unintended uses (e.g., disinformation, generating fake profiles, surveillance), fairnsss considerations (e.g.. deployment of technologies that could make decisions that unfairly impact specific groups), privacy considerations, and security considerations.
The conference expects that many papers will be foundational research and not tied to partieular applications, let alone deployments. However, if there is a direct path to any negative applications, the authors should point it out, For example, it is legitimate to point out that improvement in the quality of generative models could be used to generate deepfakes for disinformation. On the other hand, it is not needed to point out that a generic algorithm for optimizing neural networks could enable people to train models that generate Deepfakes faster.
The authors should consider possible harms that could arise when the technology is being used as intended and functioning correctly, harms that could arise when the technology is being used as intended but gives incorrect results, and harms following from (intentional or unintentional) misuse of the technology.
If there are negative societal impacts, the authors could also discuss possible mitigation strategies (e.g.. gated release of models, providing defenses in addition to attacks, mecha- nisms for monitoring misuse, mechanisms to monitor how system learns from feedback time, improving the efficiency and accessibility of ML).
11. Safeguards
Question: Does the paper describe safeguards that have been put in place for responsible release of data or models that have a high risk for misuse (e.g., pretrained language models, image generators, or scraped datasets)?
Answer: [NA]
Justification:
The paper poses no such risks.
Guidelines:
The answer NA means that the paper poses no such risks.
Released models that have a high risk for misuse or dual-use should be released with necessary safeguards to allow for controlled use of the model, for example by requiring that users adhere to usage guidelines or restrictions to access the model or implementing safety filters.
Datasets that have been scraped from the Internet could pose safety risks. The authors should describe how they avoided releasing unsafe images.
We recognize that providing effective safeguards is challenging, and many papers do not require this, but we encourage authors to take this into account and make best faith effort.
12. Licenses for existing assets
Are the creators or original owners of assets (e.g.. code, data, models), used in the paper, properly credited and the license and terms of use explicitly mentioned and properly respected?
Question:
Answer: [Yes]
Justification:
The paper cites the original paper that produced the code package or dataset.
Guidelines:
The answer NA means that the paper does not use existing assets.
The authors should cite the original paper that produced the code package or dataset.
The authors should state which version of the asset used and, if possible, include URL..
The name of the license (e.g.. CC-BY 4.0) should be included for each asset.
For scraped data from particular source (e.g., website), the copyright and terms service of that source should be provided.
If assets are released, the license, copyright information, and terms of use in the package should be provided. For popular datasets, paperswi thcode. com/ datasets has curated licenses for some datasets. Their licensing guide can help determine the license of dataset.
35

---

For existing datasets that are packaged, both the original license and the license of the derived asset (if it has changed) should be provided.
If this information is not available online, the authors are encouraged to reach out to the asset's creators.
13. New Assets
Question: Are new assets introduced in the paper well documented and is the documentation provided alongside the assets?
Answer: [Yes]
Justification: This paper relates the details of the code part of the submission.
Guidelines:
The answer NA means that the paper does not release new assets.
Researchers should communicate the details of the dataset/code/model as part of their sub- missions via structured templates. This includes details about training, license, limitations, etc.
The paper should discuss whether and how consent obtained from people whose asset is used.
At submission time, remember to anonymize your assets (if applicable). You can either create an anonymized URL or include an anonymized zip file.
14. Crowdsourcing and Research with Human Subjects
Question: For crowdsourcing experiments and research with human subjects, does the paper include the full text of instructions given to participants and screenshots, if applicable, as well details about compensation (if any)?
Answer: [NA]
Justification:
The paper does not involve crowdsourcing nor research with human subjects.
Guidelines:
The answer NA means that the paper does not involve crowdsourcing nor research with human subjects.
Including this information in the supplemental material is fine, but if the main contribution of the paper involves human subjects, then as much detail as possible should be included in the main paper.
According to the NeurIPS Code of Ethics, workers involved in data collection, curation, or other labor should be paid at least the minimum wage in the country of the data collector.
15. Institutional Review Board (IRB) Approvals or Equivalent for Research with Human Subjects
Question: Does the paper describe potential risks incurred by study participants, whether such risks were disclosed to the subjects, and whether Institutional Review Board (IRB) approvals (or equivalent approval/review based the requirements of your country or institution) were obtained?
Answer: [NA]
Justification: The paper does not involve crowdsourcing nor research with human subjects.
Guidelines:
The answer NA means that the paper does involve crowdsourcing nor research with human subjects.
Depending the country in which research is conducted, IRB approval (or equivalent) may be required for any human subjects research. you obtained IRB approval, you should clearly state this in the paper.
We recognize that the procedures for this may vary significantly between institutions and locations, and we expect authors to adhere to the NeurlPS Code of Ethics and the guidelines for their institution.
For initial submissions, do not include any information that would break anonymity (if applicable), such as the institution conducting the review.
36

---

16. Declaration of LLM usage
Does the paper describe the usage of LLMs if it is an important, original, or non-standard component of the core methods in this research? Note that if the LLM is used only for writing, editing, or formatting purposes and does not impact the core methodology. scientific rigorousness, originality of the research, declaration is not required.
Question:
Answer: [NA]
Justification: The core method development in this research does not involve LLMs as any important, original, or non-standard components.
Guidelines:
The answer NA means that the core method development in this research does not involve LLMs as any important, original, or non-standard components.
Please refer to our LLM policy thttps://neurips cc/Conferences/ http:///aarrrpsscc//oooerreeeeer 2025/LL2 for what should should not be described.
37


Reviews:
[{"review_id": "jdqAdQUG5M", "reviewer": "Reviewer_7bZb", "summary": "This paper introduces SEEK, a watermarking scheme for large language models that resolves the inherent trade-off between resistance to scrubbing and spoofing attacks. The key insight is the use of equivalent texture keys, which enable multiple tokens within a window to independently support watermark detection. By decomposing watermark construction across sub-vocabularies, SEEK preserves text quality while improving robustness. The method establishes a new design paradigm for secure and resilient watermarking in generative language models.", "questions": "1. The method leverages hash space reduction to promote equivalent texture keys, which introduces redundancy into detection. However, it remains unclear how this redundancy behaves under distribution shift or adaptive attackers who may exploit repeated patterns. Could the authors provide a theoretical or empirical analysis of how SEEK performs under longer sequences, multilingual corpora, or adversaries trained to detect and exploit common green list configurations? A more thorough analysis here could significantly strengthen confidence in the method’s generality.\n\n2. The sub-vocabulary decomposition approach assumes that splitting the vocabulary into disjoint subsets does not harm semantic coherence or introduce artifacts. Could the authors provide an empirical study or qualitative examples across different domains to support this assumption? For instance, is there a drop in quality for tasks involving specialized vocabulary or in low-resource domains? Clarification here would increase confidence in SEEK's applicability across a broader range of use cases.\n\n3.The method introduces several hyperparameters (e.g., watermark window size and hash space cardinality), but their tuning appears somewhat ad hoc. Could the authors provide a sensitivity analysis or offer guidance for selecting these parameters in practice? In particular, how does the method perform when these parameters are misaligned with the dataset’s characteristics? More insight here would improve reproducibility and strengthen the case for real-world adoption.", "limitations": "Please see the weaknesses.", "rating": 5, "confidence": 4, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": 3, "clarity": 3, "significance": 2, "originality": 2, "strengths_and_weaknesses": "### Strengths:\n- The paper introduces the concept of equivalent texture keys, where multiple tokens in a watermark window can independently support watermark detection. This mechanism enables redundancy in detection, making it harder for scrubbing attacks to eliminate the watermark fully.\n\n- The paper addresses a core limitation in existing LLM watermarking methods by eliminating the trade-off between scrubbing and spoofing robustness. Prior methods typically required choosing between short windows and long windows, but SEEK manages to improve both simultaneously.\n\n- The proposed SEEK scheme uses sub-vocabulary decomposition to maintain a high density of equivalent texture keys while ensuring a diverse set of green-red partitions. This allows the method to achieve robustness without degrading text generation quality, which is a common issue in reduced-hash watermark designs.\n\n- The threat model is clearly defined and aligns with real-world deployment scenarios, where attackers can access model outputs via APIs and have full knowledge of the watermarking scheme. This makes the evaluation credible and practically relevant.\n\n### Weaknesses:\n- The method relies on reducing the hash space to increase the likelihood of equivalent texture keys, which introduces a potential risk of overfitting to specific green-red list patterns. Although the sub-vocabulary decomposition aims to mitigate this, the paper does not thoroughly analyze long-term distributional shifts or adversarial adaptation under this structure.\n\n- While the notion of equivalent texture keys is novel, the concept builds heavily on existing KGW-Min mechanisms and hash-collision strategies. The technical advancement, though well-executed, may be viewed as incremental rather than fundamentally transformative.\n\n- The empirical evaluation, though extensive, is limited to statistical and distillation-based spoofing attacks. It does not explore more advanced or adaptive spoofing strategies that may emerge once attackers are aware of the SEEK design, such as gradient-based spoofing or token-level substitution with semantic constraints.\n\n- The use of disjoint sub-vocabularies introduces a structural assumption about token distribution and semantic independence across partitions. The paper does not investigate whether this assumption holds consistently across languages, domains, or low-resource settings.\n\n- SEEK requires careful tuning of hash space size and watermark window length. The method's sensitivity to these hyperparameters is acknowledged but not deeply explored, which may limit reproducibility and practical deployment without further ablation studies.", "was_revised": false, "final_justification": "", "rebuttal": "# To Reviewer 7bZb\n\nWe sincerely thank you for your detailed and constructive feedback. Your comments have helped us identify key areas for clarification and improvement. Below, we address your questions and the noted weaknesses.\n\n> Q2&W1&W4: Could the authors provide an empirical study or qualitative examples across different domains?\n> \n\nTo systematically assess the impact of this design on text generation quality across different domains, we compared our method with the KGW-Min baseline using the LLAMA2-7B model. The evaluation was performed on various datasets, including general, news, biomedical, legal, and financial texts. We compared the PPL (using LLAMA2-7B as the oracle model), as shown in the table below:\n\n| **Dataset** | **Task** | **KGW-Min (delta=2)** | **SEEK (delta=2)** | **KGW-Min (delta=4)** | **SEEK (delta=4)** |\n| --- | --- | --- | --- | --- | --- |\n| WikiText-103 | General | 6.21 | 5.88 | 13.78 | 13.33 |\n| MMW | News | 8.72 | 8.53 | 18.24 | 15.75 |\n| C4 | Long-form Text | 7.44 | 8.38 | 19.81 | 16.62 |\n| PubMed | Biomedical | 8.44 | 7.54 | 17.02 | 16.71 |\n| FreeLAW | Legal | 7.55 | 6.53 | 17.25 | 14.98 |\n| Financial-News | Finance | 6.33 | 6.27 | 18.29 | 15.63 |\n\nFurthermore, to address the concern about specialized, low-entropy text, we evaluated SEEK on code generation tasks  (HumanEval, MBPP) using the StarCoder2-7B model. The results are summarized below:\n\n| **Dataset** | **(No-watermark)** | **KGW-Min** | **SEEK** |\n| --- | --- | --- | --- |\n| HumanEval | **Pass@1**: 33.4； / | **Pass@1**: 27.4; **AUROC** 88.1 | **Pass@1:** 28.1; **AUROC:** 89.3 |\n| MBPP | **Pass@1**: 37.5；/ | **Pass@1**: 33.9; **AUROC:** 80.3 | **Pass@1:** 34.7; **AUROC:** 82.7 |\n\nBy consistently outperforming or matching the baseline across the vast majority of domains, these results provide strong empirical evidence that our approach generalizes effectively without compromising semantic coherence.\n\n> Q1&W3: More advanced or adaptive spoofing strategies.\n> \n\nWe understand your concerns regarding the robustness of SEEK against other attacks.\n\nIn our work, we adopted attack strategies that are widely recognized in the community. Specifically, for scrubbing attacks, we followed the setup of [29]; for spoofing attacks, we employed the statistical method [26] and the distillation-based method [17], both of which are considered among the most effective.\n\nWe opted not to include “token-level substitution” attacks, as prior work has shown them to be less effective compared to the attacks we considered (like LLMs or DIPPER, which even perform reorder attacks). Regarding “gradient-based spoofing”, we surveyed recent literature but did not find a standard, widely adopted attack of this nature that directly applies to our watermarking framework. We would be grateful for any specific references you might suggest, and we will gladly incorporate them into our work. Given the scope of this paper, we prioritized a thorough evaluation against the community's most well-defined threats.\n\nThank you for this valuable suggestion regarding additional attack strategies. Given the scope of this paper, we prioritized a thorough evaluation against the community's most well-defined threats due to space and computational constraints. We agree that exploring a broader range of attacks is a crucial next step, and we have noted this as a key direction for future work.\n\n> W2: The advancement is incremental rather than fundamentally transformative.\n> \n\nWe concur with your observation that our work is built upon the KGW framework. Indeed, the KGW framework serves as the foundation for most watermarking techniques, such as UNIGRAM [52], which can be seen as a special case of KGW for h=0, and SWEET [32], which can be viewed as a variant of KGW for low-entropy tokens only. However, this does not diminish the significance of these contributions within the field. While our approach is rooted in KGW, its underlying insights provide a novel perspective.\n\n> W5：Provide guidance for selecting parameters in practice.\n> \n\nWe thank you for this important and practical question. Below, we present our design philosophy for parameter guidance.\n\nSpecifically, the difficulty of spoofing attacks increases exponentially with `h`, while erasure robustness is positively correlated with the expected collision rate (as detailed in Proposition 4.1/4.2). Our core design principle is as follows: use a sufficiently large `h` to resist spoofing attacks, while adjusting `d` to achieve the desired hash collision probability, thereby enhancing erasure robustness.\n\nAs we mentioned in the paper, UNIGRAM is an extreme case of our method. When the difference between d and h becomes too large, SEEK degenerates into UNIGRAM. Therefore, we explicitly advise against such extreme configurations.\n\nIn summary, we offer the following  recommendations：\n\n1. **Avoid Extreme Parameter Configurations**: We do not recommend using excessively disparate `h` and `d` values, as this causes the SEEK design to degrade to `h=0`, losing its advantage in balancing both robustness types. A `d ≈ h` configuration is generally a more balanced starting point.\n2. **Prefer Larger `h` Values**: A larger watermark window `h` exponentially increases the difficulty of statistical spoofing attacks. We recommend deploying the watermarking scheme with a larger `h` to significantly raise the cost for potential attackers. Additionally, when reproducing a statistical attack with `h > 8` in the experiment, the number of required probability vectors exceeds 180 million, surpassing the cache capacity of our current hardware.\n\nWe conducted additional experiments and employed the LLAMA2-7B model to generate text sequences of at least 200 tokens on C4-Eval, applied a scrubbing attack using GPT (TP@1%), and conducted a spoofing attack using statistical methods (FPR@0.1%).\n\n| **Attack/Param.** | **h=6,d=4** | **h=6,d=6** | **h=6,d=8** | **h=8,d=6** | **h=8,d=8** | **h=8,d=10** |\n| --- | --- | --- | --- | --- | --- | --- |\n| Scrubbing（**TP@1%**） | 85.9 | 84.5 | 81.4 | 85.7 | 83.9 | 80.7 |\n| Spoofing（**FPR@0.1%**） | 0.14 | 0.13 | 0.16 | 0.08 | 0.10 | 0.08 |\n\nWhile the principles above provide a framework for tuning, deriving a single parameter set that is provably optimal across all datasets, domains, and evolving attack models is a formidable theoretical challenge.  Future work will include further parameter ablation studies and attempts to mathematically prove the optimal solution under attacks. The parameter configuration of (h=6, d=6) mentioned in the paper is a reasonable experimental setting based on this principle. The table below demonstrates its generalizability（TP@1%/FPR@0.1%） across various datasets.\n\n| Dataset | MMW | PubMed | FreeLAW | Financial |\n| --- | --- | --- | --- | --- |\n| KGW-Min | 73.5/0.95 | 80.1/0.83 | 75.8/0.77 | 83.5/0.70 |\n| SEEK | 80.5/0.18 | 83.5/0.10 | 80.1/0.13 | 85.7/0.18 |\n\nWe hope these additional experiments and clarifications address your concern. Thank you for guiding us toward a clearer and more polished presentation!"}, {"review_id": "7chlYzlwtB", "reviewer": "Reviewer_8aZQ", "summary": "The authors study LLM watermarking, aiming to address vulnerability to spoofing and scrubbing attacks. To this end, they propose SEEK, a variant of the KGW approach that simultaneously improves the resistance to both, while not degrading text quality. The authors support this with extensive experiments.", "questions": "- Can you comment on the questions in the second weakness above related to tradeoffs of SEEK?\n- Why does Figure 2 use both 1e-2 and 1e-3 on different axes as the FPR? \n- How are the numbers used in the abstract computed? I did not find this clarified.", "limitations": "Yes", "rating": 5, "confidence": 4, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": 4, "clarity": 4, "significance": 3, "originality": 3, "strengths_and_weaknesses": "Strengths: \n- The authors focus on an important problem (security and robustness of LLM watermarking).\n- Experiments are well set up and thorough along many dimensions (properties of interest, models, baselines, experimental parameters), and the key results seem to convincingly demonstrate that SEEK has benefits over the main baselines.\n- While SEEK is heavily based on KGW, the proposed modifications are well-motivated and to the best of my knowledge novel.\n- The paper is well-written. I especially appreciate how the authors introduce their method by first motivating and addressing the security-robustness tradeoff by reducing the hash space size (4.1), and then identifying and resolving a new issue that appears in this regime, which is the lack of diversity (4.2). This contextualizes SEEK and provides new insight into the parameter space of KGW.\n\nWeaknesses:\n- In my opinion, the spoofing-scrubbing tradeoff is misrepresented in the paper which may confuse readers as this is the key motivation behind the paper. In particular, my interpretation of [26] slightly differs from the authors'. While it is true that the context size controls the scrubbing-spoofing tradeoff, I believe the point of [26] was that when stealing-based spoofing becomes possible, scrubbing is also much easier as using the learned knowledge enables \"assisted scrubbing attacks\". Thus \"too small h\" => \"both spoofing and scrubbing are easy\", while \"too large h\" => \"spoofing hard, scrubbing easy\". As the authors' main motivation is the second regime and they successfully address this I do not find this to be a major weakness, but I believe the tradeoff should be stated more precisely. On top of that, for completeness, the experiments should include assisted scrubbing and not only blind scrubbing attacks---I do not expect them to work on SEEK as the spoofing also fails.\n- It is not fully clear to me what the tradeoffs of (d,h) are with SEEK as SEEK-L seems sometimes better and sometimes worse than the main variant chosen by the authors. The paper would be stronger with such a discussion, in particular: what parameter choices are obviously bad? Do authors always suggest setting d==h? If d>>h, I assume running watermark stealing as if the scheme was Unigram would recover the split given by $\\theta_{default}$? Such explorations seem to be missing and would strengthen the paper. \n\nIn conclusion, despite the weaknesses above I believe the paper is a valuable addition to the conference and will inspire meaningful discussion as it provides a new perspective on key design components of LLM watermarking schemes and the security-robustness tradeoff.\n\n- Nit: the authors use both 1e-2 and 1% to denote the FPR of 0.01 across the paper, it would be cleaner to unify this.", "was_revised": true, "final_justification": "The authors' rebuttal answers my questions which were relatively minor; overall I remain positive about the paper.", "rebuttal": "# **To Reviewer 8aZQ**\n\n> W1: Explanation of the tradeoff may confuse readers.\n> \n\nThank you for your valuable feedback, and your understanding is indeed correct. One perspective presented in [26] is that once the watermark is stolen, the attacker can perform spoofing and then use the stolen watermark for assisted scrubbing attacks. However, the scrubbing attacks in [26] are not as effective as those from LLMs like ChatGPT. If strong spoofing is not achieved, the scrubbing process becomes less efficient and thus unsuitable as a robust scrubbing method. Due to limitations in paper length and computational resources, we primarily used [26] as a spoofing test method in the earlier version of our paper. To address any potential for confusion, we will revise the manuscript introduction section to articulate the explanation of [26] and our core motivation more clearly.\n\n> Q1&W2: What parameter choices are obviously bad.\n> \n\nThank you for your valuable feedback. Here, we present our design philosophy for parameter guidance.\n\nSpecifically, the difficulty of spoofing attacks increases exponentially with `h`, while erasure robustness is positively correlated with the expected collision rate (as detailed in Proposition 4.1/4.2). Our core design principle is: use a sufficiently large `h` to resist spoofing attacks, while adjusting `d` to achieve the desired hash collision probability, thereby enhancing erasure robustness.\n\nThe insight you raised in W2 is indeed correct.  As we mentioned in the paper, UNIGRAM is an extreme case of our method. When the difference between d and h becomes too large(like `d >> h`), SEEK will degenerate into UNIGRAM. If an attacker has access to the actual parameters of the watermarking scheme, they could exploit the degenerated watermark in the same way as attacking UNIGRAM. As with any method, it is important to avoid arbitrary hyperparameter choices. Therefore, we explicitly advise against such extreme configurations.\n\nIn summary, we offer the following recommendations：\n\n1. **Avoid Extreme Parameter Configurations**: We do not recommend using excessively disparate `h` and `d` values, as this causes the SEEK design to degrade to `h=0`, losing its advantage in balancing both robustness types. A `d ≈ h` configuration is generally a more balanced starting point.\n2. **Prefer Larger `h` Values**: A larger watermark window `h` exponentially increases the difficulty of statistical spoofing attacks. We recommend deploying the watermarking scheme with a larger `h` to significantly raise the cost for potential attackers. Additionally, when reproducing a statistical attack with `h > 8` in the experiment, the number of required probability vectors exceeds 180 million, surpassing the cache capacity of our current hardware.\n\nWe conducted additional experiments and employed the LLAMA2-7B model to generate text sequences of at least 200 tokens on C4-Eval, applied a scrubbing attack using GPT (TP@1%), and conducted a spoofing attack using statistical methods (FPR@0.1%).\n\n| **Attack/Param.** | **h=6,d=4** | **h=6,d=6** | **h=6,d=8** | **h=8,d=6** | **h=8,d=8** | **h=8,d=10** |\n| --- | --- | --- | --- | --- | --- | --- |\n| Scrubbing（**TP@1%**） | 85.9 | 84.3 | 81.4 | 85.7 | 83.9 | 80.7 |\n| Spoofing（**FPR@0.1%**） | 0.14 | 0.13 | 0.16 | 0.08 | 0.10 | 0.08 |\n\nWhile the principles above provide a framework for tuning, deriving a single parameter set that is provably optimal across all datasets, domains, and evolving attack models is a formidable theoretical challenge.   Future work will include further parameter ablation studies and attempts to mathematically prove the optimal solution under attacks. The parameter configuration of (h=6, d=6) mentioned in the paper is a reasonable experimental setting based on this principle. The table below demonstrates its generalizability across various datasets.\n\n| Dataset | MMW | PubMed | FreeLAW | Financial |\n| --- | --- | --- | --- | --- |\n| KGW-Min | 73.5/0.95 | 80.1/0.83 | 75.8/0.77 | 83.5/0.70 |\n| SEEK | 80.5/0.18 | 83.5/0.10 | 80.1/0.13 | 85.7/0.18 |\n\nWe hope these additional experiments and clarifications address your concern.\n\n> Q2：FPR Metric in Figure 2.\n> \n\nThank you for your question. In Figure 2, the vertical axis uses FPR@1e-3 to represent the success rate of the spoofing attack. We are uncertain where the value \"1e-2\" you refer to appears. Could you please give more information?\n\n> Q3: Other Writing Improvements.\n> \n\nThank you for your feedback. We will make the following revision to the paper:\n\n- We will change \"1e-2\" to \"1%\" in the next version.\n- The spoofing robustness gains were calculated by comparing SEEK against KGW-MIN (h=4) under statistics-based spoofing attacks, using the FPR@0.1% metric. We acknowledge that the basis for the scrubbing robustness gains was not clearly specified. To rectify this and improve clarity, we will revise this claim in the final manuscript. The gain will be defined as the improvement of SEEK over KGW-MIN (h=4) on the TP@1% metric under the DIPPER-I attack. Based on Table 2, this will update the abstract to: `\"achieving scrubbing robustness gains of +10.8% on WikiText, +13.4% on C4, and +8.6% on LFQA, respectively.\"`\n\nWe recognize that there are some inaccuracies and errors in the article, which will be corrected in subsequent editions. Thank you for guiding us toward a clearer and more polished presentation!"}, {"review_id": "VP2tlM4WwA", "reviewer": "Reviewer_kvdR", "summary": "The paper introduces a novel watermarking algorithm designed to be robust against both spoofing and scrubbing attacks. The core idea leverages insights: increasing the context window enhances resistance against spoofing attacks, while reducing the cardinality of the hash space used to generate the watermark key enhances robustness against scrubbing attacks. Consequently, the proposed algorithm, SEEK, minimizes the output space of the hashing function while maintaining a large context window size for watermark key construction. Additionally, SEEK adapts the method for partitioning vocabulary into green and red sets based on this refined watermark key construction. Extensive evaluations demonstrate SEEK’s robustness against both spoofing and scrubbing attacks.", "questions": "1. In Algorithm 1, Line 5, how is the vocabulary partitioned into sub-vocabularies? Does the partition depend solely on \\$\\zeta\\$? Please explicitly clarify this in Line 221 and Algorithm 1, Line 5.\n\n2. The paper claims, \"reducing the hash space size does not influence the watermark window size, thereby maintaining similar spoofing resistance.\" However, there seems to be an implicit relationship between the window size (\\$h\\$) and the cardinality of the hash space (\\$d\\$). My question: With a fixed hash space cardinality (\\$d\\$), isn't there a limit to how large or small the window size (\\$h\\$) can practically be?\n   1. If \\$h\\$ is significantly larger than \\$d\\$, it becomes likely that all hash values appear within the window, making SEEK behave like a unigram watermark (assuming sub-vocabularies partitioned solely by \\$\\zeta\\$). For example, with \\$d=6\\$ (as in the paper), if \\$h=30\\$, the probability of encountering all hash values is already greater than 0.96. Could you verify and discuss this explicitly?\n   2. Conversely, if \\$h\\$ is much smaller than \\$d\\$, collisions become less likely (as indicated by Proposition 4.1), potentially decreasing robustness against scrubbing attacks.\n   Given these points, isn't there still an explicit trade-off between robustness against spoofing and scrubbing attacks by varying \\$h\\$ for a fixed \\$d\\$? If this reasoning is correct, please explicitly state it in the paper and provide evaluations of SEEK against spoofing and scrubbing attacks for different combinations of \\$h\\$ and \\$d\\$.\n\n3. There are significant concerns about the text quality evaluation:\n\n   1. The P-SP scores reported in Figure 6 are very low (<0.4). Previous literature (e.g., \\[29]) indicates that a score above \\~0.7 is necessary to maintain linguistic fidelity. Thus, current results do not support the claim of preserving linguistic fidelity. Clarify or provide new evidence supporting your claims.\n   2. The paper uses OPT-2.7B as the oracle model to evaluate LLaMA2-13B-generated text. Typically, the oracle model should have higher quality than the target model (e.g., \\[28], \\[31]). Using OPT-2.7B might bias results unfairly against LLaMA2-13B. Clarify this and justify the oracle model choice.\n   Given this, Figure 6, as presented, inadequately demonstrates SEEK’s true impact on text quality compared to KGW. Proper and additional evaluations or metrics are necessary to substantiate your claims clearly.\n\n4.  Although SEEK aims to increase vocabulary diversity, log-diversity metrics are missing. Include these results explicitly.\n\n5. The paper mainly evaluates the case where \\$\\delta=5.0\\$ and \\$\\gamma=0.25\\$. These settings inherently cause significant performance drops (as indicated in Figure 6) and simplify spoofing attacks. Include evaluations using more practical values of \\$\\delta\\$ that maintain acceptable text quality.\n\n6. What is the time complexity of text generation for SEEK compared to KGW? How does complexity scale with increased vocabulary size, given the necessity of partitioning vocabularies and assigning different keys? Provide empirical timing results explicitly for SEEK and KGW.", "limitations": "yes", "rating": 5, "confidence": 4, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": 3, "clarity": 2, "significance": 2, "originality": 4, "strengths_and_weaknesses": "## Strengths\n1. The algorithm effectively reduces the trade-off between robustness against spoofing attacks and scrubbing attacks, which is a common limitation in existing watermarking algorithms.\n2. The paper is clearly written and easy to follow. Figure 4, in particular, provides an excellent overview of the differences between SEEK and KGW.\n3. The motivation behind the algorithm, specifically how the window size and hash space cardinality impact watermark robustness against spoofing and scrubbing attacks, is clearly articulated. This analysis highlights the potential reasons for the observed trade-offs.\n\n## Weaknesses\n1. The evaluation of text quality has significant limitations, resulting in an unclear assessment of SEEK's actual impact on generated text quality. Additional experimental results are required to conclusively demonstrate that SEEK-generated text remains usable and maintains high quality. \n2. The paper claims reduced hash space cardinality improves robustness against scrubbing attacks without affecting spoofing resistance. However, an implicit trade-off exists and should be explicitly analyzed.\n3. SEEK may incur increased computational overhead during text generation, potentially impacting its practical application, particularly with larger vocabulary sets. This concern should be explicitly addressed in the evaluation to clarify the practical limitations.", "was_revised": true, "final_justification": "1. The paper initially misrepresented the trade-off that SEEK addresses. SEEK cannot be simultaneously highly robust against both spoofing and scrubbing attacks—there is an inherent trade-off. The authors acknowledged this during the rebuttal and have committed to revising the relevant parts of the paper.\n\n2. There were some issues with the text quality evaluation setup. The authors have provided new results and committed to updating the paper accordingly.\n\nWith these changes, I believe my main concerns have been addressed.", "rebuttal": "# **To Reviewer kvdR**\n\nWe sincerely thank you for your detailed and constructive feedback. Below, we address your questions and weaknesses.\n\n> Q1: How is the vocabulary partitioned into sub-vocabularies.\n> \n\nSorry for the confusion. The sub-vocabulary partitioning depends on the key `ξ`, ensuring that, without the key, an attacker cannot predict the sub-vocabulary of any given token. To avoid potential ambiguity, we will provide further clarification in both the main text and the algorithm in a later version.\n\n> W2&Q2: An implicit trade-off exists and should be explicitly analyzed.\n> \n\nThank you for pointing this out. We would first like to clarify that the trade-off between robustness to scrubbing and spoofing in our work reflects movement along a Pareto frontier.\n\nWe fully acknowledge that such a trade-off also exists within SEEK, and we do not claim otherwise in our paper. However, we would like to re-emphasize our key contribution: SEEK **breaks the existing trade-offs observed in the KGW-family,** which are widely believed to be governed by window size, and **expands the boundary of the previous Pareto frontier.**\n\nNotably, SEEK with d=6,h=6 achieves strong performance on both fronts (about 0.98 AUROC against ChatGPT-scrubbing and about 0.1 FPR@0.1% against spoofing), which demonstrates that it is practically viable for real-world deployment scenarios. \n\nAs with any method, it is important to avoid arbitrary hyperparameter choices—in SEEK’s case, the parameters d and h play a critical role. The difficulty of spoofing attacks increases exponentially with `h`, while scrubbing robustness is positively correlated with the expected collision rate (as detailed in Proposition 4.1/4.2). Our core design principle is: use a sufficiently large `h` to resist spoofing attacks, while adjusting `d` to achieve the desired hash collision probability, thereby enhancing scrubbing robustness.\n\nThe perspective you raised in Q2-1 is correct. As we mentioned in the paper, UNIGRAM is an extreme case of our method. When the difference between d and h becomes too large (like `d << h` or `d >> h`), SEEK will degenerate into UNIGRAM. Therefore, we explicitly advise against such extreme configurations.\n\nIn practice, we generally offer the following recommendations for parameter settings:\n\n1. **Avoid Extreme Parameter Configurations**: We do not recommend using excessively disparate `h` and `d` values, as this causes the SEEK design to degrade to `h=0`, losing its advantage in balancing both robustness types. A `d ≈ h` configuration is generally a more balanced starting point.\n2. **Prefer Larger `h` Values**: A larger watermark window `h` exponentially increases the difficulty of statistical spoofing attacks. We recommend deploying the watermarking scheme with a larger `h` to significantly raise the cost for potential attackers. \n\nWe conducted additional experiments and employed the LLAMA2-7B model to generate text sequences of at least 200 tokens on C4-Eval, applied a scrubbing attack using GPT (TP@1%), and conducted a spoofing attack using statistical methods (FPR@0.1%).\n\n| **Attack/Param.** | **h=6,d=4** | **h=6,d=6** | **h=6,d=8** | **h=8,d=6** | **h=8,d=8** | **h=8,d=10** |\n| --- | --- | --- | --- | --- | --- | --- |\n| Scrubbing（**TP@1%**） | 85.9 | 84.3 | 81.4 | 85.7 | 83.9 | 80.7 |\n| Spoofing（**FPR@0.1%**） | 0.14 | 0.13 | 0.16 | 0.08 | 0.10 | 0.08 |\n\nThe default parameter configuration of (h=6, d=6) mentioned in the paper is a reasonable experimental setting based on this principle. The table below demonstrates its generalizability across various datasets.\n\n| Dataset | MMW | PubMed | FreeLAW | Financial |\n| --- | --- | --- | --- | --- |\n| KGW-Min | 73.5/0.95 | 80.1/0.83 | 75.8/0.77 | 83.5/0.70 |\n| SEEK | 80.5/0.18 | 83.5/0.10 | 80.1/0.13 | 85.7/0.18 |\n\nWith the current parameter settings, we achieve a superior Pareto frontier by delivering a much better balance of robustness than the baseline. Future work will include further parameter ablation studies and attempts to mathematically prove the optimal solution under attacks. \n\nWe hope this detailed analysis and the additional experiments have addressed your concerns and clarified the contributions of our work.\n\n> W3&Q6 : Time Cost in Watermarking.\n> \n\nIn watermark text generation, methods modifying logits directly (e.g., SEEK, KGW, SWEET) are highly efficient. And, the primary time cost of watermark deployment **still arises from the forward propagation of the large language model itself**. To validate this, we conducted empirical timing tests in the following environment:\n\n- **Hardware**: NVIDIA A40-48GB\n- **Language Model**: LLAMA2-7B\n- **Configuration**: batch_size=1, temperature=1.0, torch_seed=42, num_beams=1, gamma=0.25\n- **Data**: Averaged over 500 prompts from the C4 dataset\n\n| **Method** | **KGW-Min** | **SEEK(d=6)** | **SEEK(d=8)** | **SEEK(d=10)** | **SEEK(d=16)** |\n| --- | --- | --- | --- | --- | --- |\n| Generation | 16.6561s | 16.8245s | 16.7202s | 16.7445s | 16.7855s |\n| Detection | 44.13ms | 43.38ms | 45.54ms | 46.66ms | 49.26ms |\n\n> Q3-1: Clarification on P-SP Scores.\n> \n\nWe sincerely appreciate your insightful question regarding the P-SP scores. We would like to clarify a key distinction regarding the context in which the P-SP scores are evaluated.\n\nIn the referenced work [29], the threshold \"P-SP > 0.7\" is applied in their work to evaluate the quality of human scrubbing attacks. Specifically, it compares `(watermarked text)` vs. `(human-scrubbed text)` to select more meaning-preserving attack samples, ensuring that the attack is high-precision.\n\nIn contrast, our experiment in Figure 6 is designed to measure whether there is a significant gap in the semantic deviation introduced by different watermarking schemes, which we assess by comparing `(watermarked text)` vs. `(unwatermarked text)`.\nIn fact, the study in [29] also conducts a similar experiment, as shown in **Figure 11** [29], comparing P-SP scores between watermarked and non-watermarked texts. The results indicate that the P-SP scores are similarly **well below 0.7 (typically <0.5)**, and decrease further with increasing watermark strength (z-score).  Additionally, in Section A5 of [29], it is stated that **\"We note that the drift between human text and unwatermarked text can be estimated as just under 0.50 P-SP”**. Even considering high-quality human-generated text, the P-SP score between such text and unwatermarked text is still below 0.7. Therefore, it is reasonable that watermarked texts cannot reach the threshold, and the P-SP scores reported in Figure 6 of our paper fall within a reasonable range.\n\n> W1&Q3-2: Concerns about the text quality evaluation.\n> \n\nOur choice of OPT-2.7B as the oracle model follows the experimental setup established in seminal prior work, such as KGW [28] (see Section 6) and WaterMax [16]. As noted in [16], “while absolute perplexity values may differ depending on the oracle model, the *relative performance* between different watermarking algorithms remains consistent. Using a common oracle like OPT-2.7B is therefore recommended to ensure comparability with state-of-the-art methods”.\n\nTo fully address your concern and provide even stronger evidence, we have included a new evaluation within our generalizability study that we evaluated our method against the KGW-Min using the LLAMA2-7B model to systematically assess PPL across different domains (using the base model as its own oracle provides the most direct measure of distributional shift).   As shown in the table below:\n\n| **Dataset** | **Task** | **KGW-Min (delta=2)** | **SEEK (delta=2)** | **KGW-Min (delta=4)** | **SEEK (delta=4)** |\n| --- | --- | --- | --- | --- | --- |\n| WikiText-103 | General | 6.21 | 5.88 | 13.78 | 13.33 |\n| MMW | News | 8.72 | 8.53 | 18.24 | 15.75 |\n| C4 | Long-form Text | 7.44 | 8.38 | 19.81 | 16.62 |\n| PubMed | Biomedical | 8.44 | 7.54 | 17.02 | 16.71 |\n| FreeLAW | Legal | 7.55 | 6.53 | 17.25 | 14.98 |\n| Financial-News | Finance | 6.33 | 6.27 | 18.29 | 15.63 |\n\nIn response to your concern, we also conducted experiments on well-established low-entropy text datasets like code generation tasks (HumanEval, MBPP), using the StarCoder2-7B model. The results are summarized below:\n\n| **Dataset** | **(No-watermark)** | **KGW-Min** | **SEEK** |\n| --- | --- | --- | --- |\n| HumanEval | **Pass@1**: 33.4； / | **Pass@1**: 27.4; **AUROC** 88.1 | **Pass@1:** 28.1; **AUROC:** 89.3 |\n| MBPP | **Pass@1**: 37.5；/ | **Pass@1**: 33.9; **AUROC:** 80.3 | **Pass@1:** 34.7; **AUROC:** 82.7 |\n\nBy consistently outperforming or matching the baseline across the vast majority of domains, these results provide strong empirical evidence that our approach generalizes effectively without compromising semantic coherence.\n\n> W4: Diversity Metric in SEEK.\n> \n\nThank you for your question. The diversity metric for SEEK is already included in Figure 3 of the paper.\n\n> W5: Delta Parameter Settings.\n> \n\nWe selected a larger value for δ in line with the parameter settings used by [26] for more challenging spoofing attacks on the victim’s watermark model M. As you correctly noted, a larger δ generates a stronger watermark signal, which creates a more favorable condition for the attacker, who can then learn the watermark pattern with fewer queries. We adopted this setting, which means our defense (the SEEK method) was evaluated under a more challenging and stringent scenario.\n\nWe agree that such a delta might not be the first choice in a practical deployment where text quality is the priority. However, it still ensures fairness in experimental comparisons as long as delta is kept consistent across all methods. And all experimental conclusions drawn remain valid.\n\nAs mentioned in our previous responses, we have already provided evaluations of text quality (PPL) for the more practical settings of δ=2 and δ=4.\n\n***If our responses address your concerns, we would be grateful if you would consider raising your final rating to a higher score.***"}, {"review_id": "2GbZoGdDVJ", "reviewer": "Reviewer_CfYA", "summary": "The paper addresses the vulnerability of watermarking techniques used to protect large language models (LLMs) from misuse. Traditional watermarking methods face challenges from scrubbing and spoofing attacks, which can undermine their effectiveness. The authors introduce a novel mechanism called equivalent texture keys, which allows multiple tokens within a watermark window to independently support watermark detection. This innovation leads to a new watermarking scheme called Sub-vocabulary decomposed Equivalent tExture Key (SEEK).\n\nIn summary, the paper's contributions include:\n- Introducing the equivalent texture key mechanism to enhance watermark robustness.\n- Proposing SEEK, a new watermarking scheme that balances scrubbing and spoofing defenses while maintaining text quality.\n- Demonstrating through experiments that SEEK offers a Pareto improvement over prior watermarking schemes .", "questions": "- While Figure 2 shows spoofing robustness improves with larger windows for KGW, Table 2 reports SEEK results only for (h=6, d=6) and (h=16, d=16). How does SEEK's scrubbing/spoofing trade-off specifically vary with window size?\n- Table 6 tests SEEK on 7B–13B models, showing consistent gains. Is SEEK's improvement consistent across scales (e.g., <1B or >30B models)?\n- The paper only compares against KGW-family methods. Yet recent works like UNIGRAM (fixed green list), SemStamp (semantic hashing), and CATER (contextual watermarks) also address spoofing/scrubbing trade-offs. Do you compare with these methods?", "limitations": "yes", "rating": 5, "confidence": 4, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": 3, "clarity": 3, "significance": 3, "originality": 3, "strengths_and_weaknesses": "Strengths\n- Proposes a novel mechanism (equivalent texture keys) and the SEEK scheme to overcome the inherent scrubbing-spoofing robustness trade-off in LLM watermarking. Achieves Pareto improvement by enabling large watermark windows (spoofing resistance) while maintaining scrubbing robustness via hash-space optimization.\n- Demonstrates state-of-the-art results across 5+ datasets (WikiText, C4, Dolly-CW, MMW):\n   - +82–92% spoofing robustness (FPR reduction)\n   - +6–24% scrubbing robustness (TP@1% gain)\n   - Maintains text quality (perplexity/log-diversity ≈ baselines).\n- Formally proves key properties:Expected erased tokens for watermark removal increases with smaller hash space (Prop 4.2), Hash collision probability bounds (Prop 4.1), Robustness guarantees under adversarial modifications (Appendix D).\n\nWeakness\n\n- While the paper shows strong performance across various datasets, it could benefit from a more detailed discussion on how the proposed method generalizes to different types of language models and real-world scenarios. Additional experiments on diverse models and tasks could strengthen this aspect.\n- SEEK’s sub-vocabulary decomposition assumes uniform token distribution. Performance may degrade for low-entropy/specialized domains (e.g., medical/legal jargon) where sub-vocabulary imbalance could affect watermark stability.\n- Experiments only compare KGW variants (e.g., KGW-Min/Left/Sum). Lacks comparison with other methods, eg.  SIR[1]\n- This paper lacks ablation study. No component-wise analysis to disentangle contributions of: Equivalent texture keys (hash-space reduction), Sub-vocabulary decomposition, Obscures whether both innovations are essential for Pareto gains.\n\n[1] Liu, Aiwei, et al. \"A semantic invariant robust watermark for large language models, 2024.\" URL https://arxiv. org/abs/2310.06356.", "was_revised": true, "final_justification": "This paper features a novel token-level watermarking method with clear theory and rich experiments, showing solid performance against attacks. The rebuttal effectively addressed baseline selection rationale and added supplementary experiments on domain generalization and model scalability, resolving all key concerns. No significant unresolved issues remain.", "rebuttal": "# **To Reviewer CfYA**\n\n> W1&W2: Performance may degrade for low-entropy/specialized domains\n> \n\nWe greatly appreciate the reviewer's insightful question regarding the structural assumptions and generalizability. To systematically assess the impact of this design on text generation quality across different domains (especially specialized fields), we evaluated our method against the KGW-Min baseline using the LLAMA2-7B model. The evaluation was performed on various datasets, including general, news, biomedical, legal, and financial texts. We compared the PPL (using LLAMA2-7B as the oracle model, which represents the original unwatermarked generation logic), as shown in the table below:\n\n| **Dataset** | **Task** | **KGW-Min (delta=2)** | **SEEK (delta=2)** | **KGW-Min (delta=4)** | **SEEK (delta=4)** |\n| --- | --- | --- | --- | --- | --- |\n| WikiText-103 | General | 6.21 | 5.88 | 13.78 | 13.33 |\n| MMW | News | 8.72 | 8.53 | 18.24 | 15.75 |\n| C4 | Long-form Text | 7.44 | 8.38 | 19.81 | 16.62 |\n| PubMed | Biomedical | 8.44 | 7.54 | 17.02 | 16.71 |\n| FreeLAW | Legal | 7.55 | 6.53 | 17.25 | 14.98 |\n| Financial-News | Finance | 6.33 | 6.27 | 18.29 | 15.63 |\n\nIn response to the reviewer’s concern, we also conducted experiments on well-established low-entropy text datasets, specifically for code generation tasks (HumanEval, MBPP), using the StarCoder2-7B model. The results are summarized below:\n\n| **Dataset** | **(No-watermark)** | **KGW-Min** | **SEEK** |\n| --- | --- | --- | --- |\n| HumanEval | **Pass@1**: 33.4； / | **Pass@1**: 27.4; **AUROC** 88.1 | **Pass@1:** 28.1; **AUROC:** 89.3 |\n| MBPP | **Pass@1**: 37.5；/ | **Pass@1**: 33.9; **AUROC:** 80.3 | **Pass@1:** 34.7; **AUROC:** 82.7 |\n\nBy consistently outperforming or matching the baseline across the vast majority of domains, these results provide strong empirical evidence that our approach generalizes effectively without compromising semantic coherence.\n\n> Q1: How robustness trade-off varies with window size.\n> \n\nWe thank the reviewer for this important and practical question. Below, we present our parameter discussion on how the SEEK trade-off varies with window size.\n\nSpecifically, the difficulty of spoofing attacks increases exponentially with `h`, while erasure robustness is positively correlated with the expected collision rate (as detailed in Proposition 4.1/4.2). Our core design principle is as follows: use a sufficiently large `h` to resist spoofing attacks, while adjusting `d` to achieve the desired hash collision probability, thereby enhancing erasure robustness.\n\nOur core design principle is as follows: use a sufficiently large `h` to resist spoofing attacks, while adjusting `d` to achieve the desired hash collision probability, thereby enhancing erasure robustness. Specifically, the difficulty of spoofing attacks increases exponentially with `h`, while erasure robustness is positively correlated with the expected collision rate (as detailed in Proposition 4.1/4.2). Since the UNIGRAM scheme represents an extreme case of our method,  we explicitly advise against such extreme configurations. The paper configuration of `(h=6, d=6)` represents a choice based on this principle.\n\nFirst, the primary role of increasing the window size `h` is to **exponentially increase the statistical complexity for spoofing attacks**. A larger `h` forces an attacker to learn a vastly larger mapping from context to output, dramatically raising the cost of an attack. As the table below illustrates, the number of probability vectors required to reproduce a statistical attack grows explosively with `h`. When reproducing a statistical attack with `h > 8` in the experiment, the number of required probability vectors exceeds 180 million, which exceeds the cache capacity of our current hardware.\n\n| **Windows size** | **h=3** | **h=4** | **h=5** | **h=6** | **h=8** |\n| --- | --- | --- | --- | --- | --- |\n| Num. of probability vectors | 7M | 12M | 36M | 66M | 180M |\n\nAdditionally, we conducted experiments and employed the LLAMA2-7B model to generate text sequences on C4-Eval, applied a scrubbing attack using GPT (TP@1%), and conducted a spoofing attack using statistical methods (FPR@0.1%).\n\nWe conducted additional experiments and employed the LLAMA2-7B model to generate text sequences of at least 200 tokens on C4-Eval, applied a scrubbing attack using GPT (TP@1%), and conducted a spoofing attack using statistical methods (FPR@0.1%).\n\n| **Attack/Param.** | **h=6,d=4** | **h=6,d=6** | **h=6,d=8** | **h=8,d=6** | **h=8,d=8** | **h=8,d=10** |\n| --- | --- | --- | --- | --- | --- | --- |\n| Scrubbing（**TP@1%**） | 85.9 | 84.3 | 81.4 | 85.7 | 83.9 | 80.7 |\n| Spoofing（**FPR@0.1%**） | 0.14 | 0.13 | 0.16 | 0.08 | 0.10 | 0.08 |\n\nFuture work will include further parameter ablation studies and attempts to mathematically prove\n\nThe results provide evidence that the number of probability vectors an attacker must learn expands dramatically with h, rendering large-scale statistical spoofing attacks computationally infeasible.\n\n> Q2：Generalization Across Model\n> \n\nTo empirically validate this, we conducted a new set of experiments across three models of varying scales (0.5B, 7B, and 32B). We evaluated robustness against ChatGPT attacks. The results (AUROC / TP@1% / TP@5% ) are summarized below:\n\n| **Dataset/Model** | **Qwen2.5-0.5B** | **Qwen2.5-7B** | **Qwen2.5-32B** |\n| --- | --- | --- | --- |\n| KGW-Min (h=4) | 90.7/79.5/88.9 | 92.3/83.4/91.4 | 94.7/81.8/90.5 |\n| SEEK | 92.3/83.4/90.6 | 95.7/90.1/94.3 | 95.9/89.1/96.3 |\n\nAs the table demonstrates, SEEK exhibits strong and consistent robustness on both the 0.5B, 7B, and 32B models, confirming its scalability. The slightly lower performance on the 0.5B model is attributed to the lower intrinsic coherence of its generated text. This content is more susceptible to substantial rewriting when subjected to a scrubbing attack by a vastly more powerful model (ChatGPT, >175B), which can more easily disrupt the watermark's structure.\n\n> W3& Q3：Comparison with Related Work.\n> \n\nWe sincerely thank the reviewer for this insightful question regarding the scope of our baselines. Our work is specifically focused on addressing the scrubbing-spoofing trade-off within the dominant paradigm of **token-level watermarking**. We therefore prioritized direct comparisons with the most representative methods in this category, namely the KGW-family. Here we discuss other related excellent works you mentioned:\n\n1. **UNIGRAM**: We agree on UNIGRAM's relevance and have already included it as a baseline in our analysis (see **Figure 1**). UNIGRAM can be considered a special case of KGW (`h=0`). However, it completely sacrifices spoofing resistance for scrubbing robustness and has been shown to be highly vulnerable to in multiple studies [1, 15, 17, 23].\n2. **CATER**: This method represents an earlier traditional technical paradigm based on deterministic, rule-based substitution. It relies on predefined synonym rules and linguistic triggers rather than modifying the model's probabilistic output, as KGW-family methods do. Its primary threat model involves an adversary reverse-engineering and removing these rules, and it does not address spoofing attacks.\n3. **SemStamp/SIR (Sentence-Level Methods)**: Methods like SemStamp belong to another line of research: **sentence-level watermarking**. We did not select them as direct competitors for several key reasons:\n    - **Additional Overhead**: They typically require an external encoder for both generation and detection, introducing significant computational costs.\n    - **Low Statistical Efficiency**: Operating at the sentence level, they require impractically long texts to achieve acceptable detection performance (e.g., SemStamp has an FPR of ~1%), and their z-score p-value approximation is only accurate with a large number of sentences, a condition rarely met in practice.\n    - **Vulnerability to Fragment Attacks**: They are not robust against paragraph-level manipulations like copy-paste attacks (as shown in our Table 8).\n    - The state-of-the-art spoofing attacks [17] used in our evaluation are specifically designed for token-level schemes.\n    \n    Therefore, while the methods you mentioned are all significant contributions, their different technical paradigms and threat models place them outside the scope of our direct comparative analysis. We will expand our Related Work section in the final manuscript to discuss these distinctions more thoroughly and better situate our contribution.\n    \n\n> W4: Ablation Study.\n> \n\nWe sincerely appreciate the reviewer's valuable suggestion. Although ablation analysis is not presented as a standalone section, the core structure of our paper is inherently organized as a step-by-step ablation study. We begin with the standard KGW method and initially introduce the \"equivalent texture key\" mechanism by reducing the hash space. The experimental results in Table 1 and Figure 2 demonstrate the effectiveness of this component in improving the trade-off. Next, we discuss the potential compromise in text generation quality introduced by this component. To address this issue, we introduce the \"sub-vocabulary decomposition,\" which culminates in the final SEEK method. The results in Figure 3 further validate that this component successfully restores text quality.\n\nThank you for guiding us toward a clearer and more polished presentation!"}]

Author rebuttal:

Meta-review: This paper presents a novel watermarking method that effectively balances robustness against both scrubbing and spoofing attacks, a known trade-off in existing techniques. Reviewers found the experiments thorough and the results convincing. My recommendation is to accept the paper. I encourage the author(s) to improve the discussion on parameter selection and broader comparisons with related work in the camera-ready version.
Decision: Accept (spotlight)