Discovering Spoofing Attempts on Language Model Watermarks
Authors: Thibaud Gloaguen
Nikola Jovanović
Robin Staab
Martin Vechev
Venue: ICML 2025 poster (2025)
Keywords: LLM
watermarks
watermark spoofing
Primary area: social_aspects->accountability_transparency_and_interpretability

Abstract: LLM watermarks stand out as a promising way to attribute ownership of LLM-generated text. One threat to watermark credibility comes from spoofing attacks, where an unauthorized third party forges the watermark, enabling it to falsely attribute arbitrary texts to a particular LLM. Despite recent work demonstrating that state-of-the-art schemes are, in fact, vulnerable to spoofing, no prior work has focused on post-hoc methods to discover spoofing attempts. In this work, we for the first time propose a reliable statistical method to distinguish spoofed from genuinely watermarked text, suggesting that current spoofing attacks are less effective than previously thought. In particular, we show that regardless of their underlying approach, all current learning-based spoofing methods consistently leave observable artifacts in spoofed texts, indicative of watermark forgery. We build upon these findings to propose rigorous statistical tests that reliably reveal the presence of such artifacts and thus demonstrate that a watermark has been spoofed. Our experimental evaluation shows high test power across all learning-based spoofing methods, providing insights into their fundamental limitations and suggesting a way to mitigate this threat.

Discovering Spoofing Attempts on Language Model Watermarks
Thibaud Gloaguen' Nikola Jovanovié
Robin Staab Martin Vechev
Abstract
tipudi et al., 2024; Christ et al., 2024b). As such watermarks are actively deployed top of consumer LLMs (Dathathri et al., 2024a) and widely embraced by regulators (Biden, 2023; CEU, 2024), ensuring their reliability is crucial.
LLM watermarks stand out as promising way to attribute ownership of LLM-generated text. One threat to watermark credibility comes from spoof- ing attacks, where an unauthorized third party forges the watermark, enabling it to falsely at- tribute arbitrary texts particular LLM. De- spite recent work demonstrating that state-of-the- art schemes are, in fact, vulnerable to spoofing, no prior work has focused on post-hoc methods to discover spoofing attempts. In this work, we for the first time propose a reliable statistical method distinguish spoofed from genuinely wa- termarked text, suggesting that current spoofing attacks are less effective than previously thought. In particular, we show that regardless of their underlying approach, all current learning-bas -based spoofing methods consistently leave observable artifacts in spoofed texts, indicative of watermark forgery. We build upon these findings propose rigorous statistical tests that reliably reveal the presence of such artifacts and thus demonstrate that watermark has been spoofed. Our experi- mental evaluation shows high test power across all learning-based spoofing methods, providing insights into their fundamental limitations and suggesting a way to mitigate this threat. We make all our code available here.
LLM watermarks To embed a signal, at each step of generation, using private key the watermark algorithm scores each token, preferentially sampling higher-scoring ones. While a wide range of watermarking schemes have been proposed (Christ et al., 2024b; Kuditipudi et al., 2024; Aaronson, 2023), the most studied and at this time the only ones deployed in prominent consumer LLMs (Dathathri et al., 2024a) are from the Red-Green (Kirchenbauer et al., 2023) family. In Red-Green watermarks, the algorithm uses & and a few previous tokens (context) to partition the vocabulary into green and red tokens. It then increases the probability of sampling green tokens. Given a text, the watermark detector first computes the color of each token under &, wherein high proportion of green tokens in this color sequence indicates watermarked text.
Spoofing attacks Recent works have demonstrated tar- geted attacks on Red-Greed watermarks that allow for imper- sonating (spoofing) the watermark (Sadasivan al., 2023; Jovanović et al., 2024; Gu et al., 2024: Zhang et al., 2024). In spoofing attacks, a malicious actor (spoofer) generates, without knowing the private key &, a text that is detected as watermarked. State-of-the-arr attacks are learning-basee and adhere to a common pipeline (see App. for a full taxonomy). First, the malicious actor queries the targeted model to build dataset D of genuinely watermarked text. Then, by either applying statistical methods (Jovanović et al., 2024), integer programming (Zhang et al., 2024), or fine- tuning (Gu et al., 2024), the spoofer learns how to forge the watermark and can generate watermarked text without additional queries to the original model (Step 1 in Figure 1).
1. Introduction
The abilities of large language models (LLMs) to generate human-like text at scale (Bubeck et al., 2023; Dubey et al., 2024) come with growing risk of potential misuse. This makes reliable detection of machine-generated text increas- ingly important. Researchers have proposed the concept of watermarking: augmenting generated text with an impercep- tible signal that can later be detected attribute ownership of specific LLM (Kirchenbauer et al., 2023; Kudi-
Being able to generate spoofed text at scale poses a serious threat to the credibility of watermarks. Spoofed text can be falsely attributed to the model provider, causing reputational damage, used as argument to evade accountability (Zhou et al., 2024). Moreover, in the case of multi-bit watermarks that embed client IDs in generated text (Wang et al., 2024). spoofing attacks can be used to impersonate and incriminate specific user.
'ETH Zurich. Correspondence to: Thibaud Gloaguen <tgloaguen@ student.ethz.ch..
Proceedings of the 4nnd International Conference Machine Learning, Vancouver, Canada. PMLR 267, 2025. Copyright 2025 by the author(s).

---

Discovering Spoofing Attempts Language Model Watermarks
Leader
followers
Watermarked model with &
E
V Genuine
...charismatic leader and obdurate followers...
Query
charismatic
obdurate
D
...charismatic leader and obdurate
Zg =
W Spoofed
Distillation, Stealing
charismatic
obdurate E
..charismaaicc leader and obdurate resistance..
E E
D
Spoofer
D
Leader
resistance
(1) Learning
(2) Generating
(3) Spoofing artifacts
Figure 1: Overview of why spoofed text contains measurable artifacts. First, in (1), the spoofer generates a dataset 'D of g-watermarked texts from which they learn the watermark. As (2) illustrates, when later generating text, the spoofer is better at sampling a green token if (and only if) the context and the sampled token were D. This uncertainty introduces artifacts in the spoofed text. In contrast, the genuine watermarking algorithm consistent with respect to the context and hence contains no such artifacts. Lastly, in (3), we build statistical tests for discovery of these artifacts, distinguishing between spoofed and ç w-atermarkedd texts even if their Z-scores Zg computed using the watermark detector are the same.
Discovering spoofing attempts In this work, we show for the first time that sfate-of-the-ard spoofing attacks leave artifacts in the generated text that can be used to distinguish between spoofed text and text generated with knowledge of the private key (Step 2 in Figure 1). This suggests that, unlike previously thought, simply fooling the watermark detector is not enough to generate text that is indistinguish- able from genuine watermarked text. The high-level intu- ition behind these artifacts that, at each step of generation, spoofer has a chance emit green token only if the context and that token present in their training data D, previously obtained by querying the watermarked model. If the context is not in D, the spoofer is forced to select the next token independently of its color. Leveraging these artifacts, we construct statistical tests that can effectively distinguish between spoofed text and genuine watermarked text generated with the private key (Step 3 in Figure I).
2. Background and Related Work
In this section we introduce the necessary background on LLM watermarking, and discuss related work.
LLM watermarks Given sequence of tokens (text) from vocabulary Σ, an autoregressive language model (LM) M outputs a logit vector I of unnormalized next-token proba- bilities, used to sample the next token. LM watermarking is a process of embedding a signal within the generated text w using a private key ç. such that this signal is later detectable by any party with access to & using a watermark detector De. We set De(w)= = the signal is detected. We call text generated by the watermarking algorithm E-watermarked text, and text where Dg
watermarked text.
The most prominent approach to LLM watermarking are Red-Green watermarks (Kirchenbauer et al., 2024; Zhao et al., 2024; Lee et al., 2024; Wu et al., 2024; Yoo et al.,
In addition to enabling the discovery of spoofing attempts on widely researched, deployed, and attacked Red-Green watermarking schemes, we show in App. G that our tests generalize to all schemes that are vulnerable to learning- based spoofing (Aaronson, 2023; Kuditipudi et al., 2024).
2024; Fernandez et al., 2023; Liu et al., 2023; Fairoze et al., 2023; Ren al., 2024; Lu et al., 2024; Guan et al., 2024; Zhou al., 2024; Dathathri et al., 2024a), which all share common structure. Let Wt E Σ be the token generated by the LM at step f, h E N the watermark's 's context size (we refer to / previous tokens wi-hit- as the context), & e N the watermark's 's private key, H Σ -
Key contributions Our main contributions are:
N a hash function, x N + P(E) pseudorandom function, and 7, 8 E R watermark parameters. At each step t. PRF uses the hash of the context H (wi-A:t-1) and the private key & to partition the vocabulary Σ into two colors, >|Σ| green tokens (greenlist) and the remaining red tokens (redlist), where is the watermark parameter. To insert the water- mark, we modify the logit vector 4, by increasing the logit of each green token by 8 > O. While many hash functions have been proposed (Kirchenbauer al., 2024), we fo-
We provide the first in-depth analysis of artifacts in spoofed text, highlighting common limitations of learning-based watermark spoofing methods ($3).
PRF
We design rigorous statistical tests to practically distin- guish spoofed and genuine watermarked texts ($4).
We provide extensive validation of our test hypotheses and empirically show that our tests achieve arbitrarily high power given a long enough text ($5).
2

---

Discovering Spoofing Attempts on Language Model Watermarks
cus on two variants proposed in Kirchenbauer et al. (2023): SumHash and SelfHash. The shift by 5 increases the ratio of green tokens in generated text, which is detectable by the de- tector. Namely. given a text w E Σ", the watermark detector De determines the number of green tokens Vgreen and com- /T<(1 7), which under the null hypothesis follows a standard normal distribution. Finally, (2(w)) = 1 (i.e., w is considered watermarked) if Ze(w)= > p. As in Kirchenbauer et al. (2023), we set p = 4.
watermark properties (Jovanović al., 2024; Kirchenbauer et al., 2024), preventing their practical adoption (Dathathri et al., 2024b). Similarly, even in Red-Green schemes, higher values of the context length h reduce the success rate of spoofing at the cost of watermark robustness.
putes Ze (ww (Agreen
Broader work on LLM watermarking Other directions in the realm of LLM watermarking includes scrubbing at- tacks (Jovanović et al., 2024; Wu & Chandrasekaran, 2024; Chang et al., 2024), detection of the presence of watermark (Tang et al., 2023; Gloaguen et al., 2025; Liu et al., 2025; Gao et al., 2025), and attempts to imprint the watermark into the model weights (Li et al., 2024; Christ et al., 2024a).
Other alternative approaches to LLM watermarks are pro- posed by, among else, Christ et al. (2024b); Kuditipudi et al. (2024); Hu et al. (2024); Aaronson (2023). Among these, prior work demonstrates learning-based spoofing attacks on Kuditipudi et al. (2024) and Aaronson (2023) (see App. G).
3. Can Spoofing Attempts Be Discovered?
LLM watermark spoofing A threat to watermark cred- ibility are spoofing attacks, as they can lead to falsely at- tributing text ownership to a model provider. One type of spoofing attack is piggyback spoofing (Pang et al., 2024), where an attacker substitutes a few tokens in genuinely watermarked text to produce a spoofed text, simply lever- aging the robustness of the watermarking scheme. Another type of spoofing attacks is step-by-step spoofing (Pang et al., 2024; Zhou et al., 2024; Wu & Chandrasekaran, 2024), where for every spoofed text, the attacker queries the water- marked model at each step of its generation process. Lastly, state-of-the-art spoofing attacks are learning-basee spoof- ing attacks (Jovanović et al., 2024; Gu et al., 2024; Zhang et al., 2024), where an attacker first queries the watermarked model to build a watermarked dataset DD and then learns the watermark from such dataset. Unlike the two other spoofing techniques. learning-based based spoofers are able to produce arbitrary watermarked text at a low cost without relying on the attacked model during generation. We extend the discussion on watermark spoofing in App. J.
In this section, we discuss the discoverability of spoofing, introduce the problem of distinguishing &watermarred and spoofed texts, and formalize it within a hypothesis testing framework ($3.1). We describe the intuition behind our approach ($3.2), that we later present in detail in $4.
3.1. Problem statement
Current spoofing methods (spoofers) are typically evaluated based on their success rate in generating high-quality wa- termarked text. Yet, due to the limitation of lcarning from a finite dataset of watermarked text, we hypothesize that these spoofers, despite adopting fundamentally different approaches, may all leave similar artifacts in spoofed texts.
Showing the existence of such artifacts would give valuable insight into the shared limitations of current state-of-the-art watermark spoofers. Morcover, reliably identifying them would enable us to distinguish between f-watermarked and spoofed texts. lowering the effective accuracy of spoofers, without compromising other desirable properties, as is often the case when trying to specifically design watermarking schemes more resistant to spoofing (see $2).
Among learning-based spoofers, there are two approaches that generalize across most Red-Green schemes: Steal- ing (Jovanović et al., 2024) and sampling-based Distilla- tion (Gu et al., 2024). Stealing approximately infers the vocabulary splits by comparing the frequencies of tokens in D (conditioned on the same context) with -eeeangeeeeeteddd text, and uses this information to generate spoofed text using an auxiliary LM. In contrast, Distillation directly fine-tunes an auxiliary LM on D, effectively distilling the watermark into the model's weights. Both Stealing and Distillation are applicable on Red-Green schemes, but Distillation also expands to other schemes (see App. G).
Concretely, we assume the perspective of the model provider with private key & and a model M. We receive a text w E ΣT that is flagged as watermarked by our detector De. and aim to decide whether was generated using our private key &, or by spoofing method. Our threat model also includes the case where we receive a set of texts from the same source, whose concatenation we denote as w E XT for simplicity (see the bottom of $4.2 for details). We assume that our private key & was not simply leaked; else, spoofed texts are hardly distinguishable from waateemaarked texts.
Spoofing defenses Some watermarking schemes have been specifically designed for attribution and hence are more resistant to spoofing (Zhou et al., 2024; Christ et al., 2024b; Christ & Gunn, 2024; Fairoze et al., 2023). Yet, in their focus on providing attribution, they trade off other desirable
Formalization Determining whether text w was spoofed can be formulated within the hypothesis testing framework:
Ho : The text w is f E-watermarked,
(1)
H1 : The text w is spoofed.
3

---

Discovering Spoofing Attempts on Language Model Watermarks
We introduce the random variable Q E" and the received text w Σ is realization of 22. We note that the distri- bution of Ω under the null hypothesis and its distribution under the alternative hypothesis are different. Similarly, let X E {0, 1)- be the associated sequence of (non-i.i.d.) Bernoulli random variables, where X, = 1 represents the event where the token is green, and let x E {0. 1)a be the observed color of w under Dş (realization of X). In this hypothesis testing framework, the challenge is to build statistic S(2)) that satisfies two key properties. First, the dis- tribution of (S2)) under the null hypothesis should be known in order to rigorously control the Type error. Second, the distributions of S(2)) under the null and S(2) under the alternative should be different, enabling us to distinguish spoofed and Ewwttrmaarked texts.
in D, the more certain a spoofer is regarding the color of the token wy. For now, we will also assume that for each token in E-watermarked text, Ip is independent of its observed color. For Vt E {h + , T), we assume:
X₂ is not independent from (p(0c-
if the text spoofed:
(3a)
X₂ is independent from TD TD(A not)
if the text is E-watemmarked.
(3b)
This dependence between the color and 1p (p4e results in spoofing artifacts under the alternative.
Influence of the LM Counterintuitivelly the indepen- dence assumed in Eq. (3b) may be violated. To generate Wes the model provider first computes the logit vector 4 knowing Wer- Then, it computes the greenlist defined by PRF(H (wh-Art-1), 1), E), and increases the logits of green tokens by 6. Finally, it samples from the newly defined probability distribution to generate the token wy- The green- list itself is thus indeed independent of ($4 Yet, 1 was originally computed using Wer due to the autore- gressive property of the model M, and hence may not be independent of (S2,
3.2. Artifact: dependence between the color sequence and the context
Next, we explain why spoofed texts contain observable artifacts, as was illustrated in Figure 1.
A simple example To expand on this intuition. we start by considering an example of perfect spoofer that produced the text w E ΣT, and knows the color of a token wM if and only if w E DD where D is the training data of the spoofer. Otherwise, if -hitt # DD we assume that the spoofer has chosen We independently of its color. Let ID E4+1 + {0, 1} be the indicator function of the presence of (h++))-rrmm in D. To can be interpreted as the knowledge the spoofer has over the vocabulary splits. From above, we can assume that for all t E {h- + TT.:
To illustrate this point, consider a case where the token wy is the only viable continuation of w:-hut-1. i.c., 4 is low-entropy. Then, Bayes' theorem implies that (w) is likely to be high. On the other hand, the logit increase of ő has less influence on the sampling, as it is less likely to cause token other than Wi to be sampled- sampled-thus, the color of we is effectively random, i.e., P(X, = 1) = Y. even for g-watermarked text. Hence, the events P(X, = 1) R "y and (S2 ) is high, are correlated, as they occur simultaneously in case of low entropy. We investigate this dependence pattern and confirm it experimentally in App.
P(X, = 19 = 1) 2 P(X, = [ID = 0)
if the text is spoofed;
(2a)
P(X, (14 = 1) P(x. = 111D = 0)
With this in mind, properly control for Type 1 error, we need to design a test statistic S where this dependence pattern is known or can be learned for g-watermarked texts. Moreover, to maintain power, we aim to distinguish this dependence from the dependence present in the case of spoofed text, as described above in Equations (3a) and (3b).
if the text is E g-watermarked.
(2b)
Equations (2a) and (2b) reflect that the knowledge of the vocabulary split at token helps the spoofer color wi green, which is its original goal. For a E-watermarked text, the knowledge of a potential spoofer has no influence on its coloring. Hence, we may be able to use In to distinguish whether a sentence is spoofed or not. We now generalize this intuition to more realistic spoofing scenarios.
4. Designing a Test Statistic
We proceed to introduce our test statistic S, deriving funda- mental results regarding its distribution under the indepen- dence assumption from Eq. (3b), and in the more general case where it may be violated ($4.1). Then, we present and discuss two concrete instantiations of S (64.2).
Color sequence depends on the context distribution In practice, learning how to spoof may require observing an multiple times. Moreover, spoofing techniques may, albeit not necessarily explicitly, have different levels of certainty regarding the color of a token given a context. Therefore, we generalize In : Ih+1 - [0. 1] to be the function of the frequencies of (h- + (h+1)-grams 1)-grams in D. We make a natural assumption that the higher the frequency of wi-his
4.1. Controlling the distribution
We introduce the main results regarding the distribution of S(12) under the null hypothesis.
4

---

Discovering Spoofing Attempts on Language Model Watermarks
Color-score correlation Let E Σ", sampled from Ω2 denote the text of length T received by the model provider, E {0, 10" sampled from X, denote its color sequence under Dg, and y E [o, 1]" denote sequence of scores for each token sampled from a sequence of 7 random variables Y. We defer the construction of Y to $4.2, where we will build on the intuition from $3.2. As the test statistic, we use the sample Pearson correlation coefficient between x and y.
we need use the model to generate the new text), we can, in the independent case, compute p-value using Z-test on the statistic fe(w,w"), which is sampled from a standard normal distribution. We later refer to this test as the Reprompting method. For consistency, in Reprompting experiments in $5, we use T to implicitly refer to c. We additionally study Reprompting in greater details App. F.
4.2. Concrete instantiations
S(w) = En (a+ (y 5)
(4)
In this section we instantiate the score sequence Y and propose practical modifications to S.
EE=(( (Te 8)2 JPEE= (Ve 8)2
Independence case We first study the distribution of S(2) under the assumption that X; and Y; are independent for all in Eq. (3b) (we refer this as iroseinndeee between X and Y). From this assumption, we derive:
Construction of the token score We propose two instan- tiations of the score function Y: one that closely follows the intuition from $3.2, and another that aims to achieve the independence assumption from Lemma 4.1. Achieving cross-indepenue cross-independence allows the construction of a test that does not require reprompting the model, hence reducing computational complexity.
Lemma 4.1. Under the cruss-independence between X and Y, and technical assumptions (detailed in App. 1), we have the convergence in distribution
(h+1) --g-am score For the first instantiation, the idea is to directly approximate ID. the function of (h + 1)-grams frequencies in D. As D is not known to the model provider, we approximate it with a text corpus D. We define
Zs(12)= = VTS(1)= NONNO (0.1).
We defer the proof to App. I. Therefore, given text w, we can compute a p-value using a two-sided 7.test on the statis- tic (w),,, which is sampled from a standard normal distri- bution. We will refer to this test as the Standard method.
y/2 == Io(we-hat)
(7)
In practice, we set D to C4 (Raffel et al., 2020). We study the influence of D in App. D. Finally, to reduce the required size of D needed to obtain a good estimate of Ip. we compute the frequency of unordered (h + 1)ggrss. Because the independence assumption from Lemma 4. 1 not met here (see $5.1), we use Reprompting with this score.
The general case In practice, however, the assumption of prossindeeenndece between X and Y does not always hold (see $3.2). We make a modeling assumption motivated by the results from the independent case. Let μi := E[)). Under the null hypothesis (and the practical considerations outlined below), we assume that
Unigram score For the second instantiation, the intuition is to trade-off between cross- iosssindeeeeneece and reflecting Ip. Let f Σ + [0, 1] be the unigram frequency in human generated text. We define
VTS( N(Ha, 1).
(5)
Compared to Lemma 4.1, the difference is that the normal distribution is offset by μa. This introduces a key challenge: finding way to estimate Hg. To this end, we propose to use Wge. a prefix of w of length c, to prompt our model .M to generate a new sequence w' of length T' = T c (which is a realization of Ω¹). In practice, we set c = 25. Given the shared prefix, we expect that f2ye S2' and hence that [{{xs)) [[S(R)) = un- Then we introduce the statistic (12 22), defined by
yt = (W)--)
(8)
We look at the unigram frequency furthest from f, to make the dependence between X and Y negligible. Yet, we re- main within the context window so ye partially reflects the information from [D( and hence still allows distin- guishing spoofed and E-watermarked texts. We will see in $5.1 that cross-independence is satisfied for SumHash h 3. Hence, in settings where cross-independence is verified, we use this score with the Standard method.
Zn(w,w')= S(wSc) - S(ww) VI/(T-c) -c 1/T
(6)
Under the null hypothesis, we have that Zx(2,9) ~ (0 1), as S(wse) and 5(w') are two independent sam- ples from a normal distribution. Therefore, in the general case, at the cost of higher computational complexity (since
Practical considerations In practice, we add modifica- tions to the statistic S. First, as suggested in Kirchenbauer et al. (2023), we ignore repeated h-grams in the sequence w. This is required to enforce the independence assumption
5

---

Discovering Spoofing Attempts on Language Model Watermarks
within X and the independence within Y. Second, to limit the influence of outliers on the score, use the Spear- man rank correlation instead of the Pearson correlation and further apply a Fisher transformation. This means that in Eq. (19), x and y are respectively replaced by R(x) and R(y), where R is the rank function. Hence, the statistic (ww) used in practice is defined as the arctanh of
App. G, we show that the tests generalize to two additional watermarking schemes, AAR (Aaronson, 2023) and KTH (Kuditipudi et al., 2024), which only Distillation can spoof.
Experimental setup We primarily focus on the KGW SumHash scheme, using a context size E (1, and
7 = 0.25. For h {1,2], we set 8 2. For h 3. we use 8 4 for Stealing ensure high spoofing rates and note that Distillation is unable to reliably spoof in this setting, and therefore is excluded from our h = 3 experiments. In cach experiment, we generate either spoofed or &- E-watermarked continuations of prompts sampled from the news- like C4 dataset (Raffel et al., 2020), following the methodology from prior work of Kirchenbauer et al. (2023). For each parameter combination, each being between 50 and 400 tokens long. Then, we con- catenate continuations (see $4.2) to reach the targeted token length T. Finally, each concatenated continuation is filtered by the watermark detector, and only watermarked sequences are kept. We use those concatenated continuations to com- pute the test statistic S. In practice, we have on average a total of 10 /T samples per parameter combination.
T
Σ (R(x), - R(T)) - R(y))
T Σ (R(x), - R(a))2 Σ (R(U) - R(v))²
Therefore, we also use the variance / / instead of / to reflect the influence of the rank function, as suggested in Fieller et al. (1957) for the i.i.d. case.
generate 10,000 continuations,
Combining texts Given a set of texts from a single source, we concatenate all its elements to create a single text of size T. In particular, let n E and w²,.... .......... w E 5Th, x...x x ETH such that Ti +...+, = T for given T. For the Standard method, we set w = o...ow" ow"" For the Reprompting method, we compute w'l... " inddpen- dently enforcing T = T: - and then set == w/o.. .o.,,, and define w>e == w o. -ww".. We verify experimentally in App. B that the concatenation operation has no influence on the distribution of the statistic. Our experiments with large T in $5 are thus conducted on concatenated texts.
We match the experimental setup from Jovanović et al. (2024) and Gu et al. (2024). In particular, we use LLAMA2- 7B as the watermarked model. More specifically, in line with their original setups, we use the instruction fine-tuned version for Stealing and the completion version for Distil- lation. For the spoofer LM, we use MISTRAL-7B as the attacker for Stealing and PYTHIA- 1.4B as the attacker for Distillation. Finally, for the spoofer training data D, we use E-waterarrked completions of C4. For Stealing, D is com- posed of 30,000 samples, each 800 tokens long, whereas for Distillation, D is composed of 640,000 samples, each 256 tokens long. We further study the impact of [D] in App. E.
Extending to other watermarking schemes The frame- work can naturally be extended to most other watermarking schemes. On a high level, a watermarking scheme is a se- quence of random vectors Çt and mapping IR x E such that the next token is sampled according to the logit vector (1,,, (.) instead of 4. In the case of Red-Green wa- termark, G is simply the coloring of the vocabulary, and we set z; = Ee [ in Eq. (4). Hence, for any other schemes, when setting zt = Çe [we] the results from $4.1 still hold.
5.1. Validating the normality assumption
In $4 we discuss two cases, each relying on one fundamental assumption. The Independence case: we assume indepen- dence between the color sequence X and scores Y, from which we derive the normality of SS2) with known mean (Lemma 4. For this case, we use the Standard method with the unigram score (Eq. (8)). The General case: we alternatively assume that S(2) is normally distributed with an unknown mean (Eq. (5)). Here, we use the Reprompting method with the (h- + 1)-gram score (Eq, (7)). In Figure 2, we test the Independence case assumption by validating if (S()) with the Standard method and unigram score fol- lows standard normal distribution (Top), and the General case assumption by validating the same for Z n R') with the Reprompting method and (h + 1)-gram score (Bottom), We additionally perform both a Saapogorov-Smmmmoo test for standard normality and a Pearson's 's normality test (not necessarily standard normal).
We show conerete instantiations of and experiment eval- uation of our method for both AAR (Aaronson, 2023) and KTH (Kuditipudi et al., 2024) in App. G.
5. Experimental Evaluation
We present the results of our experimental evaluation. In $5.1, we validate the normality assumptions from $4.1. In $5.2, we validate the control of Type I error and evaluate the power of the tests from $4 both spoofing techniques introduced in $2: Stealing (Jovanović et al., 2024) and Dis- rillation (Gu et al.. 2024). In App. A, we compare the test results across a wider range of spoofer LMs, and we show additional results with a different watermarked model M, parameter combinations, and another prompt dataset. In
6

---

Discovering Spoofing Attempts Language Model Watermarks
-- 0.00+ 00
p-32c-56 pm
M/0, 1)
T: 500
T: 1000
Danutg
0.3
T: 2000
0.0
2x
2,.02
6m.
0- 422 -22
p-4 de-
p-120-02
2.76
p.4.50 01
3.00
Dinstty
9.2
0.1
0.0
2gp hm1
2..0=2
Ze. 6m3
Figure 2: Histograms of (S2) (top) and (S2 52') (bottom), with y-axes scaled to represent normalized density. The top row is computed using the unigram score and the Standard method, and the second row is computed using the (h + 1)-gram score and the Reprompting method. A green line indicates that the (O, 1) hypothesis is not rejected (top p-value), an orange line that a normality test is not rejected (bottom p-value), and red line that both are rejected at 5%.
1.0
Instruction (Repromting)
FPR (experimental
0.8
Instruction (Standard)
0.6
Completion
0.4
500
T: 1000
T: 2000
0.0
0.00
0.25
0.50
0.75
1.00
0.00
0.25
0.50
0.75
1.00
0.00
0.25
0.50
0.75
1.00
a. h=1
h=2
h=3
Figure 3: Experimental rejection rate of E-aaermmarked text LLAMA2 7B.
Regarding the Independence case, we see that in the top row, (S2) follows standard normal distribution only for
rejection rates a E [0,1].
3. This confirms our intuition behind the unigram score: as h increases, the dependency between X, and (94-n) becomes negligible. Hence, for h Standard method with the unigram score.
P(p S a) aa
(10)
We further evaluate the test power on Stealing and Distilla- tion, i.e. how effective it distinguishing spoofed text from Ç w-ttemmmrrked text. Additionally, that the Type 1 error rate remains properly controlled in the case of g-watermarked text that has been edited by humans.
we may use the
show in App. H
For the General case, we see in the bottom row that the histogram approximately matches the standard normal dis- tribution for = 2, and that the normality assumption always holds. Overall, these results suggest that the assump- tions behind the Reprompting method are sound, allowing it 1)-gram
Type error To evaluate Type error, we compare the experimental rejection rate under the null hypothesis against the set rate a. Per Eq. (10), if the test controls Type error well, we expect the curve be below the identity function.
(with the (h (h+1)-gram score) be used for all tested param- eter combinations. Hence, all results in $5.2 computed with the Reprompting method and (h + -rrm score.
In Figure 3, show the experimental rejection rate of g-watermarked text on LLAMA2-7B (both instruction fine- tuned and completion models) for different values of h and T. We observe that the experimental rejection rates align closely with the identity function. These results show that, in practice, setting rejection rate of guarantees that the
5.2. Evaluating the spoofing detection tests
To ensure the statistical test is sound, we check whether the Type I error rate is properly controlled. This means that, under the null, letting be the resulting p- value, for all

---

Discovering Spoofing Attempts Language Model Watermarks
1.0
Stealing (Repromting)
0.8
Stealing (Standard)
0.6
Distillation
0.4
T: 500
T: 1000
0.2
T: 2000
0.0
т:
10
10-2
10 10 10
10
10
10a
10 3
10
10 10c
0, h=1
a, h=2
a, h=3
Figure 4: Experimental True Positive Rate of spoofed text. The dotted lines are the identity and serve as a reference for the expected rejection rate under the null. Since, in practice, low false positive rate (a) is desirable, the logarithmic scale on highlights the true positive rate at low values.
Table 1: Experimental FPR and TPR for both spoofers at a E (1%, 5%), for different h and T. h = (R) denotes the Reprompting method with (h + 1)-gram score while = 3 (S) denotes the Standard method with unigram score. All other entries are for the Reprompting method with (h + 1)-gram score.
T 500
T 1000
2000
3000
FPR
TPR
FPR 05%
TPR 05%
FPR @ 16
TPR 1%
FPR 05%
TPR @5%
FPR
TPR
FPR
TPR 55%
FPR
TPR @1%
FPR
TPR
Spoofer
1%
@ 1%
@1%
@ 1%
@5%
@ 1%
05%
@5%
h=
0.00
0.62
0.04
0.81
0.00
0.93
0.04
0.99
0.01
1.00
0.05
1.00
0.01
1.00
0.07
1.00
=
0.01
0.16
0.04
0.35
0.00
0.37
0.04
0.59
0.01
0.73
0.05
0.88
0.01
0.91
0.04
0.97
STEALING
= (R)
0.47
0.05
0.73
0.01
0.85
0.05
0.95
0.01
0.99
0.05
1.00
0.01
1.00
0.06
1.00
h= (S)
0.01
0.27
0.05
0.53
0.01
0.55
0.04
0.80
0.01
0.88
0.03
0.97
0.00
0.97
0.03
1.00
0.48
0.04
0.71
0.01
0.86
0.05
0.96
1.00
0.06
0.01
1.00
0.03
1.00
DISTILL ATION
h=
0.01
0.57
0.06
0.78
0.01
0.91
0.06
0.97
0.01
1.00
0.05
1.00
0.00
1.00
0.07
1.00
experimental False Positive Rate of the test is indeed a.
behind our approach ($3.2) is general and that points to a fundamental limitation current spoofing techniques.
Test power To evaluate the power of the test, we com- pute the empirical true rejection rate (i.c., TPR) under the alternative hypothesis for given threshold a.
6. Conclusion
In this work, building on the intuition that spoofed text con- tains artifacts reflecting the spoofer's knowledge, we suc- cessfully constructed statistical tests to distinguish between spoofed and genuine watermarked texts. The tests behave similarly on the studied spoofers, and across a wide range of watermark settings. Our results show that spoofed text can be reliably distinguished from genuine watermarked text with arbitrary accuracy given long enough text, and high- light shared limitations of current learning-based spoofers.
In Table 1, we provide the experimental False Positive Rate (FPR, rejection under the null) and True Positive Rate (TPR, rejection under the alternative) for a fixed value of a. For T = 3000, under all tested scenarios, we achieve more than 90% TPR at a rejection rate of 1%. This suggests that, given a long enough text (or concatenation text), spoofed text from both -fate-of-the-art methods can be distinguished from g-watermarked text with high accuracy and reliable control over the false positive rate. Moreover, we that the Reprompting method yields higher power than the Standard method for all values of T. Yet, the Standard method, in the cases where is applicable, does not require prompting the model M, and thus may still be preferable.
While we can provide an experimental eval- uation of power on current state-of-the-att spoofers, the proposed tests come with no theoretical guarantee power. We build our tests on reasonable assumptions regarding the limitations of learning-basse spoofing techniques. Yet, we hypothesize that spoofing techniques that adaptively learn the vocabulary split may avoid leaving similar artifacts in generated text. Designing such attacks can be an interesting path for future work. Additionally, to have high power, our tests require that the total length of the input texts is not too small. Future work could try to improve the efficiency of our method from this perspective.
Limitations
Additionally, in Figure 4, we show the evolution of the TPR with respect a. We observe that for any fixed a E [0, 1], the power at converges to 1 as T grows. This indicates that the test can achieve arbitrary TPR at a, given suffi- ciently long text. Also, we see that despite the fundamental differences between the two spoofing techniques, the texts produced by both Stealing and Distillation can be distin- guished with the same test. This highlights that the intuition
8

---

Discovering Spoofing Attempts Language Model Watermarks
Impact Statement
Conover, M., Hayes, M., Mathur, A., Xie, J., Wan, J., Shah, S., Ghodsi, A., Wendell, P., Zaharia, M., and Xin, R. Free dolly: Introducing the world's first truly open instruction- tuned 1lm, 2023.
This paper presents a detailed analysis of texts produced by learning-base spoofing methods and proposes a way to mitigate spoofing issues. While our findings can be used to improve the stealth of future spoofing methods, we believe that improving the understanding of watermark spoofing outweighs the potential misuse of our work.
Dathathri, S., See, A., Ghaisas, S., Huang, P.-.,, McAdam, R., Welbl, J., Bachani, V., Kaskasoli, A., Stanforth, R., Matejovicova, et al. Scalable watermarking for identi- fying large language model outputs. Nature, 634(8035): 818-823, 2024a.
Acknowledgments
This work has been done as part of the SERI grant SAFEAI (Certified Safe, Fair and Robust Artificial Intelligence, con- tract no. MB22.00088) Views and opinions expressed are however those of the authors only and do not necessarily reflect those of the European Union or European Commis- sion, Neither the European Union nor the European Com- mission can be held responsible for them. The work has received funding from the Swiss State Secretariat for Edu- cation, Research and Innovation (SERI) (SERI-funded ERC Consolidator Grant).
Dathathri, S., See, A., Ghaisas, S., Huang, P.-S., McAdam, R., Welbl, J., Bachani, V., Kaskasoli, A., Stanforth, R., Matejovicova, T., et Scalable watermarking for identi- fying large language model outputs. Nature, 634(8035): 818-823, 2024b.
Dubey, A., Jauhri, A., Pandey, A., Kadian, A., Al-Dahle, A., Letman, A., Mathur, A., Schelten, A., Yang, A., Fan, A., et al. The llama 3 herd of models. arXiv, 2024.
Fairoze, J., Garg, S., Jha, S., Mahloujifar, S., Mahmoody, M., and Wang. M. Publicly detectable watermarking for language models. arXiv, 2023.
References
Aaronson, S. Watermarking of large language models. In Workshop Large Language Models and Transformers, Simons Institute. UC Berkeley, 2023.
Fernandez, P., Chaffin, A., Tit, K., Chappelier, V., and Furon, T. Three bricks to consolidate watermarks for large lan- guage models. In WIFS, 2023.
Biden, J. R. Executive order on the safe, secure, and trust- worthy development and of artificial intelligence, 2023.
Fieller, E. C., Hartley, H. O., and Pearson, E. S. Tests for rank correlation coefficients. Biometrika, 1957.
Fourrier, C., Habib, N., Wolf, T., and Tunstall, Lighteval: A lightweight framework for Ilm evaluation, 2023. URL httptttwww..o/ooggggggga//////ee
Bubeck, S., Chandrasekaran, V., Eldan, R., Gehrke, J., Horvitz, E., Kamar, E., Lee, P., Lee, Y. T., Li, Y., Lund- berg, M.. Nori, H., Palangi, H., Ribeiro, M. T., and Zhang, Y. Sparks of artificial general intelligence: Early experiments with GPT-4. arXiv, 2023.
Gao, I., Liang, P., and Guestrin, C. Model equality testing:
Which model is this api serving? ICLR, 2025.
CEU. Proposal for a regulation of the european parliament and of the council laying down harmonised rules on artifi- cial intelligence (artificial intelligence act) and amending certain union legislative acts analysis of the final com- promise text with view agreement. 2024.
Gloaguen, T., Jovanovié, N., Staab, R., and Vechev, M. Black-box detection of language model watermarks, In ICLR, 2025.
Gu, C., Li, X. L., Liang, P., and Hashimoto, T. On the learnability of watermarks for language models. In ICLR, 2024.
Chang, H., Hassani, H., and Shokri, R. Watermark smooth- ing attacks against language models. arXiv, 2024.
Guan, B., Wan, Y., Bi, Z., Wang, Z., Zhang, H., Sui, Y., Zhou, P., and Sun, L. Codeip: A grammar-guider multi- bit watermark for large language models of code. EMNLP Findings, 2024.
Christ, M. and Gunn, S. Pseudorandom enror-correctng codes, 2024. URL 09370.
Christ, M., Gunn, S., Malkin, T., and Raykova, M. Prov- ably robust watermarks for open-source language models. arXiv, 2024a.
Hu, Z., Chen, L.,, Wu, X., Wu, Y., Zhang, H., and Huang, H. Unbiased watermark for large language models. In ICLR, 2024.
Christ, M., Gunn, S., and Zamir, O. Undetectable water- marks for language models. COLT, 2024b.
Jovanović, N., Staab, R., and Vechev, M. Watermark steal- ing in large language models. ICML, 2024.

---

Discovering Spoofing Attempts Language Model Watermarks
Kirchenbauer, J., Geiping, J., Wen, Y., Katz, J., Miers, I., and Goldstein, T. A watermark for large language mod- els. In Krause, A., Brunskill, E., Cho, K., Engelhardt, B., Sabato, S., and Scarlett, J. (eds.), International Con- ference on Machine Learning, ICMI. 2023, 23-29 July 2023, Honolulu, Hawaii, USA, volume 202 of Proceed- ings of Machine Learning Reseurch, pp. 17061-17084. PMLR, 2023. URL https: https/////oceeeeesseeeeeeeeeesss , ml wtt.sssmmrpppeeeeee press/ v282/kirchennauerrBaa html.
Pang, Q., Hu, S., Zheng, W., and Smith, V. Attacking l1m watermarks by exploiting their strengths. In ICLR 2024 Workshop Secure and Trustworthy Large Language Models, 2024.
Raffel, C., Shazeer, N., Roberts, A., Lee, K., Narang, S.,
Matena, M., Zhou, Y., Li, W., and Liu, P. Exploring the limits of transfer learning with unified text-to-text trans- former. J. Mach. Learn. Res., 11:140:1-140:00 2020. URL http://tjp/mttt http:://mmrr.orgpaapeeeeee www.oorggggg//////...
874. html.
Kirchenbauer, J., Geiping, J., Wen, Y., Shu, M., Saifullah, K., Kong. K., Fernando, K., Saha, A., Goldblum, M., and Goldstein, On the reliability of watermarks for large language models. In ICLR, 2024.
Ren, J., Xu, H., Liu, Y., Cui, Y., Wang. S., Yin, D., and Tang. J. A robust semantics bbaeed watermark for large language model against paraphrasing. In Duh, K., Gomez, H., and Bethard, S. (eds.), Findings of the Association for Computational Linguistics: NAACL 2024, pp. 613-625, Mexico City, Mexico, 2024. Association for Computa- tional Linguistics. URL https://aclantholoogy org/ 2024. maa.iinggsnaacl.4...
Krishna, K., Song, Y., Karpinska, M., Wieting, J., and lyyer, M. Paraphrasing evades detectors of ai-generated text, but retrieval is an effective defense. In Oh, A., Naumann, T., Globerson, A., Saenko, K., Hardt, M., and Levine, S. (eds.), Advances in Neural Information Processing Systems 36: Annual Conference on Neural Information Processing Systems 2023, NeurIPS 2023, New Orleans, LA, USA, December 10 16, 2023, 2023. URL http:
Sadasivan, V. S., Kumar, A., Balasubramanian, S., Wang, W., and Feizi, S. Can ai-generated text be reliably de- tected? arXiv, 2023.
//papers nips. ccppppppppppppcccccccccceee..eee files//leeeeeee wwt.://ieeeeeeeeeeeeeeeeeeeeeeee httpppppppe//2//aaa/la/ 575c455001330e99944bbeec (atp:eedddbbboed0oaaaeaaa (2wd88 faa4 Abs wacstaaattoooo Cenntrrrrt--oofeenneee Conference ing watermarked large language models. arXiv, 2023.
Tang, L., Uberti, G., and Shlomi, T. Baselines for identify-
html.
Wang, L.., Yang, W., Chen, D., Zhou, H., Lin, Y., Meng, F., Zhou, J., and Sun, X. Towards codable text watermarking for large language models. ICLR. 2024.
Kuditipudi, R., Thickstun, J., Hashimoto, T., and Liang, P. Robust distortion-free watermarks for language models. TMLR, 2024.
Fikimediaaoomddtion. Wikimediadownloudds URL https: / //dumps. wi kikedia.oor.
Lee, T., Hong, S., Ahn, J., Hong, I., Lee, H., Yun, S., Shin, J., and Kim, G. Who wrote this code? watermarking for code generation. ACL Long Papers, 2024.
Wu, Q. and Chandrasckaran, V. Bypassing LLM watermarks with color-aware substitutions. ACL Long Papers, 2024.
Li, S., Yao, L., Gao, J., Zhang, L., and Li, Y. Double- i watermark: Protecting model copyright for Ilm fine- tuming. arXiv, 2024.
Wu, Y., Hu, Z., Zhang, H., and Huang, H. Dipmark: A stealthy, efficient and resilient watermark for large lan- guage models. ICML, 2024.
Liu, A., Pan, L., Hu, X., Li, S., Wen, L., King, I., and Philip, S. Y. An unforgeable publicly verifiable watermark for large language models. In ICLR, 2023.
Yoo, K. Ahn, W., and Kwak, N. Advancing beyond identification: Multi-bit watermark for language mod- els. NAACL, 2024.
Liu, A., Guan, S., Liu, Y., Pan, L., Zhang, Y., Fang, L.,, Wen, L., Yu, S., and Hu, X. Can watermarked 1lms be identified by users via crafted prompts? ICIR, 2025.
Zhang, 7.,, Zhang, X., Zhang, Y., Zhang, L. Y., Chen, C., Hu, S., Gill, A., and Pan, S. Large language model watermark stealing with mixed integer programming. arXiv, 2024.
Lu, Y., Liu, A., Yu, Li, J., and King, I. An entropy- text watermarking detection method. ACL Long Papers, 2024.
Zhao, X., Ananth, P. V., Li, L., and Wang, Y. Provable robust watermarking for ai-generated text. In ICLR, 2024.
Zhou, T., Zhao, X., Xu, X., and Ren, S. Bileve: Securing text provenance in large language models against spoof- ing with bi-level signature. NeurlPS, 2024.
Monteiro, J., Noel, P.-A., Marcotte, E., Rajeswar, S., Zant- edeschi, V., Vazquez, D., Chapados, N., Pal, C., and Taslakian, P. Repliqa: A question-answering dataset for benchmarking llms on unseen reference content. ArXiv preprint, abs/2406.1 11....181111 11811, 2024. URL https://arxiv. org/abs/2406. 11811.
10

---

Discovering Spoofing Attempts Language Model Watermarks
Table 2: Experimental FPR and TPR for Stealing and Distillation using Dolly instead of C4 as the basis for the generation of w. The row h = 3 (R) corresponds to the Reprompting method with (h + 1)-aram score whereas = (S) corresponds to the Standard method with unigram score. Else only the Reprompting method with (h + 1)-gram score is used.
200
T 500
= 1000
FPR
TPR
FPR
TPR 69.5%
FPR 15
TPR
FPR 695%
TPR 69.5%
FPR
TPR
FPR 69.5%
TPR 6155%
Spoofer
@ 81%
1 ==1
0,00
0.12
0.00
0.28
0.00
0.41
0.02
0.67
0.00
0.80
0.01
0.92
A
0.00
0.04
0.15
0.00
0.12
0.04
0.28
0.01
0.34
0.50
STEALING
h == 3 3
0.01
0.25
0.03
0.42
0.02
0.51
0.05
0.81
0.01
0.87
0.05
0.97
A (
0.00
0.16
0.02
0.43
0.00
0.27
0.02
0.51
0.01
0.48
0.04
0.75
A ==1
0.01
0.14
0.05
0.33
0.00
0.42
0.02
0.64
0.00
0.69
0.02
0.83
DISTILLATION
==2
0.02
0.12
0.07
0.27
0.01
0.36
0.07
0.59
0.02
0.67
0.07
0.84
Table 3: Experimental Rejection Rate (RR) for Stealing with SelfHash and / = 3 for both g-watermarked text and spoofed text.
T 200
T 500
T 1000
T 2000
RR
RR
RR
RR
RR
RR
RR
RR
Experiment
Method
Spoofer LM
@ 11
@5%
@ 11%
55%
@1%
65%
@ 11%
55%
Reprompting
/
0.01
0.04
0.00
0.03
0.00
0.01
0.00
0.03
& g-watermarked
Standard
0.00
0.03
0.00
0.02
0.01
0.02
0.00
0.01
LLAMA2-7B
0.12
0.30
0.31
0.59
0.70
0.90
0.99
1.00
Reprompting
GEMMA-2B
0.14
0.30
0.45
0.73
0.83
0.93
1.00
1.00
MISTRAL--BB
0.10
0.29
0.38
0.63
0.79
0.93
1.00
1.00
STEALING
LLAMA2-7B
0.03
0.13
0.06
0.20
0.11
0.35
0.32
0.63
Standard
GEMMA-2B
0.03
0.14
0.07
0.26
0.15
0.40
0.36
0.62
MISTRAL-7B
0.05
0.22
0.15
0.39
0.35
0.63
0.74
0.83
A. Additional Experimental Results
In this section, we conduct several thorough ablation studies. We evaluate the test using different dataset base prompts (App. A.1), with different variation of the watermark scheme (App. A.2), using another watermarked model (App. A.3), using other spoofer models (App. A.4), and using both a different watermarked and spoofer model in App. A.5. In all additional settings tested, the results are similar to those presented in $5, which emphasizes the validity of the test and shows that the spoofing artifacts studied are a fundamental property of learning- bearning-based spoofers.
Unlike in $5, we generate 1,000 continuations per parameter combination for the ablation study. It means that on average we have 10$ /T' samples per parameter combination.
A.1. Mitigating potential methodological biases
Here, we use the same settings as $5 (Stealing and Distillation with SumHash, different values of h, and for h the Reprompting and Standard methods), but use text continuations of prompts sampled from Dolly (Conover al., 2023) instead of the C4 dataset. We show that the methodology used to generate the spoofed and w-atermmarked texts has no influence on the results.
3, both
In Table 2, we show the experimental FPR and TPR at a of 1 1% and 5%. The results are similar to those on C4 from Table I: the Type error is controlled, and the power is similar. This suggests that the methodology we use to generate the prompts does not influence the results. Hence, we can expect that for most texts w, the empirical results presented hold, and that w spoofed, the spoofer's 's artifacts remain present and discoverable.

---

Discovering Spoofing Attempts Language Model Watermarks
Table 4: Experimental Rejection Rate (RR) for Stealing with MISTRAL7B as M at a of 1% and 5% on both E-watemmarked text and spoofed text.
T == 200
T = 500
T = 1000
T T= 2000
RR
RR
RR
RR
RR
RR
RR
RR
Experiment
Method
Spoofer I.M
@ 1%
05%
@ 11
685%
@ @1%
65%
@ 11%
65%
Reprompring
0.02
0.05
0.03
0.08
0.00
0.04
0.00
0.02
Ewatermarked
Standard
0.01
0.04
0.01
0.02
0.00
0.02
0.00
0.02
LLAMA2-7B
0.45
0.73
0.89
1.00
0.99
1.00
1.00
1.00
Reprompting
GEMMA-2B
0.48
0.90
0.97
1,00
1.00
1.00
1.00
1.00
MISTRAL--7B
0.59
0.81
0.97
1.00
1.00
1.00
1.00
1.00
STEALING
LLAMA2-7B
0.19
0.41
0.25
0.60
0.45
0.79
0.83
0.95
Standard
GEMMA-2B
0.21
0.48
0.40
0.66
0.64
0.81
0.85
0.96
MISTRAL-7B
0.27
0.55
0.70
0.88
0.94
0.99
0.99
1.00
Table 5: Experimental FPR 1% and 5% with SumHash h = 2, across spoofer LMs. Bold corresponds to the case where both the spoofer and watermarked models are the same.
T 200
T = 500
T 1000
T = 2000
TPR
TPR
TPR
TPR @5%
TPR
TPR @5%
TPR 1%
TPR 55%
Experiment
Spoofer LM
@ 11%
05%
@ 11%
1%
LLAMA2-7B
0.07
0.16
0.14
0.34
0.36
0.62
0.68
0.88
STEALING
GEMMA-2B
0.02
0.17
0.09
0.32
0.29
0.52
0.61
0.82
MISTRAL.-7B
0.05
0.16
0.16
0.35
0.37
0.59
0.73
0.88
LLAMA2-7B
0.20
0.46
0.60
0.80
0.94
0.99
1.00
1.00
DISTILLATION
PYTIIA-1-49
0,27
0.55
0.57
0.78
0.91
0.97
1.00
1.00
A.2. Results for the SelfHash scheme
Next, focus SelfHash with h 3 and 5 4 for Stealing. We use both the Reprompting and the Standard method with their respective score functions ($4.2).
In Table 3, we show the experimental FPR at a 1% and a = 5% for é-watermarked and spoofed text. Similarly to the SumHash variant, the Type error is properly controlled for both the Standard and the Reprompting methods. Moreover, the empirical power scaling with T is similar to the SumHash scheme from Table This means that the spoofing artifacts are not tied to a specific scheme, but rather represent a fundamental limitation of learning- baaed watermark spoofing techniques such Stealing and Distillation. Additionally, we also see that the power of the Standard method fixed T is lower than that of the Reprompting method. This confirms the expected trade-off of the unigram score: enforcing irossindepenndece is traded for power ($4.2).
A.3. Alternative watermarked model
In this experiment, we MISTRAL-7B the watermarked model M for SumHash at h on Stealing. We do not use different M for Distillation, as Distillation was only empirically validated on LLAMA2-7B (Gu et al., 2024).
In Table 4, we show the experimental FPR at a of 1% and 5% for E-watermarked text and spoofed text on different spoofer LMs. Similar to the results in Table 1, the Type I error is controlled in both the Reprompting and Standard methods. Moreover, the power scaling with T is also similar to the results from Table 1. This suggests that the model M used by the model provider has no influence on the artifacts left by spoofing attempts on such a model.
A.4. Influence of the spoofer model
In this experiment, we run our tests on SumHash with h = 2, using for Stealing LLAMA2-7B, MISTRAL-7B and GEMMA 2B, and for Distillation LLAMA2-75 and PYTHIA-1.499
12

---

Discovering Spoofing Attempts Language Model Watermarks
Mistral-78
Llama2-7B
Gemma-22 28
Q')
Ulama2-78
Pythla-1.4B
4
1000
2000
3000
4000
5000
T
Figure 5: Evolution of E 22)] for different spoofer LMs with T.
1.0
0.8
Rejection Rate
T: 500
0.6
T: 1000
2000
0.4
E-watermarkd Spoofed
0.2
0.0
10-3
10-2
10-1
10"
a, h=3
Figure 6: ROC curves with LLAMA3-8B as the watermarked model, Stealing with QWEN2.5-7B the spoofer model, and h with Reprompting.
In Figure 5, we show the evolution of the expected value of (S2, 52') for spoofed texts with respect to T, across different spoofer LMs. We see that the evolution of the average Z-score is similar across all models and both spoofing techniques. This suggests that the choice of the spoofer LM has almost no influence on the power.
Additionally, in Table 5, show the FPR and TPR for the 5 spoofer LMs tested. For T = 2000, obtain similar results across all models, with TPR at 1% of at least 60% for Stealing and 100% for Distillation, similar to the results from $5.2. Moreover, counterintuitively, spoofer using the same model as the model owner does not significantly lower the test power. This suggests that the artifacts we are detecting in spoofed text indeed reflect the lack of knowledge of the spoofer ($3.2), and not the difference between the LM used by the spoofer and the LM used by the model provider.
A.5. Alternative watermarked and spoofer model
In this experiment, we use LLAMA3-8B as the watermarked model M with SumHash at = 3, and we spoof using Stealing with QWEN2.5-7B as the spoofer model.
Figure shows the ROC curves of our detection test using these alternative models. We see that our test remains valid, with the Type ervor being properly controlled (solid line) and the power remaining high (dashed line). This suggests that neither the model M used by the model provider nor the spoofer model influences the artifacts left by spoofing attempts.
B. Validating the Concatenation Procedure
In this section, we experimentally validate the claim that concatenating texts w according to the procedure from $4.2 has no influence on the resulting distribution of the statistic.
13

---

Discovering Spoofing Attempts Language Model Watermarks
0.25
0.20
E-watermarked
Density
E-watermarked
0.15
(Shuffled)
Spoofed
0.10
Spoofed
(Shuffled)
0.05
0.00
-5
0
5
-10
-5
Figure 7: Histogram of Z-scores for both g-watermarked and spoofed corpora, well as their shuffled counterparts.
0.4
h 1
h h= 2
Density
0.3
h = 3
0.2
0.1
0.0
-6
-4
-2 Zs(w)
Figure 8: Histogram of Zs(W) for w-atermarkked text with (h + 1)-gram score and Standard method.
Experimental setup Let W = (w], wn) be corpus of n texts of the same length, and W the corresponding corpus of Reprompting texts of the same length T. Let X, X' E (0, 1]mxTT be the color matrices of the corpora, and Y, Y' E [o, 1| be the associated (h++ + 1)-gram score matrices of the corpora. For permutations a E G,xT, we define o ) X (
and (Y)),, = (o(()))) We define o (W) as the shuffled corpus with the corresponding (X) color and oYY) score. Given SaxT, we test the hypothesis that shuffling has no influence the distribution of ZW, W").
Zn(o(W),o(W)) ~ Fe(WWW)..
(11)
where ZR,W, W') := (ZR(wi, wi),... (n( w())). The shuffling operation can be interpreted as a concatenation of texts of length Hence, the shuffling has no influence, this implies that the concatenation texts of longer length has no influence either. To test for the equality of distribution, we use a Mann-Whitney U rank test.
Results In practice, we generate n = 1000 g-watermarked and spoofed texts of length 175 and their corresponding T 150-Length Reprompting text corpora. We sample o G,xT uniformly in G,xT. In Figure 7, we show the resulting histogram of (W, W') and (W)) (W")). The histograms between the non-shuffled and shuffled versions perfectly overlap for both the E-watermarked texts and the spoofed texts. Moreover, the resulting p-values from the Mann- WainnWy U rank test are 0.86 and 0.44, respectively. Hence, procedure has no influence the distribution of the statistic.
conclude that Eq. (11) is verified and that the concatenation
C. Dependence Between the Context Distribution and the Color
In this section, we study in detail the dependence between the color of token wi and I (wr in E-watermarked text from $3.2.
Problem statement We recall that D is the training data of the spoofer, and Ip is the function of frequencies of 1-grams in D. $3.2, we hypothesize that low entropy is a common factor that implies T (52 is high and
14

---

Discovering Spoofing Attempts Language Model Watermarks
logh (p)
Stealing Distillation
Stealing Distillation
0.5
Dw
Bwo ct
n dolly
i witipeda
i repliaa
be
Left: Evolution of the p-value distribution with the Total Variation distance between DD and D. Note that the x-scale is not linear. Right: Evolution of the p-value distribution for different choices of D. Each p-value is computed with 1000-token long completions. The whiskers are set at 0.5 of the IQR for visibility.
Figure 9:
P(X; 1) Under such an assumption, we therefore expect the correlation between the observed color sequence x and the (/D(wt-itt) (tte=::))))eee....... to be negative. In other words, we expect Zs(w) with the (h + 1)-gram score to be negative for g-watermarked text.
Results We verify this claim by computing (s(w) with the (h 1)-gram for corpus W of 1000 &- &-watermarked texts, each of length T = 500. In Figure 8, we see the histograms of Zs(W) for different values of h. We see that for all h, E[Z5s(W) (W)) is indeed negative. Furthermore, the assumption underlying the Reprompting method (Eq, (5)). Therefore, these results show that the proposed intuitive explanation of the dependence due to .M is coherent, and further highlight the need for the Reprompting method in order to build statistic with a known distribution when using the (4 + 1)-gram score.
notice that the histograms appear normally distributed, agreeing with
D. Influence of the Training Dataset
In this section, we study the influence of D our ability detect spoofed texts. As do not know the true distribution of D. we hope that using different dataset does not significantly affect our results. We study the influence of D on both Stealing and Distillation methods with SumHash h = 1.
Evolution with the TV distance First, we analyze the influence of the choice of D in a controlled setting. let Da be the counts of the different (h + 1)-grams in We then build a perturbed dataset D. by adding centered normal noise with standard deviation to Dg. Finally, we compute the total variation distance between D, and D. In Figure 9 (left), observe that the vavuus increase average with the total variation distance between D. and D. This confirms the intuition that the better the estimate of 'D, the more powerful our tests are. Furthermore, it appears that the p-values increase slowly with the total variation distance, which suggests that the choice of D is not crucial for obtaining a powerful test.
Comparing different training datasets We run the test for different choices of D (C4 (Raffel al., 2020), Dolly (Conover al., 2023), Wikipedia (WikimediiaFuundation) Repliga (Monteiro et al., 2024), and Math (Fourrier al., 2023)) well as D D for comparison. In Figure 9 (right), we see that even the Math dataset has reasonable p-values for Stealing despite our experimental evaluations using significantly different prompt completions from news articles. This confirms that our test is robust to the choice of D.
Lastly, given a received watermarked text w, a model provider could adjust the choice of based on the topic of w. Such a heuristic could ensure that the choice of 'D is always relevant, and further mitigate its impact on the test power.
15

---

Discovering Spoofing Attempts Language Model Watermarks
1.0
T
0.8
500
1000
0.6
TPR
DD]
500
0.4
1500
2500
0.2
3500
4500
0.0
10
10 h=1
100
Figure 10: Experimental True Positive Rate of spoofed text with different sizes D. The size of D is measured training steps, where each step comprises 16, 284 tokens.
E. Influence of the Size of the Spoofer Training Data
In this section, we study how the power of the test impacted by the size the training dataset D. We show that increasing the spoofer training dataset reduces the presence of artifacts at a slow rate, and hence is not an effective way to remove the spoofing artifacts.
We the test for Distillation with LeftHash, h LLAMA2-7B the watermarked model, and PYTHIA- 1.4B the spoofer model. We train the spoofer with different sizes of D. We note that, for practicality, the smaller instances of D are subsets of the larger ones. Otherwise, we run the same experimental procedure as in $5.2, but using only 1000 completions.
We see in Figure 10 that the influence of increasing the spoofer training dataset is very mild. This suggests that, even though increasing the spoofer training data indeed lowers the power of the test, the rate which it does so is so slow that not an effective way to hide spoofing artifacts. Indeed, a 9-time increase in the size of only reduced the TPR 0.1 percent from 94 percent 87 percent with T 1000.
F. Influence of the Reprompting
In this section, we study in greater detail the impact of the Reprompting method. Specifically, in App. F.1 we find that estimating the prompts compared to using the original prompts only has a minor impact, and in App. F.2 we show that using fixed corpus to estimate the correlation directly (i.e., ug from Eq. (5)) leads to non-controllable Type error. Hence, the Reprompting method is needed to, in practice, properly control the Type error.
F.1. Impact of Estimating the Prompts
In this part, we analyze whether the distribution shift between the original prompts and the estimated prompts significantly influences our test power.
We run the detection test using the Reprompting statistic (Eq. (6)) once with the original prompt and once without. Otherwise, we use L.AAMA2-77 7B as the watermarked model and MISTRAL--B as the spoofer model with Stealing, h 1, and the same experimental setup as in $5. Figure 11 (left) shows the ROC curve with and without using the original prompts. As expected, we find that estimating the prompt consistently slightly degrades the TPR (at worst 5% TPR at 1 1% FPR decrease). This suggests that, for non-adversarial cases, the drift incurred by using the beginning of the received text as proxy for the prompt has minimal impact on our detection accuracy.
F.2. Using a Fixed Corpus Instead of Reprompting
In this part, we show that using a fixed estimate of the correlation distribution for any text leads to an uncontrollable Type I error, hence justifying why the Reprompting method is needed.
16

---

Discovering Spoofing Attempts Language Model Watermarks
1.0
1.0
0.8
0.8
Rejection Rate
T: 500
0.6
T: 1000
T: 500
TPR
T: 2000
T: 1 1000
0.4
E E-watermarkd Spoofed
Without prompts
With prompts
0.2
0.2
0.0
0.0
10-3
10-2
10-2
0.0
0.2
0.4
0.6
0.8
1.0
h=2
Figure 11: (Left) Influence of the prompt estimation with Reprompting: ROC curves with LLAMA2-7B as the watermarked model, Stealing with MISTRAL-7B the spoofer model, and h 1. The dotted black line corresponds to the identity function as reference. (Right) ROC curves with LLAMA2-78 the watermarked model, distillation with LLAMA2-78 as the spoofer model, and h = 2, using fixed corpus of E-watermarked text to estimate the average watermarked correlation.
1.0
FPR
TPR
0.8
100
Rejection rate
200
0.6
5:500
0.4
0.2
0.0
0.0
0.2
0.4
0.6
0.8
1.0
0.0
0.2
0.4
0.6
0.8
1.0
a, h=3
a, h=4
Figure 12: Rejection rates for the Reprompting method on the AAR watermark with h 3 and h 4. The solid lines correspond to g-watermarked text and the dashed lines to Distillation-ppoofed text.
We first generate corpus of 512k tokens of &-watermarked text using LLAMA2-7B the watermarked model and h generated using 50-token-long prompts from OPENWEBTEXT For a text to compute p-value, we directly apply Eq. (6), where we replace the "reprompted correlation" S(w') with the fixed correlation estimate from the corpus, independent of w. In Figure 11 (right), we show the ROC curve of such modified detection text using Distillation with LLAMA2-7B as a spoofer. We see that the FPR is higher than what we would expect, suggesting that the estimated mean does not capture the true mean. This why Reprompting, albeit more costly, is more reliable in practice.
G. Extending the Method to Other Schemes
While we design our method to detect spoofing attempts on Red-Green schemes (Kirchenbauer et al., 2023), as these are the primary target of several spoofing works, we show that the method can be generalized to other watermarking schemes. Excluding the unigram scheme by Zhao et al. (2024), which Zhang et al. (2024) shows can be perfectly spoofed, we can study the AAR scheme from Aaronson (2023), as well as one the KTH schemes from Kuditipudi et al. (2024), as both schemes were shown be spoofable via Distillation (Gu et al., 2024).
AAR watermark In the AAR watermark, h previous tokens are hashed using private key & obtain score r; uniformly distributed in [0, 1] for each token index i in the vocabulary E. Given P:, the original model probability for token index i, the next token then deterministically chosen as the token i that maximizes r. 1/p:
17

---

Discovering Spoofing Attempts Language Model Watermarks
1.0
FPR
TPR
0.8
T: 200
Rejection rate
5:500
0.6
T: 1000
0.4
0.2
0.0
0.0
0.4
0.6
0.8
1.0
0.2
0.4
0.6
0.8
1.0
s=4
s=255
Figure 13: Rejection rates for the Reprompting method on the KTH watermark with E (1,4,256). The solid lines correspond to E-watermarked text and the dashed lines to Distillation-spoofed text.
Given text w E LT, we naturally generalize Eq. (4) by defining E R as loggwr Two whereas previously was the color of the f-th token. The rest of the method remains identical.
We evaluate both the FPR and TPR of our test using h E (3,4), 4). LLAMA2-7B as both the watermarked model and the attacker model, the Reprompting method, and the same experimental procedure in $5, except that we generate only 500 completions. We discarded h = 2 as the watermarked model output was too low-quality and repetitive (Gu et al., 2024). In Figure 12, we see that the generalized method can successfully detect spoofed text with 90% TPR at rejection rate of 1% for 500 tokens. In fact, it is even more powerful than the detection in the Red-Green scheme, where we achieved a similar TPR at I % with 3000 tokens ($5.2). However, the test hypothesis appears slightly violated, as the empirical FPR at 1 1% is around 2% for both h = 3 and h
KTH watermark In the KTH watermark (EXP variant), single watermark key sequence of length Tkkyys { 5!....,5hie,, is uniformly distributed, where each g² E 1] To generate the j-th token (modulo nkkey), the wa- termark samples the token i" that maximizes E
l/p.
Additionally, to allow more diversity in the generated text, the key randomly shifted by a constant at each query. As in Gu et al. (2024), we denote by s the number of allowed shifts.
Given text ΣT, naturally generalize Eq. (4) by defining the color of token f.. To account for the permutation of the key. we further replace log(1 - S.. with the Levenshtein cost introduced in Kuditipudi et al. (2024). Moreover, the scheme, being based on fixed key, lacks any context h that can be used to compute the N-gram score y/ (Eq. (7)). Following the intuition from Gu et al. (2024) that, in the limit, their spoofing ability comes from learning contiguous watermarked sequences length 7kkey we suspect that setting h R Tkky would enable greater test power. In practice, due practical constraints, we set 5. The rest of the method remains unchanged.
R as x; log(1 S whereas previously x; was
We evaluate both the FPR and TPR of test, using s 4, and 256, along with key of length Tikey 256, LLAMA2-7B as both the watermarked model and the attacker model, the Reprompting method, and the same experimental procedure in $5, except that we generate only can successfully detect spoofed text for both s = 4 and 256, albeit with a TPR of 65% at confidence of 99% for s = 4 and TPR of 30% for s = 256. In all three cases however, the Type error is controlled, i.e., empirical FPR corresponds to the theoretical FPR.
500 completions. In Figure 13, we see that this generalized method
H. Influence of Human Modifications on FPR
Here, we study the behavior of E-watermarked text that has subsequently been edited by humans. We consider two different use cases. The first is the case of cropping. Given g-watermarked text, we assume human inserts non-watermarked text in the middle. This corresponds to a plausible use case of LLMs, where humans merge generated text with their own. Second, we consider paraphrasing. Given watermaarked text, we paraphrase it using DIPPER (Krishna et al., 2023).
We evaluate the FPR of numan-modifin text using on both the Standard and Reprompting methods and follow
18

---

Discovering Spoofing Attempts Language Model Watermarks
Reprompting
Standard
1.0
0.8
1%
0.6
5%
10%
0.4
20%
0.2
0.0
0.0
0.2
0.4
0.6
0.8
1.0
0.2
0.4
0.6
1.0
a
Figure 14: Experimental rejection rate of mixed E-watermarked text and human text LLAMA2-7B for both the Reprompt- ing method (left) and the Standard method (right) different percentages of human text. Each mixed text is in total 500 tokens long.
Reprompting
Standard
1.0
0.8
0.6
FPR
500
0.4
1000
0.2
0.0
0.0
0.2
0.4
0.6
0.8
1.0
0.0
0.4
0.6
0.8
1.0
a,
a,
Figure 15: Experimental rejection rate of paraphrased g-watermarked text LLAMA2-7B for both the Reprompting method (left) and the Standard method (right).
similar experimental procedure as in $5, except that we generate only 500 samples. Given a percentage p. for each generated C4 prompt completion of length T, randomly insert another random human text sampled from C4 such that p percent of the resulting text is suman-generated. We used this procedure for p {0.01, 0.05, 0.1, 0.2). As in $5, apply the test only text that appears watermarked according to the original watermark detector. In Figure 14, we see that even for the highest percentage of human text (20%), the test properly controls Type error.
We evaluate the FPR of the paraphrased text using h = 3 on both the Standard and Reprompting methods and follow a similar experimental procedure as in $5, except that we generate only n = 1000 samples. We note that we apply the test only on text that is considered watermarked by the original watermark detector. In Figure 15, see that the test still properly controls Type error for both methods and for different text lengths.
Both results show that rejection rate of a still guarantees an experimental FPR of even if the w-wtermaarked texts have been altered by humans.
I. Proof of Lemma 4.1
In this section, we detail the proof of Lemma 4.1. First, let's recall some statistical results that need.
19

---

Discovering Spoofing Attempts Language Model Watermarks
Theorem I.1 (Lindeberg CLT). Let X Xn,15.. my Xnn be independent random variables in I& with mean zero. foro all
0
BlllX = 12 =}] 0, (Lindeberg Condition)
(12)
and
n
cool (Xn,k) + V,,
(13)
then
Xn,k 4 N(O.V)..
(14)
Theorem 1,2 (Delta method). Let X1...., Xn be sequence of random variables in Rd,
Vn(X,, 11) 4 V).
(15)
and u Rd + R is differentiable at μ. with Vu(u)= + 0, then
VT(u(X,)) u(n)) 5 N00, Vuln)")Vu(())
(16)
Now we proceed to prove Lemma 4.1. We first state the result formally.
Lemma 4.1. Let == ; XT be a sequence of independent (non i..d) Bernoulli random variables, and gi
P(X, 1). Let Y = Y1,...,y; Yr be a sequence of ii.dd random variables. Let (X, Y). Assuming that, for all E {O....,T) T}, X; and Y, are independent, that there exist g .g(²) E [0, such that
lim =
1
T - - (1))
=0 -0))==
(0 and lim
T Σg
96
(17)
T-toc T
T-
T
1=1
i=1
and assuming that Y admits at least 4 moments Jy, My,, My., 4. Then, we have that
Zs(It) VTS(2) 4 N (0.1). (w (1m11)/u11) Vn
Proof. Let wi == (X1, Yú, X?, XY,)).
X.Y.). Let X,,k
We recall the definition of 5,
EL, XY)((, Pr)
S(22)
(18)
ET- (X, 87)² Etr (Y, (Y,- Y7)²
+ X,Y, - (+ Et x,) ( Y,)
(19)
T' H - X² (+ x1) 1 + Y? (+ +t Yi
where Xy denotes the mean of XI.T-
The proof goes as follows:
First, we show that the sum of the covariance matrix X,,k converges (Eq. (13)).
Then, we show that X,. satisfies the Lindeberg condition (Eq. (12)). We can then apply the Lindeberg theorem to show that w; converges to a normal distribution.
Finally, we apply the Delta method (Theorem 1.2) to show that S(2) is normally distributed.
20

---

Discovering Spoofing Attempts Language Model Watermarks
We have that for all i + j. wy is independent of wy. For each i, we have
9:(2-9:)
9x(1-9))
pyg((1-si)
" tpya
a a(-ri-tays
Cov(w;)
(111)))
9x(1-9:)
0
(
-AY ty
(ayz)
a m(-MypyatRys)
(1-0:)) oi 0:(-u3++t tuys) pr oi( (-arrayz*aya) 9: (=(rpgrttya))
where we denote Mys the k-th moment of Y.
Then, using Eq. (17), have that 1/T El= Cov(w;; =/== Cov(Xn..) converges towards V R5x5 defined
g (
0
gD)
o
By -g 9 (-ur) Hv (x
g (2)
0
-
0
0 Hy
-(yrs)*+aya (-uyuvvatyva))
(- Myz taya) -HEs +1y2g
" (a ())
(-18 My ()
We have completed the first step of the proof.
Now we want to show that X,, satisfies the Lindeberg condition (Eq. (12)). Let > 0. Because X,, Y, E [0, we have that for all i S n, X S V 10 n, 1 {1] x > }} = 0. So, for all n 2 no.
n. There exists no > 0 such that Vn 2 no, V <. Therefore, Vn 2 no. vk: S
10
E 11
<}| 0.
(20)
Hence, we have shown that for all e > 0,
El E1]lx,,11110 11 > e}] + O.
(21)
Therefore, using the Lindeberg CLT (Theorem I.I), have that
TT Σ (w, - B(w))) 4 (O.
(22)
We have completed the second step of the proof. Now, we want to apply the Delta method (Theorem 1.2) to show that S(w) is normally distributed.
Let pw : lim
1/T
E
[w:] = (g. Ny2, g,y)) We introduce
T
E =
I3 [wo
Ma
(23)
VT
R [ws]]
(24)
T
1=1
0
(Using Eq. (17)).
(25)
Therefore, we have
1
(w: pmw)
(w: E((w:)) F, &(0,
(26)
VT
1=1
Let u: 5 + R be defined as
u(x))
25 1182
(27)
/ 2 (x0s 23
21

---

Discovering Spoofing Attempts Language Model Watermarks
We have that S(Q) = u (1/TE wi) (using Eq. (19)) and u(Hw) = 0, and therefore using the Delta method (Theo- rem I.2) we have that
VTS(1) S N (0. Vu(pw) (oolp)""VVll VVu((u)))
(28)
Because Vu(uw) VVu(u)))) = 1, we have shown that
VTS(2) N(0,
(29)
J. Extended Discussion of the State of Watermark Spoofing
In this section, we overview of the state of the field of watermark spoofing to further motivate our work and highlight its practical implications. In App. J.1, we identify three categories of spoofing techniques and highlight learning- bearning-based spoofing techniques as the most practically relevant. We put our findings in this context, discussing the potential for adaptive spoofing that does not leave artifacts in the spoofed text. In App. J.2 we discuss how latest schemes attempt to tackle the issue of spoofing.
J.1. Approaches spoofing
Learning- based spoofing As explained in $1, learning-based spoofing operates in two phases. In the first phase, the spoofer queries the model generate dataset D of {-watermarked text. From this dataset DD the spoofer learns the watermark, which allows them to generate spoofed text. In the second phase, using their knowledge and a private LM, the spoofer can generate arbitrary watermarked text al scale, without having to query the original model again. In particular, spoofed texts can be created as answer to any prompt, even the one that would be refused by the original LLM, which gives learning-based spoofers great flexibility, and illustrates the potential threat they pose. Additionally, as long as the cost of the first phase is reasonable, learning- based spoofing is cost-effective, as the subsequent per-spoofed- cost zero. Learning-basee spoofing includes the works of Jovanović al. (2024): Gu al. (2024); Zhang et al. (2024).
Piggyback spoofing As discussed in $2, the second family of spoofing techniques is piggyback spoofing, introduced by (Pang et al., 2024), which directly exploits the desirable robustness property of the watermarks. Given a E-watermarked sentence, the attacker modifies few tokens to alter the meaning of the original sentence while maintaining the watermark, interpreting the result an instance of spoofing. While illustrating the potential drawbacks of high robustness, this comes with several caveats. First, abusing the robustness of the watermark naturally raises the question of the boundary between spoofed text and edited &- g-watermarked text. Indeed, mixing human and LM text is a realistic use of LMs, and it is agreed that watermarks should account for this use (Kirchenbauer et al., 2023; Kuditipudi et al., 2024). Second, piggyback spoofing is limited in the scope of text it can generate, as it relies on the original model to generate the majority of the text. This greatly reduces the flexibility of the attack, i.e., does not allow the attacker to generate texts on harmful topics that would be refused by the watermarked model. Finally, the same property makes the cost of spoofing scale with the number of spoofed texts, as the attacker needs to query the original model each time.
Step-by-step spoofing Finally, as also briefly mentioned in $2, a third category of spoofing techniques is step-by sep-by-step spoofing. This line of works considers spoofing techniques that require queries at each step of the generation process of every spoofed text (Pang et al., 2024; Zhou et al., 2024; Wu & Chandrasekaran, 2024), using the feedback obtained this way to choose the next token. While they have higher flexibility compared to piggyback spoofing, key limitation of these techniques is the high cost, even compared to piggyback spoofing. Further, some of these methods assume access to the watermark detector itself (sometimes also its confidence score) to obtain the desired feedback, which is not always realistic. For instance, in the case of the first public large-scale deployment of a watermark, SynthID-Text, Google does not provide public access to the watermark detector.
Summary In summary, learning- based spoofing is the most practically relevant category of spoofing techniques, as it is eoft-effective, flexible, and does not require querying the original model for each spoofed text. Another advantage from the perspective of our research question is the fact that current learning- based methods are based on fundamentally different principles, making the question of their common limitations relevant and interesting. In this work, we study that question,
22

---

Discovering Spoofing Attempts Language Model Watermarks
showing that all learning-based spoofers leave visible artifacts in spoofed text, which can be leveraged distinguish between spoofed and g-aatemaarked text.
J.2. Spoofing-aware watermarking schemes
The field of watermarking is evolving rapidly, as explained in $2, with different schemes proposed in the literature. We distinguish two approaches to watermarks in LMs. The first one is the statistical approach, notably including schemes from Kirchenbauer al. (2023); Kuditipudi et al. (2024); Aaronson (2023), which place great emphasis watermark robustness and practicality. The second is the cryptographic approach, with schemes stemming from Christ et al. (2024b), which focus primarily on watermark security and rigorous guarantees.
In particular, schemes with cryptographic features have not been shown to vulnerable to spoofing attacks. Yet, they enhance security by trading off other key watermark properties, such as robustness watermark removal. Moreover, recent robust
work (Zhou et al., 2024) suggests merging both fields to create watermarking scheme that is not only watermark removal but also watermark spoofing. However, they show that their approach trades off with generation quality. This highlights that, from the perspective of a model provider, there is no single scheme that is the most desirable. Hence, choosing particular scheme is complex task that involves navigating the tradeoffs between different properties. From this perspective, our work provides new evidence that schemes derived from (Kirchenbauer et al., 2023) are harder to spoof than previously thought (as these attempts can be detected by observing the artifacts), and can help model providers adjust their expectations.
23


Reviews:
[{"review_id": "89SlMNdqyy", "reviewer": "Reviewer_Ai6u", "summary": "This paper investigates whether learning-based attacks that attempt to spoof watermarking schemes in language models leave detectable artifacts in the generated text. The authors propose a statistical method to distinguish genuinely watermarked text from spoofed text by modeling the relationship between the observed watermark color sequence and the frequency of (h+1)-grams in the spoofer’s training data. They introduce a reprompting-based approach that generates new text samples to approximate the expected correlation in genuine watermarked text, enabling reliable detection of spoofed text.", "questions": "", "limitations": "", "rating": 4, "confidence": null, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "We thank the reviewer for their positive feedback and address their individual questions below. We have attached two additional figures [here](https://drive.google.com/file/d/1-bFmwwrN_kGELtsuhMUTbU4gOBB-UBru/view).\n\n**Q1: Can the authors explicitly analyze the distribution shift between Reprompting with or without the original prompts?**\n\nWe thank the reviewer for their suggestion and included a new experiment where we compare Reprompting with and without using the original prompt on Stealing with $h=1$.\n\nWe see in the attached Figure 1 that estimating the prompt indeed consistently slightly decreases TPR (at worst a 5% TPR at 1% FPR decrease). This suggests that, for non-adversarial cases, the drift incurred by using the beginning of the received text as a proxy for the prompt has a minimal impact on our detection accuracy.\n\nWe will add this experiment to the next revision of our paper.\n\n**Q2: Could the authors use a fixed corpus of LLM generated text from the same domain as the received text instead of Reprompting?**\n\nIntuitively, using a corpus of LLM-generated text whose correlation follows the distribution of Eq. (5) would allow us to estimate the mean and perform the test. However, in practice, it is hard to design a general rule establishing what the mean depends on (e.g., is it the topic of the text, is it the type of words used…).\n\nTo strengthen our point, we show in the attached Figure 2 an experiment with Distillation, $h=2$, where we estimate the mean and variance using a $\\xi$-watermarked text corpus generated from OpenWebText completions. We then apply Eq. (6), where we replace $S(\\omega’)$ with the estimated mean. We see that the FPR is higher than what we would expect, suggesting that the estimated mean does not capture the true mean. This is why we argue that Reprompting, albeit more costly, is more reliable.\n\nWe will explicitly include this discussion in the paper.\n\n**Q3: Is the method prohibitively expensive for real world usage?**\n\nWe do not think so. As spoofing detection is more targeted than watermark detection—it can be run only when one suspects spoofing—performance issues are, by nature, less critical. We also note that, for some settings ($h=3$), the Unigram score is applicable and doesn’t require additional text generation.\nYet, we agree that our work is a first step in detecting spoofing attempts on tested schemes, and we believe that making the test more efficient is a worthwhile direction for future work.", "claims_and_evidence": "Yes.", "methods_and_evaluation_criteria": "Yes.", "theoretical_claims": "None. No theoretical claims in this paper.", "experimental_designs_or_analyses": "Yes. I checked all parts.", "relation_to_broader_scientific_literature": "This paper contributes to the broader scientific literature on watermarking for generative models, adversarial attacks against watermarking, and statistical detection of manipulated text. It builds on prior work in probabilistic watermarking (Kirchenbauer et al.) by demonstrating that learning-based spoofing attacks (Jovanovi´c et al.; Gu et al.) introduce detectable artifacts, aligning with forensic linguistics and stylometry research. The study’s reliance on statistical correlation tests and reprompting-based detection parallels prior work in AI-generated text forensics, reinforcing the challenge of forging watermarks without introducing anomalies.", "essential_references_not_discussed": "No.", "other_strengths_and_weaknesses": "Pros:\n\n1. This paper is novel and well-organized. It introduces a new statistical approach for detecting spoofed watermarked text, highlighting the challenge of forging watermarks without introducing anomalies.\n\n2. This paper uses well-defined statistical tests to evaluate spoofing artifacts. This provides a formal framework that could be extended to other watermarking and detection settings.\n\nCons:\n\n1. An important issue is that since the detector receiving the text does not have access to the original prompt, this paper reprompts the model using only a prefix of the received text rather than the original prompt used for generation. This approach may introduce distributional drift, potentially making the reprompted text an imperfect control sample. The extent of this drift in their method is not explicitly analyzed, raising concerns about its impact on detection accuracy. A comparison with prompt-preserving reprompting should be conducted to better assess the validity of the proposed approach and strengthen the paper’s conclusions. Additionally, why not involve using LLM-generated text from a similar domain as the detected text to serve as a baseline directly? \n\n2. The method requires additional text generation to establish a statistical baseline, and it should be very expensive for large-scale detection systems.", "other_comments_or_suggestions": "No.", "questions_for_authors": "See Weaknesses.", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "6UvldrlcP6", "reviewer": "Reviewer_nCP1", "summary": "One use of watermarking schemes for generative models is for attribution. In recent years there have been several \"spoofing\" attacks on watermarking schemes which can make a certain piece of text to appear as if it was generated by a certain model by \"copying\" the watermark associated with that model. This paper comes up with an empirical technique to detect such spoofing attacks.", "questions": "", "limitations": "", "rating": 3, "confidence": null, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "$\\newcommand{\\D}{\\mathcal{D}}$$\\newcommand{\\T}{\\tilde{\\D}}$We thank the reviewer for their helpful feedback and address their individual questions below.\n\n**Q1: Does the proposed method require a good estimate of the spoofer's training corpus in order to be applicable?**.\n\nA reasonable estimate is enough for the detection to maintain sufficient power. For this, we have already included two experiments in App. D (Figure 8) to ensure that the detection remains accurate despite a weaker estimate of the spoofer’s training corpus.\n\nLet $\\D$ be the spoofer's training corpus and $\\T$ the estimate of such a corpus. For the first experiment, we build several $\\T$ by adding random noise to $\\D$. This allows us to control the distance between $\\T$ and $\\D$ and observe how the power decreases with the distance. For the second experiment, we show that for realistic choices of $\\T$ (e.g., C4, Wikipedia), the power of the test remains very similar to the best-case scenario of $\\T = \\D$. This means that ultimately a reasonable choice of $\\T$ is sufficient to maintain high power.\n\nImportantly, as we argue in App. D, the detection can select an appropriate $\\T$ from a similar domain as the received text  to be tested. Because the spoofer wants to maximize his chance of successful spoofing, it is reasonable to assume that $\\D$ is from the similar domain as the spoofed text he aims to generate, making it likely that the selected $\\T$ is close to $\\D$.\n\n**Q2: Are the schemes studied by the authors really intended for attribution? Can the authors better motivate the choice of watermarking schemes they study?**\n\nSchemes such as Red-Green, KTH, and AAR have been the focus of multiple previous works considering them for attribution, as shown by spoofing attacks [1,2,3] and multi-bit watermarking [4]. Hence, these schemes have attracted significant attention from researchers exploring broader applications beyond mere detection, including attribution.\n\nFurther, such schemes have practical relevance as shown by the deployment of Google Deepmind’s SynthID, the first publicly acknowledged deployment of LLM watermarks. We therefore argue that their prominence and practicality motivate further studies on their real-world security.\n\nYet, we agree with the reviewer that making a clear distinction as to whether schemes are explicitly designed for attribution is particularly relevant. Based on the reviewer’s suggestion, we will expand our introduction by explicitly discussing watermark attributability by design as well as its current treatment in the research community.\n\n**Q3: Does detecting spoofing attempts lead to a “cat and mouse” game? Isn’t it better to design schemes explicitly designed for attribution?**\n\nWe agree with the reviewer that, if the desire is to build an un-spoofable scheme, designing a scheme specifically for attribution, such as [5,6], is the best practice.\n\nYet, for practical use, practitioners desire multiple, often conflicting, properties: quality, practicality, security, and robustness. Schemes that are harder to spoof might compromise too heavily on these other desirable properties. Importantly, our work indicates that previously successful spoofing attacks on prominent schemes actually are detectable—ultimately making it harder for real-world adversaries to attack such schemes. Further, especially with the increasing work in watermark spoofing on these schemes, we see our work as a helpful contribution to the field by providing general insights into the properties and limitations of learning-based spoofers.\n\nWhile we ultimately agree that providing an un-spoofable scheme that achieves all of the above properties would be the optimal solution, we see a lot of value in (1) exploring the properties of popular and used watermarks and (2) raising the bar for successful attacks which can yield practical real-world benefits.\n\n**Q4: Can the authors provide more theoretical insights for Eq. (5)?**\n\nRigorously proving Eq. (5) would require further assumptions about the dependence between $X$ and $Y$. With Lemma 4.1, we assumed independence to justify the asymptotic normality, and with the Unigram score, we provided intuitive reasons why this independence assumption might hold in practice.\n\nFor the general case, we show in Appendix C that the independence is violated. While a more general assumption about the dependence structure could allow us to prove Eq. (5), we believe that this complicates the problem modelling without providing significant value to our contribution (as we would still likely rely on empirical evidence to justify our dependence structure assumption). Thus, as noted by the reviewer, we opted for a more direct approach by providing empirical evidence to support Eq. (5).\n\n[1] https://arxiv.org/abs/2402.19361 \\\n[2] https://arxiv.org/abs/2312.04469 \\\n[3] https://arxiv.org/abs/2405.19677 \\\n[4] https://arxiv.org/abs/2308.00221 \\\n[5] https://arxiv.org/abs/2402.09370 \\\n[6] https://arxiv.org/abs/2310.18491", "claims_and_evidence": "Yes", "methods_and_evaluation_criteria": "Yes, although the modeling assumption of Eq. (5) seems somewhat arbitrary and there doesn't seem to be clear theoretical reasons presented to back it up (though sec 5.1 gives some empirical evidence)", "theoretical_claims": "The paper is empirical in nature.", "experimental_designs_or_analyses": "Yes, seem valid", "relation_to_broader_scientific_literature": "One weakness of the paper in relation to the broader scientific literature is that it is not clear that watermarking schemes of many past works are really *intended* to be used for attribution. There are certain papers which specifically study watermarking schmes that are intended to be used for attribution (e.g., http://arxiv.org/abs/2310.18491) in the sense that they should be hard to spoof. Studying spoofing attacks on watermarks which weren't designed to be hard to spoof doesn't seem to be very well-motivated. Accordingly, the paper should make a clearer distinction between watermarking schemes that are designed to be used to publicly attribute text to a model (vs just detecting that the text was produced by the moel -- this distinction is made clear in e.g. https://arxiv.org/abs/2402.09370).", "essential_references_not_discussed": "See \"strengths/weaknesses\" below.", "other_strengths_and_weaknesses": "Strength: The method is based on a natural/simple idea, namely that spoofing attacks generally can only know the red/green tokens for h-grams which they have seen in their \"training corpus\", and so we can detect spoofing attacks by looking at which h-grams are \"spoofed\" vs which ones appear in the training corpus.\n\nWeakness: Accordingly, a major weakness of the paper is that (for the (h+1)-gram score method) it requires some good estimate of the \"training corpus\", which in general is hard (this paper uses C4, which may work for some use cases, but not others).\n\nWeakness: It's not so clear that detecting spoofing attacks is really the right way to go about things: it sort of contributes to a \"cat and mouse\" game, since one can design spoofing attacks which are harder to detect. The more direct way to go about things seems to be to design watermarking schemes which are harder to spoof.\n\n*** Update: thanks for your responses, which addressed some of my concerns. I have updated my score. ***", "other_comments_or_suggestions": "NA", "questions_for_authors": "See above.", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "PfiDVEcnP7", "reviewer": "Reviewer_p27M", "summary": "This paper proposes a statistical test for identifying spoofing attacks against sampling-based LLM watermarks. The method is based on the intuition that the frequency of watermark violation conditioned different n-grams in spoofed texts would be different with that in actual watermarked texts. The result shows that the statistic can indeed have a good performance in spoofed text detection.", "questions": "", "limitations": "", "rating": 3, "confidence": null, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "We thank the reviewer for their helpful feedback and address their individual questions below. We have attached one additional figure [here](https://drive.google.com/file/d/1Rh3JLVob1UuV2-ElQ7_QgnDNi2CZ-mvb/view).\n\n**Q1: Can the current spoofing methods be *easily* adapted to bypass the proposed detection? For instance, can we constrain the h-gram distribution in the spoofer’s training data?**\n\nWe do not believe so, as we are tackling a fundamental limitation of learning-based spoofers that is not specific to a given implementation. Because the spoofer learns from a finite, small training dataset (for cost reasons), it cannot “see” every h-gram combination and hence perfectly reproduce the watermark. Enforcing uniformity is not possible, as there are roughly O($\\Sigma^{h-1}$) h-gram combinations (for a fixed last token), which means that a uniform dataset needs an exponential amount of tokens.\n\nIn particular, for Stealing, their authors explain that the sparsity of the training data was the main challenge behind their attack, and they had to discard low-frequency h-grams for stability [1]. Enforcing additional constraints on the training data, such as trying to get a distribution of h-grams as uniform as possible, would only reinforce the sparsity issue, ultimately hurting spoofing performance.\n\nFor Distillation, we show in App. E Figure 9 that increasing the spoofer's training data by 10x only reduces our method's TPR by 15\\%. Further, enforcing constraints on the distribution of the data in that case might significantly degrade the spoofer model's performance. Because the token distribution would deviate further from human text, it might hinder the finetuning process.\n\nOverall, while we cannot provide guarantees against (arbitrary) adversarial attacks, our method practically increases the amount of effort needed by an adversary for successful spoofing—which we argue is overall  beneficial.\n \n**Q2: Does there exist an adversary that could bypass the proposed method?**\n\nAs briefly discussed in Section 2 and Appendix I, we believe that a step-by-step spoofing attack, combined with the idea of color-aware substitution [2], would result in a spoofing attack that is not detectable by our method. We note that such an adversary would (in the limit) be able to accurately estimate the Red-Green splits for any context, resulting in a watermark signal that is distributionally indistinguishable from the genuine watermarking algorithm.\n\nHowever, such a method would require querying the watermarked model for *every* token generated by the spoofer, rendering it prohibitively expensive and impractical. Hence, we argue that, given the current state of the field, there exists no spoofing method that could leverage the knowledge of our statistics to bypass our method while maintaining similar properties (cost and practicality) to current learning-based spoofers.\n\n**Q3: Why do the authors restrict themselves to older models?**\n\nWe evaluate our method on older models to match the experimental setup from the spoofing attacks papers. Nonetheless, in response to the reviewers comments, we evaluate Stealing with $h=3$ using *Llama3-8B* as the watermarked model and *Qwen2.5-7B* as the spoofer model, using the Reprompting method (see attached Figure 1). We see that our test is equally valid with newer models as well, with the FPR being properly controlled (solid line) and the TPR high (dashed line).\n\n[1] https://arxiv.org/abs/2402.19361 \\\n[2] https://arxiv.org/abs/2403.14719", "claims_and_evidence": "The key contribution claimed by the paper includes the in-depth analysis of artifacts in spoofed texts, statistical tests to distinguish spoofed texts and empirical evaluation on the proposed methods. These claims are supported by the theoretical analysis and empirical evaluations in the paper.", "methods_and_evaluation_criteria": "The key intuition behind the method is that the spoofing attack will only learn the vocabulary split based on existing corpus, so that the frequency of watermark violation in the spoofed text, conditioned on the different n-grams, will be different with that in the actual watermarked text. This idea is novel and makes sense for the spoofing attack with stealing and distillation (in the case of distillation the learning is also based on the frequency of different n-grams). Nevertheless, it is worth noticing that the it is limited to certain spoofing attacks and may not work against, for example, generating watermarked text by the model and modifying it manually to get spoofed texts.", "theoretical_claims": "I did not check the mathematical details of the proofs, but the they make intuitive sense to me.", "experimental_designs_or_analyses": "The model evaluates the spoofed text detection performance with two different techniques (stealing and distillation) on different models. The experiment designs and analyses look good to me. One minor concern is that the used models are rather older models (Llama2, Gemma) and there have been more advanced models in recent six months (Llama3, Gemma2).", "relation_to_broader_scientific_literature": "This paper is among the first to study the spoofing attacks in LLM watermarks and propose a statistic for detection. This helps with the robustness of LLM watermark to be applied in practice.", "essential_references_not_discussed": "As far as I know, the references are adequately discussed.", "other_strengths_and_weaknesses": "The paper does not discuss the possibility of adversarial attacks, where the adversary knows the idea of this statistic and aim to bypass it. I think that it would not be difficult for a knowledged adversary to bypass the statistic, for example, by enforcing the n-grams appearing in the spoofing learning process to be uniform.", "other_comments_or_suggestions": "N/A", "questions_for_authors": "Do you have any mitigation if the adversary knows your statistics and intentionally bypasses it?", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}]

Author rebuttal:

Meta-review: This paper tackles the problem of spoofing attacks on sampling‑based watermarks for LLM‑generated text. In this setting,  an adversary forges a valid watermark on arbitrary text. Rather than hardening watermark schemes, the authors take the complementary approach of post‑hoc forensic detection.The paper have observed that, modern spoofers  inevitably learn watermark “red/green” token splits only for n‑grams seen in their limited training corpus, leaving detectable distributional gaps on unseen contexts. With this observation, the authors  formalize a simple hypothesis test based on the correlation between watermark bit patterns and (h+1)‑gram frequencies in the spoofer’s training data, estimating the null (“genuine”) distribution via reprompting from the suspected watermarking model. The authors have shown that across multiple watermark schemes (Red‑Green, KTH, AAR) and spoofing attacks, the detector achieves high true‑positive rates at low false‑positive rates, even when using imperfect corpora or only the text prefix as a reprompt surrogate. All 3 reviewers liked the paper’s post‑hoc detection perspective, its statistical formulation, and its strong empirical performance across varied spoofers. Therefore, it is reasonable to accept the paper as is.
Decision: Accept (poster)