Can Watermarked LLMs be Identified by Users via Crafted Prompts?
Authors: Aiwei Liu
Sheng Guan
Yiming Liu
Leyi Pan
Yifei Zhang
Liancheng Fang
Lijie Wen
Philip S. Yu
Xuming Hu
Venue: ICLR 2025 Spotlight (2025)
Keywords: Large Language Models
Watermark
Identification
Primary area: alignment, fairness, safety, privacy, and societal considerations
TL;DR: We present Water-Probe, a method to identify watermarked LLMs using crafted prompts, and propose the Water-Bag strategy to improve watermark imperceptibility.

Abstract: Text watermarking for Large Language Models (LLMs) has made significant progress in detecting LLM outputs and preventing misuse. Current watermarking techniques offer high detectability, minimal impact on text quality, and robustness to text editing. 
    However, current researches lack investigation into the imperceptibility of watermarking techniques in LLM services.
    This is crucial as LLM providers may not want to disclose the presence of watermarks in real-world scenarios, as it could reduce user willingness to use the service and make watermarks more vulnerable to attacks. This work is the first to investigate the imperceptibility of watermarked LLMs. We design an identification algorithm called Water-Probe that detects watermarks through well-designed prompts to the LLM. Our key motivation is that current watermarked LLMs expose consistent biases under the same watermark key, resulting in similar differences across prompts under different watermark keys. Experiments show that almost all mainstream watermarking algorithms are easily identified with our well-designed prompts, 
    while Water-Probe demonstrates a minimal false positive rate for non-watermarked LLMs. 
    Finally, we propose that the key to enhancing the imperceptibility of watermarked LLMs is to increase the randomness of watermark key selection. Based on this, we introduce the Water-Bag strategy, which significantly improves watermark imperceptibility by merging multiple watermark keys.

Published as a conference paper at ICLR 2025
CAN WATERMARKED LLMs BE IDENTIFIED BY USERS VIA CRAFTED PROMPTS?
Aiwei Liu', Sheng Guan?, Yiming Liu', Leyi Pan', Yifei Zhang'
Liancheng Fang', Lijie Wen' Philip S. Yu" Xuming Hu
Tsinghua University "Beijing University of Posts and Telecommunications
The Chinese University of Hong Kong University of Illinois Chicago
5 Hongkong University of Science and Technology (Guangzhou)
1 Liuaw20@mall ls.
wttpmmaisssssssssssiiteeie tsinghua..... eewe..uaaeeddeeem edu. cn, guansheng20228bupt.. edu. cn, wenlj@tsinghua.co edu. cn
[[offiiia]] http::: //gi thub. www./THH--w//Waaee/eaeoeae
LLM Identi fication
ABSTRACT
Text watermarking for Large Language Models (LLMs) has made significant progress in detecting LLM outputs and preventing misuse. Current watermarking techniques offer high detectability, minimal impact on text quality, and robustness text editing. However, current researches lack investigation into the imperceptibility of watermarking techniques in LLM services. This crucial as L.MM providers may not want disclose the presence of watermarks in real-world scenarios, as it could reduce user willingness to use the service and make watermarks more vulner- able attacks. This work investigates the imperceptibility of watermarked LLMs. We design the first unified identification method called Water-Probe that identi- fies all kinds of watermarking in LLMs through well-designn prompts. Our key motivation is that current watermarked LLMs expose consistent biases under the watermark key, resulting in similar differences across prompts under different watermark keys. Experiments show that almost all mainstream watermarking algo- rithms are easily identified with our well-designed prompts, while Water-Probe demonstrates a minimal false positive rate for non- watermarked LLMs. Finally, we propose that the key to enhancing the imperceptibility of watermarked LLMs increase the randomness of watermark key selection. Based on this, we introduce the Water-Bag strategy, which significantly improves watermark imperceptibility by merging multiple watermark keys.
INTRODUCTION
The rapid advancement of large language models (LLMs) has led to remarkable achievements in tasks such as question answering (Zhuang et al., 2024), programming (Jiang et al., 2024b), and reasoning (Wei et al., 2022), with widespread applications across various scenarios. However, the extensive use of L.LMs has also raised concerns regarding copyright protection and misuse. Recent research indicates that malicious attackers can steal LLMs through model extraction techniques (Yao et al., 2024), and some users may abuse LLMs to generate and spread harmful information (Wei et al., 2024).
Text watermarking techniques for LLMs have become an important method to mitigate the above issues by adding detectable features to LLM outputs (Liu al., 2024b). Recent researches on LLM watermarking have focused on improving watermark detectability (Kirchenbauer et al., 2023a), minimizing impact on generated text (Aaronson & Kirchner, 2022), and enhancing robustness against work has considered the imperceptibility of watermarked LLMs, i.c., whether users can know an LLM service is watermarked. In real-world scenarios, LLM service providers may not disclose the existence of watermarks, as it could reduce user willingness to use the service and make it more vulnerable to attacks (Sadasivan et al., 2023). As more LLM services consider implementing watermarks, it is crucial to investigate whether users can identify watermarked LLMs solely through crafted prompts.
text modifications (Liu et al., 2024a). However,
Coorreponniigg author

---

Published as a conference paper at ICLR 2025
Key
Key
Prompt 1
LLM
LLM
Prompt 2
Name country with a large population.
2 Dina
5 Rusia
Linds
High Similarity
Watermarked
1 Irda
(Chia
RRssi
Name a rountry with large
Key
LLM
Repeated Sampling Watermark Key
Consistency
Repeated Sampling
Key
Figure 1: Illustration of our Water-Probe algorithm for identifying watermarked LLMs. We first construct two prompts with similar output distributions, then sample repeatedly using two fixed watermark keys for each prompt. The presence of watermark is determined by comparing the similarity of distribution differences between the two prompts. Details in $3.
Some studies focus on the imperceptibility of watermarked text, ensuring watermarked and non- watermarked texts are indistinguishable (Hu al., 2023; Wu et al., 2023b). However, even if individual watermarked texts are imperceptible, the distribution of numerous watermarked texts may reveal whether the LLM watermarked, especially when repeatedly sampling with the same watermark key (Wu et al., 2024). While some studies explore cracking watermarks using large volumes of watermarked text (Jovanović al., 2024; Sadasívan et al., 2023; Wu & Chandrasckaran, 2024), they assume the LLM is watermarked and cannot determine if the LLM is watermarked. The most relevant work is Gloaguen et (2024), which proposes a black-box detection method for watermarked LLMs. However, their approach uses different detection methods for different watermarks and cannot effectively detect all watermarking techniques.
In this work, propose Water-Probe, the first unified method for identifying watermarked LLMs that can detect all types of watermarks embedded during the L.LM's text generation process. (see related work section for this type of watermarking) Our motivation stems from key observation: all current LLM watermarking algorithms expose consistent bias when repeatedly sampled under the same watermark key. Based on this, Water-Probe algorithm first crafts prompts to perform repeated sampling under the same watermark key, then compares the consistency of sampling distribution differences across different prompts under pair of watermark keys. Highly consistent differences indicate a watermarked LLM.
In experiments, demonstrate that the Water-Probe algorithm achieves high accuracy in detecting various types of watermarked L.MMs. We also show its applicability across different L.MMs, maintaining a low false positive rate for non-watermarked LLMs. Furthermore, our algorithm exhibits robust performance across different sampling methods and temperature settings.
Finally, we explore methods to enhance the imperceptibilityy of watermarked LLMs. We find that increasing the randomness of watermark key selection is crucial, as it makes it more difficult construct prompts for repeated sampling using the same key. Based on this. we propose the Water-Bag algorithm, which combines multiple watermark keys into one, randomly selecting key for each generation and choosing the highest score for detection. Although increasing key selection randomness often leads to slight decrease in detectability, it significantly enhances the imperceptibility watermarked LLMs. Addressing this trade-off should be an important direction for future work.
Our main contributions are summarized as follows:
We propose Water-Probe, the first unified algorithm that can detect various types of wa- termarked LLMss by analyzing the consistency sampling distribution differences across different prompts under fixed watermark keys.
Through extensive experiments, demonstrate that Water-Probe achieves high detection accuracy across different LLMs, watermarking methods, and sampling settings, while maintaining low false positive rate for non-waterrarred L.LMs.
2

---

Published as a conference paper at ICLR 2025
We introduce Water-Bag, novel algorithm that enhances LLM watermark imperceptibility by combining multiple watermark keys, and analyze the trade-off between watermark detectability and imperceptibility.
PRELIMINARIES
Definition (Large Language Model). An LLM M is a function that, given an input x and a partial output sequence V1---1, produces probability distribution (sr V1---1) over possible next tokens yi. The model generates complete oulputs by iterativeyy sampling from these distributions.
Since this work focuses solely on watermarks embedded LLM services, we will only consider the watermark during generation type mentioned in $6. The definition of the watermark rule given below.
Definition 2 (Watermark Rule). A watermark rule is typically a function that adjusts the current L.L''s predicted probability distribution based a watermark key k to obtain a new probability distribution. Formally, given an LLM PM and a key k, the watermark rule F' modifies the distribution as follows:
Pe k)) F(PM (Ua (r, Dra 1),
where PF is the modified probability distribution for the next token ye-
The main difference between watermarking algorithms lies how they determine the watermark key. Based on this, we categorize watermarking algorithms into n-gram based watermarking and fixed-key-list based watermarking. will now introduce these two types of watermarking algorithms.
Definition 3 (N-Gram Based Watermarking). In n-gram based watermarking, the watermark key k; for generating the current token yi is determined by a function that takes two inputs:
k; = f(K, yi 9i-nsi-)
(2)
where is pre-selected master key, and fi-nci-1 represents the previous n tokens.
N-gram based watermarking ensures that for the same n-token prefix, the watermark key for generating the next token remains consistent. This approach is widely used in current watermarking algorithmms, including KGW (Kirchenbauer et al., 2023a), KGW-V2 (Kirchenbauer et al., 2023b), Aar (Aaronson & Kirchner, 2022), DiPmark (Wu al., 2023b), and SIR (Liu al., 2024a). Next, define fixed-key sked-kee-list-based watermarking:
Definition 4 (Fixed-Key-Lin) Based Watermarking). Let K = (ki, km be a fixed key list. For given starting index {1,..., m), the watermark key foror generating the i-th token is:
=(((+i-1)) mod m)+1
where the starting index 8 may be randomly chosen for each generation process.
This approach employed in algorithms such as Exp-Edit (Kuditipudi al., 2023), where keys are used sequentially from potentially random starting position in the key list.
To formalize our approach, we define our goal as follows:
Definition 5 (Black-box Watermark Identification). function PM - {0, 1) that classifies language model PM as watermarked (1) or not (0), without access to its internal parameters.
WATERMARKED LLM IDENTIFICATION
3.1 WHY WATERMARKED LLM IDENTIHCATION POSSIBLE
Definition 2 implies that LLMs typically introduce some distortion the distribution. However, there exist distortion- free watermarking algorithms that satisfy Definition 2. We first provide definition and then demonstrate that L.LMs with distortion-free watermarks can still be identified.

---

Published as a conference paper at ICLR 2025
Definition 6 (Distortion-Fres Watermark). watermarking algorithu is considered distortion-free if. for all possible inputs x and partial output sequences V1:-1. the expected output distribution of the watermarked model PM over all possible watermark keys k. K. is identical to the original model Pas:
BREK [Prs (yi U (r, V1:-11, k)] (M M1--1)-
This equation indicates that the expected output distribution of watermarked remains unchanged when the watermark key is randomly selected across all possible keys. However, as shown in Equation sampling with a specific watermark key & introduces difference between P (ys (2., 3/---1, k) and PM (y (r, 11)-1). This observation leads to the following theorem on detectability of watermarked LLMs:
Observation (Distributional Difference of Watermarked LLMs). Let PM be language model and F watermark rule as defined in Definition 2. For given watermark key k, the probability distribution of the watermarked model Pd (yel x. 11.-1, differs from the original distribution Pa (yll,, 31.-1). This distributional difference suggests the potential detectability of the watermark.
3.2 PIPELINE OF WATERMARKED LLM IDENTIFICATION
Observation implies that the key to identifying a watermarked LLM is to construct prompts that allow for multiple samplings using the same watermark key to reveal the difference between Pe (vilæ,, 11 k) and Par (1s at, (1.i-)). However, due to the black-box setting, we cannot directly access the origin logits PM (ye Ø. 11--1). Instead, we calculate the difference in LLM outputs for two distinct keys, defined as A A(x, Km kn) Pfr km. - Pf (-12,, kn), where we use Pk (-|ar, k) to represent the output distribution.
If two prompts yield similar output distributions, the same watermark key should have similar effects on both prompts (proven in Theorem 1). We determine if an LLM contains a watermark by comparing the consistency of the effects of two watermark keys two similar prompts. Specifically, given X1, J2. k1, and k2, we assess the similarity between A (x1 k1, kz) and A (Ir kı, k2). High similarity indicates the presence of a watermark; otherwise, we conclude there is no watermark. Based on the above analysis, we now present the process of the Water-Probe algorithm.
Step 1: Construct highly correlated prompts. Construct N prompts æt, 12, UN such that their output probability distributions under M are highly similar, which can be expressed as:
Ji.j E (1,2...,,NN KL (PM (x))) Par (+2g)) & and x; + zj
(5)
where KL(-II-) is the Kullback- Leibler divergence, (x(() is the output probability distribution for prompt æ; under the LLM M, and is small threshold indicating high similarity between distributions.
Step 2: Sampling with simulated fixed watermark keys. Since cannot access the logits under given watermark key, we need to use repeated sampling to estimate the distribution. We construct set of simulated watermark keys K {ki, k2 ..., kmm] based our prompt design (detailed
in subsequent sections). For each prompt æ; and each simulated key kj E K, we estimate the distribution follows:
W
= W
Σ 1 1 where yi ~ Pf (v) w=1
(6)
where W is the sample count, 1 is the indicator function, and v", is the w-th sample sampled from Pf ( (r; ky). Specific prompt techniques for different watermarking algorithms will be detailed later. Note that our prompt design for simulating watermark keys assumes the target LLM has a watermark. it doesn't. 't, then P (y) x., A;)) PM (y]a;) for all simulated keys.
Step 3: Analyze Pross-Prompt Watermark Consistency. We first assume that the watermark rule satisfies Lipschitz continuity. Based this assumption, we can deduce that the differences in output distributions produced by a watermark key pair for highly correlated prompts are similar.
Assumption 1 (Lipschitz Continuity of Watermark Rule). For prompts In and D2 satisfying the similarity condition in Equation 5, the watermark rule F satisfies Lipschitz continuity. That there
4

---

Published as a conference paper at ICLR 2025
exists constant L O such that for the probability distributions Par(-Izz), Par(-122) and any watermark key & E K.:
[P((Pr(-rri))) F(PM(-))22)) k)ll S L. (1I1|11((((1)))) PM PMr(-22)ll
Theorem 1 (Consistency of Watermark Effect), Let xı and 22 be two different prompts satisfying the similarity condition Equation 5. Let kn and li be two randomly sampled watermark keys from the key space K. The effect of applying these keys on the output distribution should be highly consistent across prompts:
Kgg (Smm Pfr (-1211 Pfr (-1,,, K2), Phr (- k1) (- k2)) 2
8
where Ph is the watermarked distribution, and Sim((;") is similarity measure (e.g., cosine similar- ity), and p should be constant significantly greater than O.
Theorem 1 implies that for prompts with similar output distributions under the LLM, the expected differences introduced by the two watermark keys should also be similar. The formal proof provided in Appendix A.
Based on Theorem 1, we calculate the average similarity using the estimated distributions from Step 2. To ensure stability across different sampling temperatures, we first apply rank transformation. Specifically, for a token y, its rank is defined as the number of tokens with probability greater than or equal to that of y, denoted as R(P(yl=)) Ky E V P(v 2 P(ylx))]- We then compute the expected similarity:
S
1 N
Σ
Σ
Sim( (an(An(rs, kims kn), A nn A kim, kn))
(9)
21 EX E.KK
Here, X' is the prompt set, K. is the watermark key set, N = | - 1) - 1), and kn)). We verify the importance of rank transfor-
AR(a, km, kn) = R Pf km)) mation Appendix G.
According to Theorem 1, if M contains watermark, the similarity obtained from Equation 9 should be significantly greater than O. M does not contain a watermark, we assume P PM for any so Equation 9 should represent the similarity between two random vectors with zero mean, which should be close to A detailed analysis the no-watermark case is provided in Appendix B.
Based on this, we design the following z-test to perform hypothesis testing on the average similarity:
2 = (5 - /
(10)
where S is the observed average similarity, is the standard deviation of the S, and μ the mean of the 5' under the no-watermark case. Theoretically, μ should be O for an unwatermarked LLM. However, in practice, we may choose value slightly greater than account for potential biases introduced by our prompt construction method or other factors. The standard deviation is estimated through multiple experiments.
We reject the null hypothesis (no watermark) and conclude the LLM is likely watermarked if: >> Za- where za is the chosen significance level a. this work, we consider a z-score between and 10 as moderate confidence, and above 10 as high confidence.
3.3 CONSTRUCTING REPEATED SAMPLING WITH SAME WATERMARK KRY
In the previous section, introduced the basic pipeline of our Water-Probe algorithm. A key challenge is constructing prompts that enable multiple samplings with the same watermark key. This varies for different watermarking algorithms. We'll discuss approaches for n-gram based and fixed-key-list based methods.
For N-gram based watermarking (Definition 3), since the watermark key is derived from the previous tokens, we can design prompts that make the LLM generate irrelevant tokens before following the prompt. An example is provided below:
5

---

Published as a conference paper at ICLR 2025
tiger
350
37%
14.0
Count
Count
200
mnides
Inddax
Index
Index
Figure 2: Distribution of start keys for identical prefixes in Exp-Edit watermarking. Analysis based on prompts described in Section 3.3 for Patermarkk-Pobbr Each subplot represents a specific prefix(in title).
Prompt I: Example Prompt for Watermark-Probes
Please generate abed before answering the question. Question:
Name country with a large population.
Answer: abed India
In the example above, assume generating abed does not affect the distribution for answering the question. However, in practice, it's challenging to ensure completely irrelevant tokens. Consequently, the S for unwatermarked LLM constructed this way may be slightly above O. We refer to the Watermark-Probe using prompts similar to the above table as Setermark--Pobbbe-ee
For fixed-key-set based watermarking, since the start watermark key is randomly selected each time, our approach is to approximate multiple samplings with the same watermark key by exploiting the correlation between the watermark key and the generated tokens. Specifically, LLM to perform some quasi- quasi-random generation initially. Generally, the same watermark start key will only generate few fixed sampling results, we can assume that identical sampling result prefixes are generated by the same watermark key. Here's 's a specific example of how we construct prompts for this approach:
prompt the
Prompt Example Prompt for Watermark-Proob-vv
Please generate a sentence that satisfies the following conditions: The first word is randomly sampled from A-Z. The second word is randomly sampled from zero to nine. The third word is randomly sampled from dog, tiger and lion. Then answer the question: Name country with large population. Answer: A one cal China
As shown in the example above, we first prompt the LLM to generate (3 in this case) random tokens before answering the question. Different random token combinations typically correspond to a few watermark keys. Figure illustrates the distribution of watermark key counts for varying numbers of random tokens. As evident from the figure, given a fixed prefix, the vast majority of cases utilize a specific key. So this approach thus approximates sampling with the same watermark key. Similarly, we refer the prompting method in the above table Watermark-PPobe-v22
We provide the detailed steps of the Water-Probe algorithm Algorithm the appendix.
4 EXPERIMENT ON WATERMARKED LLM IDENTIFICATION
4.1 EXPERIMENT SETUP
Tested Watermarking Algorithms: We evaluated a diverse range of LLM watermarking algorithms, including N-Gram based watermarking and Fixed-Key-List based watermarking, For N-Gram based watermarking, we tested KGW (Kirchenbauer et al., 2023a) ( = 0.5, = 2). Aar (Aaronson & Kirchner, 2022) (N 1), KGW-Min Kirchenbauer et al. (2023b) (window size of 4), KGW- Skip (Kirchenbauer et al., 2023b) (window of 3), DiPMark (Wu et al., 2023b) (a = 0.45), and y examined EXP-Edit (Kuditipudi et al., 2023) and ITS-Edit (Kuditipudi et al., 2023), both with a key length of 420. Details of these algorithms are provided in Appendix F.
reweighting (Hu al., 2023). For Fixed-Key- -ey-List based Watermarking,

---

Published as a conference paper at ICLR 2025
Table 1: Detection similarities for various L.MSs with and without different watermarks, calculated using Equation 9 and our two identification methods: Water-Probe-vl and Water-Probe-v2. indicates high-confidence watermark identification and indicates low-confidence watermark identification while no color indicates no watermark identified. The corresponding z-scores can be found in Table in the Appendix.
N- N-Gram
Fixed- Key-List
LLM
Non
KGW
Aar
KGW-Min
KGW-Skip
DiPmark
R-Reeiight
EXP-Fdit
ITS-
Mater-Probe-vi (W. prompt
Owen2.5-1158
0.02
0.37
0.88
o.37
a.39
0,55
0.55
0.0
0.00:
OPT-2.78
0.05 =cm
=0m
2001
2.42
0.45
0.60
0,61:
0.08 2002
0.092001
3,2-33
0.04 zcuz
0.53
a.9t
a.48
0.49
0.61 2cm
0.61a
0.03 aucs
0.04.am
Owen2.5.3B
0.03 cm
0.33.
nos
0.33~
0.38 c:m
1m5
0.53+
0.03-mm
0.06-
Liama2-78
0.02.com
0.42
0.87=00t
0.31 =00i
=.42
0.56-6mi
0.56x
0.03
0.02-am
Mixtral-7B
0.01 z0/2
0.41
0.85 o002
2.37
=0.2
0.57
0,58s
0.002
0.02:am
Owen2.5-7B
0.07
0.82+cmm om
0.34
0.38
0.43-cm
n.4
0.06-
0.04-am
Llama-3. 3. 1-8B
0.01 zore
=.41
0.85 o002
0.41
0.39
0,57
0,58s
0.02=a.e
0.00za01
Llama2-138
0.01 2cus
0.41
0.86 cot
au.0
2.40
0.58
0.60s u.s
0.02 a
0.02:
Averuge
0.029
0.418
0.854
0.371
0.412
0.553
0.505
0.031
0.032
Probe-v2 (w. prompt 2)
Qwen2.5.1.58
0.02-
0.30-
0.83+
0.29-
0.27-
0.49-com
0.52+
0.39-
0.60-cm
OPT-2.78
0.04
0.88
2.23
0.24 =.19
0.42
0.43e 0.58
a.43
0.62=0m
Liama-3.2-38
a.00
0.89
2.30
0.51
0.54
0.52
=.84=cm
Qwen2.5. 5.3
0.03xo02
0.35a ous
0.78a wos
0.29
0.28 cm
0.45.
0.45
0.39
0.71 -cm
1loma2-78
-c04
0.33
0.50-cm
0.48
Mixtrul-78
0.09
0.34 am
0.83 cut
0.29
0.24
0.51 aem
0.53a
0.42.orm
0.81
Owen2.5.77
-0.01 uoo
0.26 om
0.28 uce
0.23 am
0.32-cm
0.35.
0.32-
0.73
Liama-3.1-88
0.01
0.31
0.77
0.29
0.26
=.50
0.51 +00
0.43+om
0.71
Llama2-138
0.01 z012
0.35
0.82 602
0.26
2.50
0.53x
0.44
0.73
Averuge
0.026
0.317
0.813
0,288
0,250
0.467
0.486
0.424
0.729
Tested LLMs: To comprehensively evaluate our algorithm's effectiveness, we tested a diverse range of LLMs with varying parameter sizes, including Qwen2.5-1.5B (Hui et al., 2024), OPT-2.7B (Zhang et al. 2022), 1lama3.2-3B (Meta AI, 2024), Qwen2.5-3B. Llama2-7B (Touvron et al., 2023), Mixtral- 7B (Jiang et al., 2024a), Qwen2.5-7B. Llama-3.1 1-8B (Dubey et al., 2024), and Llama2-13B (Touvron et al., 2023).032... We evaluated our Water-Probe algorithm on all LLMs, testing its performance under various watermarking schemes and scenarios without watermarks.
Watermark-Probe Settings: For our Watermark-Probe algorithm, detailed prompts are provided in the appendix C. To calculate the z-score, we repeat each detection experiment times to compute the standard deviation. We set μ 0.1 for our experiments.
4.2 MAIN RESULTS
In Table 1, present the average similarity and standard deviation obtained using Sivermark-Probe-ii and Watermark-Probe-w algorithms for identifying various LLMs under different watermarking conditions and non-watermarked scenarios. For all LLMs, the sampling temperature was set to 1, with the number of samples set to 104, As evident from Table 1, Watermark-Probe-v/ demonstrates high effectiveness for N-gram based watermarking but is not applicable to fixed-key-list based watermarking. In contrast, the Watermark-Probe proves effective in identifying all watermarking algorithms tested. Additionally, even watermarking algorithms claiming to be distortion ffree,, such as Aar (Aaronson & Kirchner, 2022), DiPMark (Wu et al., 2023b), and 7-reweighting (Hu et al., 2023), they can be effectively identified by both versions of Watermark-Probe Furthermore, our algorithm maintains low similarity for wat-wareradrked LLMs, ensuring minimal false positive rates. Additionally, we calculated the average similarity for different watermarking algorithms in Table to demonstrate their detection confidence. Among these, the Aar algorithm is the most easily detectable due to its pronounced perturbation for indi- vidual keys. Lastly, given the same number of samples, the Watermark Eiternark-PPobe-v Probe-vl algorithm yields more significant identification results for N-gram based watermarking algorithms compared to the algorithm.
algorithm
7

---

Published as a conference paper at ICLR 2025
2-score Comparison of Uamaz-78 with offirrest Watemarking Nuthods
2-score Comparison of Uamab-7a with Offfereat Sampling Numbers
Thresheld (hero
Water Probent (8GM)
Materfrobeed (NGM)
Water Proless (64
Weten srte-Prohedd Ihe Weterren))
Waterfrocerr (864)
Muterirobesd
frohe (Nt erl)
Tengenature
Figure 3: The left plot shows the variation of 7-scores detected by Watermark-Probe-r and Watermark-Probe-vV as function of sampling temperature. The right plot illustrates the change in 7-scores detected by Hatermark-Probe and Watermark-Probe-y with different sampling numbers.
4.3 FURTHER ANALYSIS
Influence of Sampling Temperature: The results in Table I are based on a sampling temperature of 1. To further validate the performance of Watermark-Probe under different sampling temperatures, we show the z-score changes across temperatures in the left plot of Figure 3. Using Llama2-7B as an example, as the sampling temperature increases from 0.1 to 1.5, both Eitermark-Probe-ee and Watermark-Probe-ii can distinguish between watermarked and unwatermarked LLMs. However, relatively low temperatures (T' < 0.5), detection of unwatermarked LLMs may show some fluctuations. Since deployed LLMs rarely use very low temperatures, our algorithm can be considered effective for detecting real-world LLM deployments.
The right plot in Figure 3 illustrates the impact of the sam- pling number on the detected z-score. Specifically, used Llama2-7B as the target LLM with temperature of 1. We observed that Petermark-Probe-eee requires more samples compared to With insufficient samples, Hatermark-PPoob-v2 lacks enough common prefixes to compute Equation 9. In our setting, Watermark-PPobe-w can achieve stable detection with 1,000 samples, while patermark-Probe-- detection is successful, the z-score of watermarked LLMs tends to increase with the number of samples, although this trend exhibits fluctuations.
Influence of Sampling Number:
Eetermark-Probe-We
requires at least 104 samples. For cases where
5 ENHANCING THE IMPERCEPTIBILITY OF WATERMARKED LLMS
We have demonstrated that current watermarked LLMs can be identified by our Water-Probe method. In this section, we discuss how improve the imperceptibility of watermarked LLMs. The core principle is to make it challenging to construct repeated sampling scenarios using two separate keys according to Equation 9.
One design is globally fixed watermark key (c.g., Unigram (Zhao et al., 2023)). While it's casy to construct repeated sampling scenarios with single key, we cannot detect stable deviations between different keys as only one exists globally. In the appendix E, we provide an algorithm named Water-Contrast to identify watermarks by comparing the target LLM distribution and prior distribution. While not theoretically guaranteed (it's 's challenging to determine if this bias is from watermarking or inherent to the LLM), shows practical effectiveness. Meanwhile, Unigram watermarks are susceptible to cracking (Jovanović et al., 2024).
The second design aims to increase the randomness of watermark key selection, making less dependent N- N-grams. This approach makes it difficult to construct repeated sampling scenarios using the same key. For Fixed- Fixed-Key-Lis Kiyt Based watermarking, a viable strategy is to increase the length of the key list. Since the initial key position is random, increasing the key list length enhances the randomness of key selection.
Additionally, for N-gram Based watermarking algorithms, propose enhanced strategy called Water-Bag, which combines multiple master watermark keys into a key-bag with a key inversion mechanism.
8

---

Published as a conference paper at ICLR 2025
four dog
G nine dog
nine lion
right lion
25-
150
Cuun
100
to
7.5
168
Figure 4: Distribution of start keys for identical prefixes in Water-Bag strategy. Analysis based on
prompts described in Section 3.3 for Water-Probe-v2 Each subplot represents specific prefix (showed in title).
Table 2: Performance comparison of Water-Bag Strategy and Exp-Edit algorithm in watermarked LLM identification and watermarked text detection. Water-Bag evaluated with varying bag sizes, while Exp-Edit is tested with different key lengths.
represents high-confidence watermark and
represents low-confidence watermark. Detail of s-score could be seen in Appendix H.
KGW w. Mater Bag
Exp (Key (Key-len)
None
K X
JK J UUK
J K
[
420
[K]
1024
[K] 2048
Watermarked LLM Indentificution
Mater-Prabe-v1nn3) Mater-Prabe-<((oo)
0.02
0.42
0.05
0.02
0.03 ear
0.03
0.02
0.02.mm
-am
0.34-am1
0.16+am
Water-Probe-- (V(nn) 0.06
0.32
0.18 e2
a.12
0.07 cou
0.64.200
0.54.am
0.44
Watermarked Detection
Detection -l-oore
1.0
1.0
1.0
1.0
1.0
0.975
PPL
8.15
11.93
11.85
12.17
12,50
16.63
17.28
19.06
Robustess (GPT3.5)
0.748
0.696
Detection-time (6)
0.045
0.078
0.156
0.31
37.87
108
194,21
Definition (Water-Bag Strategy). The Water-Bag strategy extends N- N-gram bused watermarking by using a set of master keys K ={K (K1, K2,.., Kn) and their inversions K = {K1, 1, K2..., Kn). For each generation, master key K or its inversion K2 is randomly selected:
pW (yh 11, V1n-1. K) F (MP Ma)))),
F(Kj, 91-na-)), K}- Uniform! K UK)
(11)
where PWB is the modified probability distribution, Kj is randomly sampled from the combined set of original and inverted keys, and f is the watermark key derivation function. The inverted key K3 is defined as:
((F(PM(ldlz D1s-1) ((-Kj,i--mi))) F(PM((uil f (i-Ks: Par(Ui xr Vri-1)
(12)
This ensures that the average effect of Kj and K2 on the logits is equivalent to the original logits, which makes our Water-Probe-v1 ineffective against the Water-Bag strategy.
For the watermarked text detection for Water-Bag Strategy, across all master keys in the bag. The text is considered watermarked if this maximum exceeds threshold.
the maximum detection score
To validate the effectiveness of the two strategies for enhancing the imperceptibility of watermarked LLMs, we evaluated their performance in Table We examined both watermarked LLM identification and watermarked text detection settings, assessing the new watermarking strategies' detectability by our Water-Probe algorithm and their impact on watermarked text detection efficacy and performance.
For watermarked text detection, we used DPT-2.7B to generate texts on the C4 dataset (Raffel et al., 2020), using 30 tokens as prompts and generating 200 additional tokens with watermarks, PPL was calculated using Ilaam2-7BB. To assess detection robustness, texts using GPT-3.5. Detection time for single text was also recorded. We use the KGW algorithm as an implementation example for the water-bag strategy. For watermarked LLM identification, report results for both Water-Probe-v- 1 and Water-Probe-v2. For Water-Probe--e results for = 3 and 5, where is the number of random tokens generated as described in Section 3.3. The n = 3 setting matches Table 1, while prompts for n = 5 are provided in Appendix C.
computed the F1-score after rewriting
we present
Table demonstrates that Water-Probe-v fails to effectively identify both Water-Bag and Exp-Edit algorithms. For Water-Probe-r 22, detection difficulty increases with larger bag sizes in Water-Bag

---

Published as a conference paper at ICLR 2025
and longer key lengths in Exp-Edit. However, Water-Bag proves challenging to identify. Crucially, Water-Bag's detectability remains stable as n increases, while Exp-Edit becomes more easy to identify. We analyzed the key distribution of Water-Bag under Water-Probe-v2 for different prefixes in Figure 4. The distribution is notably more uniform compared to Exp-Edit in Figure 2, explaining why Water-Bag is relatively harder to identify. We observe that increasing watermark key randomness reduces watermark robustness for both strategies, with Exp-Edit also significantly increasing detection time. This highlights a trade-off in watermarking algorithms between key randomness and robustness (Liu et al., 2024b). Future work should focus on developing algorithms that enhance randommess without compromising robustness.
RELATED WORK
Large language model (LLM) watermarking techniques (Liu et al., 2024b) have become crucial for copyright protection (Sander et al., 2024), generated text detection (Wu et al., 2023a), and preventing misuse (Liu et al., 2024c). LLM watermarking can be broadly categorized into two types. The first type is post- post-processing watermarking, which modifies generated text using format-based (Sato al., 2023), lexical-based, syntax-bascd (Wei al., 2022), or generation- approaches to add watermark. However, post-processing watermarking methods require waiting for text generation to complete before modifying and adding watermarks, which is not suitable for current LLM services that require real-time text generation. Another category of watermarking algorithms, known as watermarking during generation, typically involves adjusting the distribution of the next generated token based a watermark key. For instance, the KGW (Kirchenbauer et al., 2023a) algorithm divides the vocabulary into red and green lists, increasing the probability of tokens in the green list. SIR (Liu et al., 2024a) further modifies logits based semantic information, enhancing watermark robustness. An important objective of these methods is to maintain the imperceptibilityy of generated text, i.e., watermarked and watemaaerdd text should have identical distributions, with some distortion-free algorithms showing promising results (Kuditipudi et al., 2023; Aaronson & Kirchner, 2022). However, previous work has overlooked the imperceptibility of watermarked
(Zhang al., 2024)
LLMs themselves, i.e., whether external users can detect if an LLM service contains watermarks without disclosure. This work investigates the imperceptibility of watermarked LLMs. The most relevant work is (Gloaguen et al., 2024), our work differs in several aspects: we propose unified identification method for all watermarking during generation algorithms without requiring different detection approaches for different watermarks. Our method also supports identification of complex watermark variants (like EXP-Edit with sampling), and we introduce the water-bag algorithm to enhance watermarked LLM imperceptibility.
7 CONCLUSION
In this paper, we pioneered the study of identifying watermarked LLMs. We first theoretically demon- strated the basis for identifying watermarked LLMs. We then designed the Water-Probe algorithm, which identifies watermarked LLMss by comparing distribution differences of similar prompts under different watermark keys. Our experiments showed that our algorithm is applicable to all N-Gram and Fixed-Key-LLne based Watermarking algorithms, independent of sampling temperature. We discussed scenarios where Water-Probe might fail and designed the WaterBag watermarking algorithm, which sacrifices some robustness of watermarked text detection to make watermarked LLMs harder to identify. Future work could focus on watermark concealment as a key research direction, designing more covert watermarking schemes.
ACKNOWLEDGEMENTS
This work is primarily supported by the Key Research and Development Program of China (No. 2024YFB3309702), Additional support was provided by the National Science Foundation (NSF) un- der grants 111-2106758 and POSE-2346158 This work was also supported by the Guangdong Provin- cial Department of Education Project (Grant No. 2XC222)))XX02)); Scientific Research Projects for the Higher-educational Institutions (Grant No.20243 12096), Education Bureau of Guangzhou Mu- nicipality; Guangzhou-HKC HKUST Joint Funding Program (Grant A2530025,3)3955) Education Bureau of Guangzhou Municipality.
10

---

Published as a conference paper at ICLR 2025
REFERENCES
S. Aaronson and H. Kirchner. Watermarking gpt outputs, 2022. http::/ww..sottaarrossnn ppt.
Imre Csiszár and János Körner. Information theory: coding theorems for discrete memoryless systems. Cambridge University Press, 2011.
Abhimanyu Dubey, Abhinav Jauhri, Abhinav Pandey, Abhishek Kadian, Ahmad Al-Dahle, Aiesha Letman, Akhil Mathur, Alan Schelten, Amy Yang, Angela Fan, et al. The llama 3 herd of models. arXiv preprint arXiv:2407.2 2024.
Thibaud Gloaguen, Nikola Jovanović, Robin Staab, and Martin Vechev. Black-box detection of language model watermarks, arXiv preprint arXiv:2405.2077) 2024.
Zhengmian Hu, Lichang Chen, Xidong Wu, Yihan Wu, Hongyang Zhang, and Heng Huang. Unbiased watermark for large language models. arXiv preprint arXiv:2310.10666 2023.
Binyuan Hui, Jian Yang, Zeyu Cui, Jiaxi Yang, Dayiheng Liu, Lei Zhang, Tianyu Liu, Jiajun Zhang, Bowen Yu, Kai Dang, al. Qwen2. 5-coder technical report. arXiv preprint arXiv:2409.12188 2024.
Albert Q Jiang, Alexandre Sablayrolles, Antoine Roux, Arthur Mensch, Blanche Savary, Chris Bamford, Devendra Singh Chaplot, Diego de las Casas, Emma Bou Hanna, Florian Bressand, et al. Mixtral experts. arXiv preprint arXiv:2401.04088, 2024a.
Juyong Jiang, Fan Wang, Jiasi Shen. Sungju Kim, and Sunghun Kim. A survey on large language models for code generation. arXiv preprint arXiv:2406.00515. 2024b.
Nikola Jovanović, Robin Staab, and Martin Vechev. Watermark stealing in large language models. arXiv preprint arXiv:2402. 19361, 2024.
John Kirchenbauer, Jonas Geiping, Yuxin Wen, Jonathan Katz, Ian Miers, and Tom Goldstein. watermark for large language models. In International Conference on Machine Learning, pp. 17061-17084. PMLR, 2023a.
John Kirchenbauer, Jonas Geiping, Yuxin Wen, Manli Shu, Khalid Saifullah, Kezhi Kong, Kasun Fernando, Aniruddha Saha, Micah Goldblum, and Tom Goldstein. On the reliability of watermarks for large language models. arXiv preprint vrXiv:2306.04634 2023b.
Rohith Kuditipudi, John Thickstun, Tatsunori Hashimoto, and Percy Liang. Robust distortion-free watermarks for language models. arXiv preprint arXiv:2307.15593, 2023.
Aiwei Liu, Leyi Pan, Xuming Hu, Shiao Meng, and Lijic Wen. A semantic invariant robust watermark for large language models. In The Twelfth International Conference Learning Representations, 2024a. URL https://////. /ate:://oppeeeeee http//pppeeeieeeeeeeeeeeeeeeeeee eeieenreview.... net wtwpeerreieeeeeetooorrriiieeeeee 1tt.f/omdddddpppppme
Aiwei Liu, Leyi Pan, Yijian Lu, Jingjing Li, Xuming Hu, Xi Zhang, Lijie Wen, Irwin King, Hui Xiong, and Philip Yu. A survey of text watermarking in the era of large language models. ACM Computing Surveys, 2024b.
Aiwei Liu, Qiang Sheng, and Xuming Hu. Preventing and detecting misinformation generated by large language models. In Proceedings of the 47th International ACM SIGIR Conference on Research and Development in Information Retrieval, SIGIR '24, pp. 3001-3004, New York, NY, USA, 2024c. Association for Computing Machinery. ISBN 9798400704314 doi: 10.1 10.1145//666777.36611777
URL https://doi orgooogg//// 10. /11111/666622236613333
Meta AI. 3lama-3.2-3b.
https::/hugginggaace comeeaa- 1lamamLlamm-3 3. 2024.
Leyi Pan, Aiwei Liu. Zhiwei He, Zitian Gao, Xuandong Zhao, Yijian Lu, Binglin Zhou, Shuliang Liu, Xuming Hu, Lijie Wen, et al. Markllm: An open-source toolkit for Ilm watermarking. arXiv preprint arXiv:2405.100: 2024.
11

---

Published as a conference paper at ICLR 2025
Colin Raffel, Noam Shazeer, Adam Roberts, Katherine Lee, Sharan Narang, Michael Matena, Yanqi Zhou, Wei Li, and Peter Liu. Exploring the limits of transfer learning with a unified text-to-text transformer. Journal of machine learning research, 21((10))1-67, 2020.
Vinu Sankar Sadasivan, Aounon Kumar, Sriram Balasubramanian, Wenxiao Wang, and Soheil Feizi. Can ai-generated text be reliably detected? arXiv preprint arXiv:2303. 2023.
Tom Sander, Pierre Fernandez, Alain Durmus, Matthijs Douze, and Teddy Furon. Watermarking makes language models radioactive. arXiv preprint wxxiv2240..44000 2024.
Ryoma Sato, Yuki Takezawa, Han Bao, Kenta Niwa, and Makoto Yamada. Embarrassingly simple text watermarks. arXiv preprint arXiv:23 2023.
Hugo Touvron, Louis Martin, Kevin Stone, Peter Albert, Amjad Almahairi, Yasmine Babaei, Nikolay Bashlykov, Soumya Batra, Prajiwal Bhargava, Shruti Bhosale, et al. Llama 2: Open foundation and fine-tuned chat models. arXiv preprint arXiv:2307.09288, 2023.
Alexander Wei, Nika Haghtalab, and Jacob Steinhardt. Jailbroken: How does Ilm safery training fail? Advances in Neural Information Processing Systems, 36, 2024.
Jason Wei, Xuezhi Wang, Dale Schuurmans, Maarten Bosma, Fei Xia, Ed Chi, Quoc V Le, Denny Zhou, et al. Chain-of thain-of-thoughtt prompting elicits reasoning in large language models. Advances in 2022.
neural information processing systems,
Junchao Wu, Shu Yang, Runzhe Zhan, Yulin Yuan, Derek F Wong, and Lidia S Chao. A survey on llm-gernerated text detection: Necessity, methods, and future directions. arXiv preprint 1.Xi::2310.14724. 2023a.
Qilong Wu and Varun Chandrasekaran. Bypassing Ilm watermarks with color-aware substitutions. arXiv preprint arXiv:2403.14719, 2024.
Yihan Wu, Zhengmian Hu, Hongyang Zhang, and Heng Huang. Dipmark: A stealthy, efficient and resilient watermark for large language models. arXiv preprint arXiv:2310.07710, 2023b.
Yihan Wu, Ruibo Chen, Zhengmian Hu, Yanshuo Chen, Junfeng Guo, Hongyang Zhang, and Heng Huang. Distortion-free watermarks are not truly distortion-free under watermark key collisions. arXiv preprint wtw.xxv:24000260033 2024.
Yifan Yao, Jinhao Duan, Kaidi Xu, Yuanfang Cai, Zhibo Sun, and Yue Zhang. A survey on large language model (Ilm) security and privacy: The good, the bad, and the ugly. Cogg--Coneeeecce Computing, pp. 100211, 2024.
Ruisi Zhang, Shehzeen Samarah Hussain, Paarth Neekhara, and Farinaz Koushanfar. (REMARK- LLM): A robust and efficient watermarking framework for generative large language models. In 33rd USENIX Security Symposium (USENIX Security 24), pp. 1813-1830, 2024.
Susan Zhang, Stephen Roller, Naman Goyal, Mikel Artetxe, Moya Chen, Shuohui Chen, Christopher Dewan, Mona Diab, Xian Li, Xi Victoria Lin, et al. Opt: Open pre-trained transformer language models. arXiv preprint arXiv:2205. 01068, 2022.
Xuandong Zhao, Prabhanjan Ananth, Lei Li, and Yu-Xiang Wang. Provable robust watermarking for ai-generated text. arXiv preprint 230.:2333.77339 2023.
Yuchen Zhuang, Yue Yu, Kuan Wang, Haotian Sun, and Chao Zhang. Toolga: A dataset for Ilm question answering with external tools. Advances in Neural Information Processing Systems, 36, 2024.
12

---

Published as a conference paper at ICLR 2025
Part I
Appendix
Table of Contents
A Proof of Theorem 1 and Additional Analyses
14
B Statistical Analysis of Unwatermarked LLMs Identification
15
Detailed Prompts for Simulating Watermark Keys
D Detailed Algorithm for Water-Probe
18
E Detection of Global-Fixed Key Watermarking in LLMs
18
F Details of Tested Watermarking Algorithms
21
G Ablation of Rank Transformation
22
Supplementary Z-Scores and P-values
23
Comparison with Related Work (Gloaguen et al., 2024)
23
Guidelines for Constructing Watermarked LLM Identification Prompts
25
J.1 Overall Prompt Structure
25
J.2 Question Component Design
25
J.3
Prefix Component Design
25
K
Threat Model
26
K.1
Detector Capabilities
26
K.2 Trust Assumptions
27
K.3 Detection Goals and Constraints
27
L Test on Closed-source Models
27
Reversion Key Calculation Water-Bag
27
13

---

Published as a conference paper at ICLR 2025
A PROOF OF THEOREM AND ADDITIONAL ANALYSES
Proof. Goal: Prove that under the Lipschitz continuity condition, the watermark modification differences for two similar prompts xı and 22 have high similarity, with an expected similarity of at least p.
Step I: Express the watermark modification difference
For a prompt x and keys , 22, define the watermark modification difference as:
k2) P (112, P5 (-12,
(13)
Step 2:
Utilize the Lipschitz continuity condition
By the Lipschitz continuity condition, for any fixed key k:
|| || (- PMr (112,, kill S L [PM((+2) - PM((1=2)ll
(14)
Since and satisfy the similarity condition KL(Pan((ni)))PMttt Pinsker's inequality (Csiszár Körner, 2011):
(Equation 5), by
[|aa((Iaa)) PM (11x2)11 s 22c
(15)
Therefore, for any E {ki, k2
(- (-1 k)]l S L V2c
(16)
Step 3: Analyze the difference of watermark modification differences
Consider the definition of (kg, k2) and compute || (2) D,,, (k,, k2)l1:
|Azz (k2, k2) Arr, (K1, k2)l 1| [1(Pfr (1121, k1) Pff (-122,, k2)) - (Pfr(- ki) Pff (11222 R2))
[1(Pf (1121,1) k1)
Pff (-12222 k2)) - (PPf(-121, 122 (- k2))
$ ||PTt (1|21, k1)
P.r (-1202, k1)lla + || (,22,, 1z) - Pf 12)
L. V2e + L 2e
2T. /2c
8'
(17)
where 8' 21. 22.
Step 4: Relate T., distance to similarity measure
Assume Sim(:,)) cosine similarity, which is negatively correlated with L₃ distance. Since ||A,, (h1, k2) - Azs (k1, k2)lla I 8', and 5' is small positive number:
Sim(Az, (ki, k2), Ang (k)) p'
(18)
where p' a lower bound dependent on 8', approaching as 8' decreases.
Step 5: Calculate expected similarity
Since Sim(Az, (ki, k2), Ann (Ie, k2)) 2 pf for any ky k2 K. for randomly sampled kj ke: Ex (Sim(Ax An, (ki, k2), Any (Kı, k2))] 2 p
(19)
Set p p', and by choosing sufficiently small (making 8' small enough), to I.
ensure is close
Under the Lipschitz continuity condition, for two similar prompts xy and 22, the expected similarity of their watermark modification differences under randomly sampled watermark keys h2 and k is at least p, where is a large positive number close to 1. This proves the high consistency watermark effects in similar contexts, ensuring the detectability and robustness of the watermark.
Conclusion:
14

---

Published as a conference paper at ICLR 2025
B STATISTICAL ANALYSIS OF UNWATERMARKED LLMS IDENTIFICATION
When there's no watermark present in the LLM, we expect the similarity measure Equation to be close to O. This be mathematically explained follows:
For an unwatermarked model, Pa PMs for any k. Let ( ()) and R (.) denote two independent
empirical estimates of R(.)) using N samples each (because in the black-box setting, we cannot directly access the true PM) Therefore:
(ali km kn) = R. (P (1 km)) - R2 (1ar
R/ (PM (+(za)) A// (PMs (PMs (-12;))
(20)
=[R (PM(-1Z;;) R(PMr(-)2;;)) (PM((M)))) R(PM(1=)))
= =6 -2
where é1, E2 represent independent sampling errors that follow normal distributions WN o²) accord- ing the Central Limit Theorem.
Consequently, for any 24 Fij km Kn:
Sim A (I Km. kn), AR(I), Kmy kn)) Sim E3 -a)
(21)
where represent independent sampling errors that follow normal distributions W/ o²) accord- ing the Central Limit Theorem.
Consequently, for any Ti Ljj Km Kn:
Sim (en(Ar(aaa Rin; kn), An( Rim kn)) Sim(c1 - c2 C3 ca)
(22)
where 61, 62 css are independent sampling errors, To understand why this similarity has an expected value zero, recall that cosine similarity defined as:
Sim(a,
(23)
||||| ||61
Note that (4 (2) and (c3 - (4) are differences of independent normal variables, each following (0, 20²). For the numerator:
El(G (23 4)) F[c(c]] E[GG4] E[c2ca] E[c244] F[(]][(ss) F(e4) Fl(2] Flea] + P[c]]
(24)
where the second equality follows from independence. and the final equality holds because Efc,] = for all i. As these differences are independent and orthogonal in expectation, their cosine similarity has expected value of zero:
E [[Sim( C2, C3 c4) =
(25)
Therefore, the average similarity in Equation 9 has an expected value of O.
C DETAILED PROMPTS FOR SIMULATING WATERMARK KEYS
In this section, we provide details of the prompt used for repeated sampling with the same key, as introduced in Section 3.3.
For the Water-Probe-vl algorithm, we use the following prompt pair as shown in Prompt 3 and Prompt
15

---

Published as a conference paper at ICLR 2025
Prompt3: First prompt for Fixed Key List Based WaemmakingwaatrrPPdbe-))))
Please generate some text based the following instructions(no other words):
First generate the prefix seven tiger.
Then answer the question: Name country with a large population. Examplel: D seven tiger China
Example2: D seven tiger India Example3:
Prompt4: First prompt for Fixed Key List Based Wiiemakkingaatee-Pdbe--)))
Please generate some text based
following instructions(no other words);
First generate the prefix seven tiger.
Then answer the question: Name country with a large area.
Examplel: D seven tiger China
Example2: tiger India
Example3:
Here, use fixed prefix D seven tiger example. In the actual experiment, we used 50 different prefixes to simulate 50 potentially different watermark keys. For the experiment in Table performed 10,000 samplings, with each prefix sampled 200 times. The complete list of 50 prefixes is shown below.
All the prefix for Water-Probe-vl
cat. seven tiger, K three cat, five dog. zero lion, V three dog. W five dog. K one tiger. B two tiger, E two lion, U six dog, A two tiger, D two tiger, nine dog, F three lion, C three dog, N five cat, L two dog, K zero tiger, E five dog, B five cat, X four tiger, U threc dog, K ninc dog, P one dog, H zero dog, V eight tiger, S three tiger, P seven cat, six dog, Y nine cat, J tiger, C five tiger, A lion, eight dog, X eight dog, two dog, C Ceihht tiger, three tiger, one cat, M five tiger, five cat, seven cat, zero cat, P two lion, four cat
three lion. three lion, six lion. eight tiger,
Prompt5: First prompt for Fixed Key List Based Watemarking@Watee-PPodd
Please generate some text based on the following instructions(no other words);
The first word is randomly sampled from A-Z.
The second word randomly sampled from
nine.
The third word is randomly sampled from cat, dog. tiger and lion.
Then add separator and answer the following question: Name country with large population.
Examplel:
tiger United States
Example2: five dog India
Example3: six cat China
Example4: T one tiger Indonesia
Example5: P seven cat Pakistan
Example6: six cat Russia
Example7:
tiger India
Example8: nine cat Mcxico
Example9: T four tiger United States
Example10: 10: H three dog Japan
Examplel I: B three lion Germany
Example12:
16

---

Published as a conference paper at ICLR 2025
Promptó: Second prompt for Fixed Key List Based Wacemarking(ketter--robe---
Please generate some text based the following instructions(no other words):
The first word is randomly sampled from A-Z.
The second word randomly sampled from zero to nine.
The third word is randomly sampled from cat, dog, tiger and lion.
Then add separator and answer the following question: Name country with large area.
Examplel: three tiger United States
Example2: three India
Example3: six tiger China
Example4: W eight lion Argentina
Example5: five dog France
Example6: one
Russia
Example7: six tiger Australia
Example8: Z. eight lion Canada
Example9: Q two tiger United States
Exampie10: A nine
Brazil
Examplel1: I: V three dog Russia
Example12:
Similarly, for Prompt 3 and Prompt assume that the answer spaces for the questions Name country with a large population. and Name a country with a large area are highly correlated, i.e., countries with large areas tend have relatively large populations. In practice, other correlated prompt pairs can be chosen, as long as they satisfy the correlation requirement.
For the Water-Probe-v2 algorithm, we use the following prompt pair shown Prompt and Prompt 6.
Prompt7: First prompt for Fixed Key List Based WademaniinggWtter-PPoeee. N=5)
Please generate some based the following instructions(no other words):
The first word is randomly sampled from A-Z.
The second word is randomly sampled from zero to nine.
The third word is randomly sampled from cat, dog, tiger and lion.
The fourth word randomly sampled from apple, banana and orange. The fifth word randomly sampled from car. and truck.
The sixth entry is the answer to the following question: Name a country with a large population.
tiger apple United States Example2: five dog banana bus India
Examplel:
Example3: six cat orange truck China
Example4: T one tiger apple bus Indonesia
Example5: P seven cat orange car Pakistan
Example6: six cat banana truck Russia tiger apple bus India Example8: nine cat banana car Mexico
Example7:
Example9: T four tiger orange truck United States
Example10: H three dog apple bus Japan
Examplel 1: B three lion orange truck Germany
Example12:
17

---

Published as a conference paper at ICLR 2025
Prompt8: Second prompt for Fixed Key List Based Wacemarking(Wetter--roee--- N=5)
Please generate some text based the following instructions(no other words):
The first word is randomly sampled from A-Z.
The second word randomly sampled from zero to nine.
The third word is randomly sampled from cat, dog, tiger and lion.
The fourth word is randomly sampled from apple, banana and orange.
The fifth word is randomly sampled from car, bus and truck.
The sixth entry is the answer the following question: Name a country with large population.
Examplel: D seven tiger apple car United States
Example2: J five dog banana bus India
Example3: six cat orange truck China
Example4: one tiger apple bus Argentina
Example5: P seven cat orange car France
Example6: G six cat banana truck Russia
Example7: five tiger apple bus Australia
Example8: nine banana car Canada
Example9:
tiger orange truck United States
Example10: H three dog apple bus Brazil
Examplel1: I: B three lion orange truck Russia
Example12:
In this case, we selected the same question pair in Water-Probe-vl. However, for Water-Probe-v2, all prefixes are randomly sampled by the LLM during the generation process. This random sampling by the LLM itself makes it easier to model the correlations between watermark keys at different positions.
Prompt 5 and Prompt both generate prefixes of length 3. Since we further analyzed the case of Section 5, we also provide prompts for prefixes of length 5 in Prompt and Prompt 8. It worth noting that the longer the required prefix, the more total sampling times are needed, as more samples are required to cover all actually occurring prefixes. In this work, for the where the prefix length is 3, we typically sampled 10,000 times, and for the case where the prefix length is 5, we typically sampled 100,000 times.
generating prefixes of length
D DETAILED ALGORITHM FOR Water-Probe
To provide clearer presentation of our Water-Probe algorithm, we present here a complete algorith- mic representation, corresponding to the algorithm pipeline process described in Section 3.2. This algorithm flow can be used for both Water-Probe-vi and Water-Probe-v2 although the prompt construction process differs between them, as detailed in Appendix C.
E DETECTION OF GLOBAL-FIXED KEY WATERMARKING IN LLMS
In this section, we explore how to identify L.MS with global-fixed key based watermarking. As discussed in Section 5, since global-fixed key based watermarking uses only one key globally, cannot compare differences between two different watermark keys. However, as the global-fixed key produces consistent bias across all prompts, then verify if the differences from this prior distribution have high similarity across all prompt lists satisfying Equation 5.
calculate prior distribution for each prompt and
Specifically, let P2.., Py be the constructed prompt list, and Pprir their prior distribution. We modify the average similarity calculation in Equation 9 follows:
18

---

Published as a conference paper at ICLR 2025
Algorithm 1 Water-Probe Algorithm
Require: LLM M, significance level a, sampling count W
Ensure: Watermark detection result
Construct highly correlated prompts P1, P2.... PN satisfying Equation 5
2: Define watermark key list K {k1. k km
3: for each prompt P; and key k; do
4: Initialize count dictionary (v)(() + 0 for all E
5: for w = 1 to W do
6: Construct sampling prompt Pi based watermarking method (see Section 3.3)
7: Generate output yig P k))
8: Ci. (1
9: end for
( P. k2) +- (t) W for all y E V
11: end for
12: Apply rank transformation to all P (-),,, kj)
13: Calculate average similarity S using Equation
14: Compute 7-score: (S 1)/a
15: if &a then
16: return LLM is likely watermarked
17: else
18:
return No evidence of watermarking
19: end if
Table 3: Identification of global fixed key watermarking (e.g., Unigram watermarking) in L.MSs using a prior distribution computed the average output distribution of all eight LLMs. indicates high-confidence watermark identification and indicates low-confidence watermark identification while no color indicates watermark identified.
LLM
Similarity
Z-Score
Unigram
Unwatermark
Unigram
Unwatermark
GPT2
0.59 20.0
0.06 2057
32.49
1.61
OPT1.3B
0.6 2002
0.06
28.77
2.07
OPT2.7B
0.65
0.03
74.81
0.63
LLama2-7B
0.48
0.04
11.31
0.92
LLamm-2-1338
0.38
0.08 0.08
10.17
4.54
1.ma-3.11-80
0.68 4006
0..12
12.09
2.42
Mixtral-TF
0.87 20000
0.038 2.002
22.24
0.74
Qwen2.5-7B
0.53 20008
1 0.028
8.96
3.29
19

---

Published as a conference paper at ICLR 2025
Table 4: Identification of global fixed key watermarking in LLMs using single proxy model's output as prior distribution. indicates high-confidence watermark identification and indicates low-confidence watermark identification while no color indicates watermark identified.
GPT2
OPT138
OPT278
Elann2-78
LLM
Simibrit
2-Scune
Simbbedt
7-Sam:
Simmbril
7-Sare:
Similarity
2-Scmm
NW
NW
NW
NW
LLama2
0.77as
0.07am
0.028e
0.4ee
Prompt9: Used Prompt Set for Global-Fiise key Based Watermarking
①:
Please
generate
random
number
sequence
between
to
1,
2
Please
generate
random
number
sequence
between to
3: Please
generate
random
number
sequence
between to
Please
generate random
number
sequence
between to
9,0,7.2,3,6,5.1 1.4.1,6655,1188444444444444,6,,,
3 Please
generate
random
number
sequence
between
111.7792046.8.0223.6111293
Please
generate
random
number
sequence
between 1o
0009322990443661587777
Please
generate
random
number
sequence
between to
9460993233380044......
3
Please
generate
random
number
sequence
hetween to
9):
Please
generate
random
number
sequence
hetween to
9174566.9221337224466333
10:
Please
generate
random
number
sequence
between 1o
98.074456.1122582224.660333
$ PPI(PP - 1)
Σ PIPPEEP
Sim (0m(RPPM(-1P))) - P.rioc(.PP.)) R(Par(-1P3) - Pprior (.)))) (26)
For the prior distribution, we select N proxy LLMs and calculate the average output probability distribution under prompt p; as the prior distribution. Specifically:
(prior( (-lp.)
PMr (-lp;),
(27)
where M, represents the j-th proxy LLM,, and N is the total number of proxy LLMs used. Specifically, we used the following proxy LLMs: GPT2, OPT1.BB, OPT2.7B, Llama2-7B, Llama2-13B. 8.B. Mixtral-7B, and Qwen2.5-7B.
Llama3-
Additionally, we utilized 10 distinct prompts for computation validate our method as shown in Prompt 9.
The intuition behind this method is that if M watermarked, the differences between output distributions and those of the proxy model should exhibit consistent patterns across different prompts, resulting a high correlation. Conversely, for an unwatermarked model, we assume that the differences in bias between different language models relatively small, and the correlation of these differences should be low.
Table 3 presents the identification results for global-fixed key watermarking using prior distribution. Our method effectively identifies models employing global-fixe -ixed key watermarking, while yielding low z-scores for unwatermarked L.LMss.
20

---

Published as a conference paper at ICLR 2025
Table 5: Details of watermarking algorithms tested in our work.
Algorithm Name
Caregory
Methodology
KGW (Kircheabauer et al., 2023a)
N-Gram
Separate vocabulary set into two lists: a red list and a green list based on the preceding token, then add bias to the logits green tokens that the watermarked text exhibits preference of using green tokens.
KGW-Min (Kirchenbauer al., 2023b)
N-Gram
Similar to KGW, this approach partitions the vocabulary set based on the minimum token ID within a window of N-gram preceding tokens.
KGW-Skip (Kirchenbauer al., 2023b)
N-Gram
Similar to KGW, this appreach partitions the vocabulary set based the left most token within window N gram preceding tokens.
Aar (Aaronson Kirchner, 2022)
N-Gram
Generate pscudo-random vector based on the N-grum proceding tokens to guide sampling at position 1, and choose the token i that maximize ( //elt) (exponential minimum sampling), where p. probability produced by LLM.
"-Reweighh (Hu et al., 2023)
N-Gram
Randomly shuffle probability voctor using a seed based on the pre- eeding N-gram tokens. Discard the left half of the vector, doubling the remaining probabilities. Conduct further sampling using reweighted distribution.
DiPmark (Wu et al., 2023b)
N-Gram
Similar to 7-RReeight, after shuffling, discard the left a portion the and amplify the remaining probabilities 1/(1 o))
EXP-Edit (Kuditipudi et al., 2023)
Fixed-Key-LLit
Based on Aar cuncepl, construct a fixed pseudo random vector list. When generating watermarked text, randomly select start index in the list. For cach watermarked token generation, sequentially use the pecude-zandom vectors from this index expenential minimum sam- pling. During detection, employ edit distance to calculate the correlation between pscudo-rundom vector and the text. Similar to EXP EXP-Edit, this method also uses fixed pseudo random vector However, uses inverse transform sampling instead exponential minimum sampling during token selection.
ITS Edit (Kuditipudi et al., 2023)
Fixed K List
Table 6: Z-scores of waterbag method and Exp-Edit method. represents high-confidence watermark and represents low- con-conieeence watermark, while color means no watermark. This table provides supplementary information on the similarity content Table 2.
KGW w. Water-Bag
Exp-Edi t (Key-len)
None
K K|=I 1
IKU =|=2
|K U F
[KUR] =
420
1024
2048
Watermarked LLM Indentificatiion
Vater-Probe-v1(ne)
-8.20
11.67
-4,22
-784
.4.34
-3.07
-5.87
-5.17
-2.87
24.87
23.49
16.19
3.08
47.74
18.70
6.17
Water-Probe-v2(m-5)
-100.43
28.56
13.68
1,38
-4.41
131.30
78.63
69.40
To further validate the key factors in using prior distribution for testing, we conducted experiments using single proxy model as the prior distribution, as shown in Table 4. We performed cross- experiments with different LLMs. These results demonstrate that global fixed-key watermarking can still be detected when using single LLM as the prior distribution. However, the 7-score detection for unwatermarked LLMs exhibits greater fluctuation. This is primarily due to the significant bias in single proxy model as the prior distribution, affecting the variance of identification and resulting in small z-score.
Although our method using prior distribution achieved good results in our experiments, this identifica- tion approach has limitation in that it can only detect stable biases in LLMs, assuming that stable bias indicates watermark. However, this assumption may not hold in real-world scenarios. challenging to distinguish whether the bias is caused by watermarking or inherent to the LLM itself. This is particularly problematic in cases where LLMS are known to have inherent biases. Future work could investigate more interpretable detection methods.
is
F DETAILS OF TESTED WATERMARKING ALGORITTM
To help understand the watermarking algorithms related to the experiments in this paper, we provide detailed information for all watermarking algorithms in Table 5, including their names, references, types, and brief descriptions. All our experiments were conducted using the MarkLLM (Pan et al., 2024) framework.
21

---

Published as a conference paper at ICLR 2025
Table 7: 7-scores for watermark detection various L.MSs with and without watermarks using Watermark-Prober and Watermark-Probes tion, indicates low-confidence watermark identification, while color indicates no watermark identified. This table provides supplementary information on the similarity content in Table 1.
indicates high-confidence watermark identifica-
N-Gram
Fixed-Key-Sel
LLM
Non
KGW
Aar
KGW-Min
KKW-Skip
DiPmark
Reweigght
EXP-Edit
ITS-Edit
Matermark Probe- 1 (w. prompt I)
Qwen2.5-1.5B
-5.02
43.32
123.57
14.70
24.29
33.35
55.90
-5.21
-25.04
OPT-2.7B
-5.99
49.21
117.95
16.69
35.78
93.25
49.98
-1.32
-1.27
Llama-3.2-3B
-4.52
49.39
79.33
80.94
71.11
87.17
76.35
-6.70
-4.29
Qwen2.5.31
-6.70
625.36
Llamu2-76
-8.20
30.01
109.00
25.51
29.75
44.35
106.37
-3.07
-28.05
Mistral-7B
-3.80
25.03
40.21
35.51
19.99
36.30
137.39
-22.01
-3.31
Qwen2. 5-BB
-1.16
38.25
30.03
22.85
50.34
47.31
50.48
-3.39
-15,92
Llama-3.1-88
-6.44
143.52
29.05
28.19
29.01
125.46
-3.79
-12.70
Llama2-13B
-3.62
29.23
79.42
11.74
18.01
30.40
49.23
-6,63
-2.78
Matermark Probee- 2 (w. prompt 2)
Qwen2.5-1.5B
-5.06
34.55
16.39
8.79
19.45
14.28
10.73
1840.44
OPT-2.7B
-1.95
42.59
67.93
15.17
4.61
32.40
11.04
26.44
1073.13
Llama-3.2-3B
-12.42
29.91
96.14
50.00
19.91
80.04
67.07
34.99
7702.12
Qwen2.5.38
-3.48
39.88
Llama2-78
-2.87
24.87
40.05
32,68
15.62
35.50
25.11
47.74
6885.04
Mistral-7B 7B
-0.87
6.09
54.39
11.14
13.49
49.02
111.12
14.83
1812.12
Qwen2. 5-7B
-2.48
8.90
185.88
10.50
13.06
7.64
12.40
13.04
1982.74
Llama-3.1-8
104.77
10.03
49.23
38.47
12701.95
1amaa.- 138
-3.98
20.72
38.26
10.36
16.60
58.10
83.27
47.74
333.37
Z-score Comparison of Llama2-78 with Different Watermarking Methods Rank)
307
103
Threshold (Z-Gcora
-10F
Water-Probe-v1 (KGW)
Water-Probe-v2 (KGW)
Water-Probe-v2 (EXP-Edit)
Water-Probe-vl (No Watermark)
Water-Probe-v2 (No Watermark)
Temperature
Figure 5: The variation of z-scores at different temperatures when caleulating similarity without using rank transformation in Equation 9.
G ABLATION OF RANK TRANSFORMATION
To illustrate the importance of the rank transformation mentioned in Equation we present in Figure 5 the variation of z-scores at different temperatures without using rank transformation It can be observed that without rank transformation, the z-scores for Unwatermarked LLMs are significantly higher, especially at lower temperatures. Comparing the left plots in Figures 3 and 5, we that rank transformation effectively reduces the z-scores Unwatermarked LLMs, making the identification and detection more stable.
22

---

Published as a conference paper at ICLR 2025
Table 8: Identification p-value for various LLMs with different watermarks and without watermarks, using our two methods: Watermark-Probe-V and Watermark-Probe-ee confidence watermark and represents low-confidence watermark, while no color means no watermark. This table provides supplementary information on the similarity content in Table 1.
represents high-
N-Gram
Fixod-Key-Set
LLM
KGW
KGW-Min
KGW-Skip
DiPmurk
7-reweight
Maternark-Probe-vl (w. prompt I)
Ow..2.5-1.58
2.9c.410
5.9e-3319
3:2a-49
1.3e-100
1/6e-244
2.0c-681
OPT-277B
1.1e-528
3.4e-3024
7.7e-63
1.1e-280
2.6c-1891
2.9e.545
1.9.3-22
1-1.0e1
L.mo--.2-38
1.6e-532
1.tc-1369
1.3e-1425
5.2c-1101
4.4e-1653
7.9e-1269
Qwen2.5-38
2.7c-507
1.8c-3509
2.1e-49
8.3c-84925
1.7c4677
1.6c-403
1-2.4e-2
Hoama2-78
3.6e-198
4.3e-2583
7.6e-144
7.0e-430
1-1.1e3
Mixtral 78
1.4e-388
8.0e-354
1.7e-276
3.4c-89
8. le
3.9e-4102
Qwen2.5-78
1-1.2o-1
2.1e-320
2.0c-198
7.3c-116
4.2e-553
7.9c-489
3.60-556
1Jama-3.
8.4e-93
4.4e44776
7.76-186
3.9e-175
2.5e-188
3.6e-3421
Llama2 13B
4.0c-188
L.ee-372
4.0e:32
8.1e-73
2.7e-203
4,3
1-2.7e:3
Maternark- (w. prompt 2)
0wen2.5.1.58
7.1e-262
3.76:27
9.8e-735500
OPT:277B
1-2.6e-2
1.2e-396
5.6e 1005
2.
2.0e-6
1.4e-300
1.2e-28
2.4e-154
1.2e 250072
Llamo-3.2-38
7.3e-197
3.5e-2010
2.5e-12881755
Qwen2
1.1e-10
1.2c-2556
7.4c-30
2.0c-148
4.4c-548
1.8c-99
3.30-73
7. 7.3--46334822
1lama2-78
1-2.1e-3
7.96-137
4.9e-381
1.5e-234
2.7e-55
2.5e:276
1.9o-139
1.0c-997
4.3e-1029440
Mixtral-7B 70
1-1.9e-1
5.6e 10
3.1e-645
4.0e-29
9.0e-42
1.3e:524
2.0e-2684
4.7e:50
6.5e- 61366
Qwen2.5.7B
1-6.6e-3
2.8e-19
3.9e-7506
4.3e-26
2.8e-39
1.le-14
1.3e-35
3.6e-39
3.1e-853666
L.mm0-31--B
7.3c-141
1.0e-2386
5.6c-24
4,30-529
4.5c-324
2.0c-1440
4.96-216
7.5e-35034440
Laam221338
1,1e-95
Ac--320
1.96255
3,56-62
6.8o-736
1.0c-5088
1.0c-497
2,0e-24136
H SUPPLEMENTARY Z-SCORES AND P-VALUES
To facilitate a better understanding of the statistical methods used in identifying watermarked LLMs, provide detailed information including z-scores and p-values for Table and Table 2 in this section.
Specifically, Table 6 provides supplementary z-score information for Table 2, Table 7 provides sup- plementary z-score information for Table 1, and Table provides supplementary p-value information for Table 1.
For all experiments, we consider a z-score below 4 to indicate no watermark, between and 10 to indicate a watermark with relatively low confidence, and above 10 to indicate watermark with high confidence.
I COMPARISON WITH RELATED WORK (GLOAGUEN ET AL., 2024)
1. Universal Detection:
Our method (particularly Water-Probe-v2) can detect all current watermarking-dure waaeiiiinngunigggeeeeee during-generation approaches (those that modify generation logits or sampling processes). In contrast, Gloaguen et al. (2024)'s method requires specific designs for different watermarking algorithms:
Monte Carlo permutation test for red-green watermarking
Mann-Whine Mann-Whitnyy' U test for EXP-edit watermarking
Potential new methods for future watermarking methods
Our approach represents the first universal detection method effective across all current LLM watermarking techniques.
2. Unified Theoretical Foundation: We provide unified theoretical analysis and explanation for why watermarked LLMs can be detected, specifically demonstrating how watermark key conflicts lead identifiable characteristics in model outputs. This theoretical framework provides a comprehensive understanding of the detection mechanism.
3. Imperceptibility Enhancement: Beyond detection methods, also contribute the Water-Bag approach for improving the impereeptibility of watermarked LLMs, demonstrat- ing significant improvements in watermark concealment while maintaining detectability.
23

---

Published as a conference paper at ICLR 2025
Table 9: Experimental Results with Argmax Sampling On Exp-Edit
Temp
Watermark F1
Perplexity
Gloaguen P-value
Water-Probe-v2 P-value
0.2
0.664
11.8
<0.001
<0.001
0.3
0.666
11.5
<0.001
<0.001
0.4
0.678
11.2
<0.001
<0.001
0.5
0.793
10.9
<0.001
<0.001
0.6
0.907
10.7
<0.001
<0.001
0.7
0.965
10.5
<0.001
<0.001
0.8
0.987
10.4
<0.001
<0.001
0.9
0.987
10.3
<0.001
<0.001
1.0 0.987
10.3
<0.001
<0.001
Table 10: Experimental Results with Multinomial Sampling On Exp-Edit, which is the most challeng- ing case for watermarked LLM identification.
Temp
Watermark F1
Perplexity
Gloaguen P-value
Water-Probe-v2 P-value
0.2
0,666
11.1
<0.001
<0.001
0.3
0.662
11.8
0.33
<0.001
0.4
0.672
11.5
0.83
<0.001
0.5
0.740
11.2
1.0
<0.001
0.6
0.877
11.0
1.0
<0.001
0.7
0.985
10.8
1.0
<0.001
0.8
0.985
10.7
1.0
<0,001
0.9
0.987
10.6
1.0
<0.001
1.0
0.987
10.6
1.0
<0.001
4. Broader Applicability for Challenging Watermarking Variants: Our method supports more challenging watermarking variants. For instance, while Gloaguen et al. (2024)'s exper- iments with EXP-edit only considered argmax sampling after exponential transformation (limiting length-N watermark key list to at most N different sampling results), our method requires no such assumptions.
To demonstrate the broader applicability of our method, we conducted experiments with EXP-edit using sampling after exponential transformation. Given logits 4 first compute probabilities ps through temperature scaling:
exp(L./T) E, exp(l,/T)
(28)
where the temperature parameter. deterministic argmax sampling variant:
While Gloaguen et al. (2024)'s analysis focused on the
= arg ())
(29)
This deterministic approach has fundamental limitation for watermark key list of length N, it can only produce at most N distinct outputs. We instead evaluate multinomial sampling from the distribution:
P(i) x (E.
(30)
Our experiments used the MarkLLM framework with EXP-Edit watermarking (key length = 420), using OPT-1.3B the base model and LLaMA-7B for perplexity caleulations. For Gloaguen et al. (2024)'s testing method, we generated 1,000 text samples of length 200 tokens each. For Water-Prohe-v2 testing, generated 10,000 text samples of length 5 tokens each.
24

---

Published as a conference paper at ICLR 2025
Tables 9 and 10 show that when applying multinomial sampling after EXP transformation, the watermark detection Fl scores and perplexity values remain largely unaffected. However, while our method maintains its effectiveness, Gloaguen et al. (2024)'s method fails to detect the watermark. This demonstrates the broader applicability of our approach to practical watermarking deployments.
J GUIDELINES FOR CONSTRUCTING WATERMARKED LLM IDENTIFICATION PROMPTS
To help better understand method, provide guidelines for constructing watermarked LLM identification prompts, divided into three parts: question space design, answer space design, and implementation and verification protocols.
J.1 OVERALL PROMPT STRUCTURE
The prompts in our identification method consist of two essential components prefix component and a question component. The specific requirements for each component will be detailed in subsequent sections. Here is basic illustration of the structure:
Basic Two-Component Prompt Structure
Input Prompt: Please start your answer with "WXYZ" (prefix component) and then answer question: What is a major city in Asia? (question component)
Response: WXYZ Tokyo
Explanation: The generated prefix would help to fix the watermark key, while the actual would reflect the model's response distribution (achieved by repeated sampling).
J.2 QUESTION COMPONENT DESIGN
As described in Section 3.2 Step 1, we should first construct highly correlated prompts with signifi- cantly overlapping but non-identical answer spaces. This enables easy assessment how potential watermark keys affect the answer spaces of different prompts.
Here a list of criteria for selecting questions:
1. Answer Space Similarity: Select questions with overlapping but non-identical answer spaces. For example:
"Name a country with a large population"
"Name a country with a large area"
"Naee country with high GDP"
"Name country with rich natural resources"
These questions typically share common answers (e.g., USA, China, Russia) while main- taining distinct probability distributions over the answer space.
2. Structural Requirements:
Questions should be concise and unambiguous
Answers should come from well-defined finite set (e.g., countries, cities)
Questions should maintain comparable difficulty levels
The target entity category should remain consistent within suite
J.3 PREFX COMPONENT DESIGN
J.3.1
WATERMARK-PRORE CONSTRUCTION
For Watermark- Eitermark-PPobe--i 11, simply instruct the LLM to generate fixed prefix before answering the question through explicit prompt instructions.
Here the design principles for the prefix component:
25

---

Published as a conference paper at ICLR 2025
1. Use meaningless character sequences (e.g., "abed", "wxyz") that have semantic meaning in any language
2. Avoid any sequences that could form acronyms, abbreviations meaningful patterns
3. Ensure the prefix is completely unrelated to any potential answers or question domains
4. Keep the prefix length sufficient for determining the watermark key while maintaining semantic independence
Here is an example of the prefix component:
Implementation Example for Natermark-Probe--
Please generate abed before answering the question.
Question: Name country with a large population.
Answer: abcd India
Explanation: The generated prefix is meaningless and unrelated the question domain, cnsuring that docs not introduce any contextual bias.
J.3.2 WATERMARK- WATERMARK-PRORE--2 CONSTRUCTION
For Watermark Pronkk--PPbb222 v2, we need design a controlled randomization process before answering the question to help fix the watermark key (see Section 3.3 for detailed reasons).
Here are the design principles for the prefix component:
1. Ensure the prefix generation does not influence the answer to the main question
2. Design multiple choice sets with logically equivalent probabilities
3. Keep the number of choices moderate and manageable
4. Maintain clear boundaries between different choice sets
Here an example of the prefix component:
Implementation Example for Watermark-Probe-
Please generate sentence that satisfies the following conditions:
First word: Randomly sampled from A-Z
Second word: Randomly sampled from zero to nine
Third word: Randomly sampled from (cat, dog, tiger, lion)
Then answer: Name a country with large population.
Answer: A one cal China
Explanation: All the possible generated prefixes not related the question domain, ensuring that they do not introduce any contextual bias.
K
THREAT MODEL
In this section, we outline the threat model under which our watermark identification method (detector) operates. We consider the capabilities and limitations both the detector and the LLM service provider.
K. DETECTOR CAPABILITIES
We assume the detector:
Has black-box access to the LLM through standard API interfaces
Can only interact with the model through normal prompt-response queries
Has access to model architecture, parameters, or training data
Can perform multiple queries
Cannot modify influence the model's internal state
26

---

Published as a conference paper at ICLR 2025
K.2
TRUST ASSUMPTIONS
The threat model assumes:
The LLM service provider may embed watermarks in the model outputs
The API interface itself is trustworthy and returns genuine model outputs
No man-in-the-middle attacks or response tampering occurs
The detection process does not require knowledge of specific watermarking algorithms
K.3
DETECTION GOALS AND CONSTRAINTS
The primary objectives within this threat model are:
Determine the presence absence of watermarks in model outputs
Maintain detection accuracy across different sampling temperatures and model configura- tions
Key constraints include:
Detection must be performed solely through black-box testing
Watermark removal or tampering is outside the scope
Detection methods must be robust against normal model output variations
L TEST ON CLOSEED-SOURCE MODELS
We evaluated Water-Probe-V2's detection capabilities several closed-source models, including GPT-4o-mini, GPT-4o, GPT-3.5-turbo. Gemini-1.55flan and Gemini-1.5-pro For experiments, utilized the latest API versions of these models (as of November 15, 2024) with temperature setting of 0.7.
Table 11: Watermarked LLM Identification Results on Closed-source Models
Model
Similarity
Std Dev
Z-score
Watermarked?
GPT-4o-mini
-0,005
0.018
-5,984
No
GPT-40
0.017
0.020
-4.11
No
GPT-3.5-turbo
0.028
0.030
-2.362
No
Gemini-1.5-flan
0.027
0.049
-1,474
No
Gemini-1..5pro 0.018
0.038
-2.135
No
Our experimental results provide strong evidence that current closed-source model APIs do not contain watermarks. However, it is important note that key limitation of this experiment is our inability to verify ground truth labels, making it impossible to definitively confirm the accuracy of our detection results.
M REVERSION KEY CALCULATION OF WATER-BBI -BAG
In this section, provide detailed calculations for determining reversion keys that satisfy the constraints Equation 11. Let p = PM x. V14-1) represent the original model distribution, and q Fp, f(K,, (i-nai-)) represent the distribution after modification using key Ky.
According to Equation 11, we have:
} (q + F(p.f(K, Mi-ni-1)))
1
(31)
Through algebraic manipulation, we can derive the required modification for the reversion key: Ppp.f( Di- (i--ni-))) 2p 4
(32)
27

---

Published as a conference paper at ICLR 2025
This equation provides the conerete method for calculating the reversion key Kj. Specifically, for any input sequence yi-nzi-1, the function f [(K,, Vi-n:i-)) must map the original distribution p to 2p q to satisfy Equation 11.
It is important to note that reversion key need not be restricted to numerical values. Any key that produces the required distributional modification qualifies as a valid reversion key, as long as accurately satisfies the constraint equation.
28


Reviews:
[{"review_id": "D6U1irciZl", "reviewer": "Reviewer_GLo8", "summary": "Considering that “even if individual watermarked texts are imperceptible, the distribution of numerous watermarked texts may reveal whether the LLM is watermarked, especially when repeatedly sampling with the same watermark key”, this paper proposed Water-Probe to detect if an LLM service contains watermarks through well-designed prompts then introduced Water-Bag strategy to improve watermark imperceptibility by merging multiple watermark keys.", "questions": "How about the blackbox detection on LLMs in real world? What is the difficulty?", "limitations": "", "rating": 8, "confidence": 4, "soundness": 4, "presentation": 4, "contribution": 4, "strengths": "1 This work is the first study on the imperceptibility of watermarked LLMs. \n2 This paper is well organized and written, make it easy to follow.\n3 The experiments are conducted across different LLMs and with different sampling methods and temperature settings. The conclusion and discussion based on the evaluation results are clear.", "weaknesses": "The threat model should be further described, especially in terms of the prior knowledge  assumptions of the detector.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "9O4ijU7Aa3", "reviewer": "Reviewer_DykZ", "summary": "The paper proposed a method called Water-Probe to detect whether a LLM is watermarked through well-designed prompts. Experiments show the method is effective to almost all mainstream watermarking algorithms. The authors also proposed a strategy to improves watermark imperceptibility by multiple keys.", "questions": "1. In Table 1, why Water-Probe-V2 has low detection confidence for KGW series watermarks?\n2. Could you provide some experiments on watermark detection on closed-source models such as ChatGPT using your tool? Since the proposed method is a black-box setting, this should be achievable.\n3. In the method for enhancing the key pseudorandomness of the Water-Bag Strategy, your use of the \"set of master keys and their inversions\" approach lacks theoretical support. Does \"inversions\" refer to bit flips, or does it refer to certain keys that satisfy the constraints of equation (12)? If it’s the former, then the claim of enhanced pseudorandomness has no basis; if it’s the latter, the paper does not explain how to derive the inversion key space that satisfies equation (12), and it seems that satisfying (12) does not have a direct relationship with the pseudorandomness of $K^*_j$. The experiments in Table 2 test methods against the attacks proposed by the authors, without directly testing the pseudorandomness of the sequences in the key space using suites like NIST SP 800-22. Ultimately, this seems to only indicate that the specific method enhances defense capabilities against the proposed attacks, rather than demonstrating that the enhancement of pseudorandomness leads to improved defense capabilities, which is inconsistent with the authors' previous claims.", "limitations": "", "rating": 8, "confidence": 3, "soundness": 3, "presentation": 3, "contribution": 3, "strengths": "1. It is reasonable to use repeated sampling to detect whether an LLM has been watermarked. \n2. Prompts have been designed to reveal the connection between generated text and watermark keys in a black-box setting.\n3. The evaluation is comprehensive, and many different watermarking methods have been tested. Experimental results also show that the proposed method works.", "weaknesses": "1. Symbol reuse. In Section 3 and Section 4, the symbol $P$ represents both the model distribution in $P_M$ and the prompt in $P_1, P_2, ..., P_N$, which can be confusing. Too many $P$ across line 222 to line 227.\n2. Line 289, Figure 2. ‘KTH’ does not refer to anything else in other contexts. Is it Exp-Edit or ITS-Edit?", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "HLqGRGLL9r", "reviewer": "Reviewer_qXQr", "summary": "This paper addresses the imperceptibility of watermarking techniques used in Large Language Models (LLMs). It highlights that while existing watermarking techniques are effective in terms of detectability, minimal impact on text quality, and robustness to editing, they have not been thoroughly examined for their imperceptibility to users. The authors introduce an identification algorithm, Water-Probe, which is capable of detecting watermarks through carefully designed prompts. To enhance watermark imperceptibility, the authors propose a Water-Bag strategy, which merges multiple watermark keys to increase randomness.", "questions": "- **Concept Confusion**: the authors mislead watermarking concept.\nLLM Watermarking is the process of injecting a invisible identifier into LLM, track unauthorized distribution, or indicate authenticity, withch contains two steps **injection** and **detection**. The author just talk about how to identify LLM output consistency, I never see any discussion about watermark injection in paper. The authors should clarify how their work relates to the injection and detection steps they have outlined, or explain why their approach focuses solely on identification without addressing injection.\n\n- **Unknown Key Sampling Strategy**: the author propose the watermark identification method using setected prompts.\nStep 1-3 only discuss the object of identification key sampling, never talk about the detail of identification key sampling.\n\n- **Method Designed**: The author says “Research on the imperceptibility of watermarked LLMs should investigate whether watermarked and non-watermarked LLMs can be distinguished.”. However, in this paper, I never see the sampled prompts and identification method can solve this problem. The authors should clearly illustrate how the proposed method differentiates between watermarked and non-watermarked LLMs. I recommend including a specific experiment or analysis that directly addresses this assertion.", "limitations": "", "rating": 6, "confidence": 4, "soundness": 3, "presentation": 3, "contribution": 2, "strengths": "- **Accuracy and Robustness**: The experimental results demonstrate the high accuracy and robustness of the Water-Probe algorithm across various LLMs, watermarking methods, and generation settings. The low false positive rate for non-watermarked LLMs further strengthens the algorithm’s reliability.\n- **Practical Solution**: The proposed Water-Bag strategy offers a practical solution to improve the imperceptibility of watermarks, which is a critical concern for LLM providers.", "weaknesses": "- **Concept Confusion**: The manuscript mislead concept of watermarking and fingerprinting, the proposed method should be categorized into fingerprinting instead of watermarking. see Question 1.\n- **Lack of Controbution**: The authors inputs prompts to see the response of watermarked LLM and non-watermarked LLM, which is so called identification algorithm. The manuscript just defeines some concept, samples prompt to see similarity of the inspected models. \n- **Limited Scope of Water-Probe**: The Water-Probe algorithm might be limited in its application to only certain types of watermarking schemes. The paper does not sufficiently explore how the algorithm would perform against a wider range of watermarking techniques, especially those that are more sophisticated or less predictable.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "yxtMUMhHd3", "reviewer": "Reviewer_AeYS", "summary": "The paper proposes an identification algorithm to detect watermarks through well-designed prompts to LLMs, as current watermarked LLMs have consistent biases under the same watermark key.", "questions": "Why is the similarity of distribution differences between prompts in the no-watermark case low?", "limitations": "", "rating": 8, "confidence": 3, "soundness": 3, "presentation": 3, "contribution": 3, "strengths": "The paper identifies watermarks in a black-box setting. Also, it proposes an effective method to improve watermark imperceptibility.", "weaknesses": "The primary concern I have with the paper is the similarity of distribution differences between prompts in the no-watermark case. First, cosine similarity measures the similarity between two non-zero vectors; thus, the equation Sim(0, 0) in the proof in Appendix B is confusing. Second, if we examine the distances of the elements directly rather than using the Sim function, the distances of the elements in the no-watermark case are approximately zero, suggesting high similarity.\n\nThe selection of prompts appears empirical. The paper could be strengthened by providing guidance on generating prompts that produce similar output distributions.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}]

Author rebuttal:

Decision: Accept (Spotlight)