Can Watermarked LLMs be Identified by Users via Crafted Prompts?

## Abstract

Abstract Text watermarking for Large Language Models (LLMs) has made significant progress in detecting LLM outputs and preventing misuse. Current watermarking techniques offer high detectability, minimal impact on text quality, and robustness to text editing.
However, current researches lack investigation into the imperceptibility of watermarking techniques in LLM services.
This is crucial as LLM providers may not want to disclose the presence of watermarks in real-world scenarios, as it could reduce user willingness to use the service and make watermarks more vulnerable to attacks. This work is the first to investigate the imperceptibility of watermarked LLMs. We design an identification algorithm called Water-Probe that detects watermarks through well-designed prompts to the LLM. Our key motivation is that current watermarked LLMs expose consistent biases under the same watermark key, resulting in similar differences across prompts under different watermark keys. Experiments show that almost all mainstream watermarking algorithms are easily identified with our well-designed prompts,
while Water-Probe demonstrates a minimal false positive rate for non-watermarked LLMs.
Finally, we propose that the key to enhancing the imperceptibility of watermarked LLMs is to increase the randomness of watermark key selection. Based on this, we introduce the Water-Bag strategy, which significantly improves watermark imperceptibility by merging multiple watermark keys.

### 1 Introduction

The rapid advancement of large language models (LLMs) has led to remarkable achievements in tasks such as question answering (Zhuang et al., ), programming (Jiang et al., ), and reasoning (Wei et al., ), with widespread applications across various scenarios. However, the extensive use of LLMs has also raised concerns regarding copyright protection and misuse. Recent research indicates that malicious attackers can steal LLMs through model extraction techniques (Yao et al., ), and some users may abuse LLMs to generate and spread harmful information (Wei et al., ).

Text watermarking techniques for LLMs have become an important method to mitigate the above issues by adding detectable features to LLM outputs (Liu et al., ). Recent researches on LLM watermarking have focused on improving watermark detectability (Kirchenbauer et al., ), minimizing impact on generated text (Aaronson & Kirchner, ), and enhancing robustness against text modifications (Liu et al., ). However, no work has considered the imperceptibility of watermarked LLMs, i.e., whether users can know if an LLM service is watermarked. In real-world scenarios, LLM service providers may not disclose the existence of watermarks, as it could reduce user willingness to use the service and make it more vulnerable to attacks (Sadasivan et al., ). As more LLM services consider implementing watermarks, it is crucial to investigate whether users can identify watermarked LLMs solely through crafted prompts.

Some studies focus on the imperceptibility of watermarked text, ensuring watermarked and non-watermarked texts are indistinguishable (Hu et al., ; Wu et al., ). However, even if individual watermarked texts are imperceptible, the distribution of numerous watermarked texts may reveal whether the LLM is watermarked, especially when repeatedly sampling with the same watermark key (Wu et al., ). While some studies explore cracking watermarks using large volumes of watermarked text (Jovanović et al., ; Sadasivan et al., ; Wu & Chandrasekaran, ; Zhang et al., ), they assume the target LLM is watermarked and cannot determine if the LLM is watermarked. Research on the imperceptibility of watermarked LLMs should investigate whether watermarked and non-watermarked LLMs can be distinguished.

Figure: Figure 1: Illustration of our Water-Probe algorithm for identifying watermarked LLMs. We first construct two prompts with similar output distributions, then sample repeatedly using two fixed watermark keys for each prompt. The presence of a watermark is determined by comparing the similarity of distribution differences between the two prompts. Details in section 4.
Refer to caption: x1.png

In this work, we conduct the first study on the imperceptibility of watermarked LLMs. We propose the Water-Probe algorithm to identify watermarked LLMs under two main types of watermarking methods. Our motivation stems from a key observation: current LLM watermarking algorithms expose consistent bias when repeatedly sampled under the same watermark key. Based on this, our Water-Probe algorithm first crafts prompts to perform repeated sampling under the same watermark key, then compares the consistency of sampling distribution differences across different prompts under a pair of watermark keys. Highly consistent differences indicate a watermarked LLM.

In our experiments, we demonstrate that the Water-Probe algorithm achieves high accuracy in detecting various types of watermarked LLMs. We also show its applicability across different LLMs, maintaining a low false positive rate for non-watermarked LLMs. Furthermore, our algorithm exhibits robust performance across different sampling methods and temperature settings.

Finally, we explore methods to enhance the imperceptibility of watermarked LLMs. We find that increasing the randomness of watermark key selection is crucial, as it makes it more difficult to construct prompts for repeated sampling using the same key. Based on this, we propose the Water-Bag algorithm, which combines multiple watermark keys into one, randomly selecting a key for each generation and choosing the highest score for detection. However, increasing key selection randomness often leads to a slight decrease in detectability. Addressing this trade-off should be an important direction for future work.

Our main contributions can be summarized as follows:

- •
We conduct the first study on the imperceptibility of watermarked LLMs, proposing the Water-Probe algorithm to effectively identify watermarked LLMs by exploiting consistent biases in watermarking keys.
- •
We demonstrate the high accuracy and robustness of our algorithm across various LLMs, watermarking methods, and generation settings, while maintaining a minimal false positive rate for non-watermarked LLMs.
- •
We introduce the Water-Bag algorithm to enhance LLM watermark imperceptibility.

### 2 Related Work

Large language model (LLM) watermarking techniques (Liu et al., ) have become crucial for copyright protection (Sander et al., ), generated text detection (Wu et al., ), and preventing misuse (Liu et al., ). These watermarking algorithms introduce subtle features during text generation that are almost imperceptible to humans but detectable by designed detection algorithms. LLM watermarking can be broadly categorized into two types. The first type is post-processing watermarking, which modifies generated text using format-based (Sato et al., ), lexical-based (Yang et al., ), syntax-based (Wei et al., ), or generation-based (Zhang et al., ) approaches to add watermark. However, post-processing watermarking methods require waiting for text generation to complete before modifying and adding watermarks, which is not suitable for current LLM services that require real-time text generation. Another category of watermarking algorithms, known as watermarking during generation, typically involves adjusting the distribution of the next generated token based on a watermark key. For instance, the KGW (Kirchenbauer et al., ) algorithm divides the vocabulary into red and green lists, increasing the probability of tokens in the green list. SIR (Liu et al., ) further modifies logits based on semantic information, enhancing watermark robustness. UPV (Liu et al., ) utilizes two independent networks for watermark embedding and detection, enabling public verification. Additionally, some research has explored watermarking techniques for code generation scenarios (Lu et al., ) or perform watermark detection on longer documents (Pan et al., ) or even translated documents (He et al., ). An important objective of these methods is to maintain the imperceptibility of generated text, i.e., watermarked and non-watermarked text should have identical distributions, with some distortion-free algorithms showing promising results (Kuditipudi et al., ; Aaronson & Kirchner, ). However, previous work has overlooked the imperceptibility of watermarked LLMs themselves, i.e., whether external users can detect if an LLM service contains watermarks without disclosure. This work investigates the imperceptibility of watermarked LLMs.

### 3 Preliminaries

To facilitate subsequent discussion, we introduce LLMs and LLM watermarking algorithms, then present the objectives of this work. We first provide the definition of LLM used in this work as follows:

###### Definition 1 (Large Language Model) .

An LLM $M$ is a function that, given an input $x$ and a partial output sequence $y_{1:i-1}$, produces a probability distribution $P_{M}(y_{i}|x,y_{1:i-1})$ over possible next tokens $y_{i}$. The model generates complete outputs by iteratively sampling from these distributions.

Since this work focuses solely on watermarks embedded in LLM services, we will only consider the watermark during generation type mentioned in section 2. The definition of the watermark rule is given below.

###### Definition 2 (Watermark Rule) .

A watermark rule is typically a function $F$ that adjusts the current LLM’s predicted probability distribution based on a watermark key $k$ to obtain a new probability distribution. Formally, given an LLM $P_{M}$ and a key $k$, the watermark rule $F$ modifies the distribution as follows:

$$ $P_{M}^{F}(y_{i}|x,y_{1:i-1},k)=F(P_{M}(y_{i}|x,y_{1:i-1}),k)$ (1) $$

where $P_{M}^{F}$ is the modified probability distribution for the next token $y_{i}$.

The main difference between watermarking algorithms lies in how they determine the watermark key. Based on this, we categorize watermarking algorithms into n-gram based watermarking and fixed-key-list based watermarking. We will now introduce these two types of watermarking algorithms.

###### Definition 3 (N-Gram Based Watermarking) .

In n-gram based watermarking, the watermark key $k_{i}$ for generating the current token $y_{i}$ is determined by a function $f$ that takes two inputs:

$$ $k_{i}=f(K,y_{i-n:i-1})$ (2) $$

where $K$ is a pre-selected master key, and $y_{i-n:i-1}$ represents the previous $n$ tokens.

N-gram based watermarking ensures that for the same n-token prefix, the watermark key for generating the next token remains consistent. This approach is widely used in current watermarking algorithms, including KGW (Kirchenbauer et al., ), KGW-V2 (Kirchenbauer et al., ), Aar (Aaronson & Kirchner, ), DiPmark (Wu et al., ), and SIR (Liu et al., ). Next, we define fixed-key-list-based watermarking:

###### Definition 4 (Fixed-Key-List Based Watermarking) .

Let $K=\{k_{1},k_{2},...,k_{m}\}$ be a fixed key list. For a given starting index $s\in\{1,...,m\}$, the watermark key $k_{i}$ for generating the $i$-th token is:

$$ $k_{i}=k_{((s+i-1)\bmod m)+1}$ (3) $$

where the starting index $s$ may be randomly chosen for each generation process.

This approach is employed in algorithms such as Exp-Edit (Kuditipudi et al., ), where keys are used sequentially from a potentially random starting position in the key list.

To formalize our approach, we define our goal as follows:

###### Definition 5 (Black-box Watermark Identification) .

A function $\mathcal{D}:P_{M}\rightarrow\{0,1\}$ that classifies a language model $P_{M}$ as watermarked (1) or not (0), without access to its internal parameters.

###### Definition 1 (Large Language Model) .

###### Definition 2 (Watermark Rule) .

###### Definition 3 (N-Gram Based Watermarking) .

###### Definition 4 (Fixed-Key-List Based Watermarking) .

###### Definition 5 (Black-box Watermark Identification) .

### 4 Watermarked LLM Identification

In this section, we introduce our Water-Probe algorithm, which identifies watermarks in LLMs through well-designed prompts. We first analyze why watermarked LLM identification is possible, then present the general algorithm process as shown in Fig. 1, and finally describe two specific algorithm variants.

#### 4.1 Why Watermarked LLM Identification is Possible

Definition 2 implies that LLMs typically introduce some distortion to the distribution. However, there exist distortion-free watermarking algorithms that satisfy Definition 2. We first provide a definition and then demonstrate that LLMs with distortion-free watermarks can still be identified.

###### Definition 6 (Distortion-Free Watermark) .

A watermarking algorithm is considered distortion-free if, for all possible inputs $x$ and partial output sequences $y_{1:i-1}$, the expected output distribution of the watermarked model $P_{M}^{F}$ over all possible watermark keys $k\in\mathcal{K}$ is identical to the original model $P_{M}$:

$$ $\mathbb{E}_{k\in\mathcal{K}}[P_{M}^{F}(y_{i}|x,y_{1:i-1},k)]=P_{M}(y_{i}|x,y_{ 1:i-1}).$ (4) $$

This equation indicates that the expected output distribution of watermarked text remains unchanged when the watermark key is randomly selected across all possible keys. However, as shown in Equation 1, sampling with a specific watermark key $k$ introduces a difference between $P_{M}^{F}(y_{i}|x,y_{1:i-1},k)$ and $P_{M}(y_{i}|x,y_{1:i-1})$. This observation leads to the following theorem on detectability of watermarked LLMs:

###### Observation 1 (Distributional Difference of Watermarked LLMs) .

Let $P_{M}$ be a language model and $F$ a watermark rule as defined in Definition 2. For a given watermark key $k$, the probability distribution of the watermarked model $P_{M}^{F}(y_{i}|x,y_{1:i-1},k)$ differs from the original distribution $P_{M}(y_{i}|x,y_{1:i-1})$. This distributional difference suggests the potential detectability of the watermark.

###### Definition 6 (Distortion-Free Watermark) .

###### Observation 1 (Distributional Difference of Watermarked LLMs) .

#### 4.2 Pipeline of Watermarked LLM Identification

Observation 1 implies that the key to identifying a watermarked LLM is to construct prompts that allow for multiple samplings using the same watermark key to reveal the difference between $P_{M}^{F}(y_{i}|x,y_{1:i-1},k)$ and $P_{M}(y_{i}|x,y_{1:i-1})$. However, due to the black-box setting, we cannot directly access the origin logits $P_{M}(y_{i}|x,y_{1:i-1})$. Instead, we calculate the difference in LLM outputs for two distinct keys, defined as $\Delta(p,k_{m},k_{n})=P_{M}^{F}(y_{1}|p,k_{m})-P_{M}^{F}(y_{1}|p,k_{n})$, to aid in identification.

If two prompts yield similar output distributions, the same watermark key should have similar effects on both prompts (proven in Theorem 1). We determine if an LLM contains a watermark by comparing the consistency of the effects of two watermark keys on two similar prompts. Specifically, given $p_{1}$, $p_{2}$, $k_{1}$, and $k_{2}$, we assess the similarity between $\Delta(p_{1},k_{1},k_{2})$ and $\Delta(p_{2},k_{1},k_{2})$. High similarity indicates the presence of a watermark; otherwise, we conclude there is no watermark.
Based on the above analysis, we now present the process of the Water-Probe algorithm.

Step 1: Construct highly correlated prompts. Construct $N$ prompts $P_{1},P_{2},...,P_{N}$ such that their output probability distributions under $M$ are highly similar, which can be expressed as:

$$ $\forall i,j\in\{1,2,...,N\},\text{KL}(P_{M}(\cdot|P_{i})||P_{M}(\cdot|P_{j})) \leq\epsilon\text{ and }P_{i}\neq P_{j}$ (5) $$

where $\text{KL}(\cdot||\cdot)$ is the Kullback-Leibler divergence, $P_{M}(\cdot|P_{i})$ is the output probability distribution for prompt $P_{i}$ under the LLM $M$, and $\epsilon$ is a small threshold indicating high similarity between distributions.

Step 2: Sampling with simulated fixed watermark keys. Since we cannot access the logits under a given watermark key, we need to use repeated sampling to estimate the distribution. We construct a set of simulated watermark keys $K=\{k_{1},k_{2},...,k_{m}\}$ based on our prompt design (detailed in subsequent sections). For each prompt $P_{i}$ and each simulated key $k_{j}\in K$, we estimate the distribution as follows:

$$ $\hat{P}_{M}^{F}(y|P_{i},k_{j})=\frac{1}{W}\sum_{w=1}^{W}\mathbf{1}_{y_{i,j}^{w }=y},\quad\text{where }y_{i,j}^{w}\sim P_{M}^{F}(y|P_{i},k_{j})$ (6) $$

where $W$ is the sample count, $\mathbf{1}_{A}$ is the indicator function, and $y_{i,j}^{w}$ is the $w$-th sample sampled from $P_{M}^{F}(y|P_{i},k_{j})$. Specific prompt techniques for different watermarking algorithms will be detailed later.
Note that our prompt design for simulating watermark keys assumes the target LLM has a watermark. If it doesn’t, then $P_{M}^{F}(y|P_{i},k_{j})=P_{M}(y|P_{i})$ for all simulated keys.

Step 3: Analyze Cross-Prompt Watermark Consistency.
We first assume that the watermark rule satisfies Lipschitz continuity. Based on this assumption, we can deduce that the differences in output distributions produced by a watermark key pair for highly correlated prompts are similar.

###### Assumption 1 (Lipschitz Continuity of Watermark Rule) .

For prompts $p_{1}$ and $p_{2}$ satisfying the similarity condition in Equation 5, the watermark rule $F$ satisfies Lipschitz continuity. That is, there exists a constant $L>0$ such that for the probability distributions $P_{M}(\cdot|p_{1}),P_{M}(\cdot|p_{2})$ and any watermark key $k\in\mathcal{K}$:

$$ $\|F(P_{M}(\cdot|p_{1}),k)-F(P_{M}(\cdot|p_{2}),k)\|_{1}\leq L\cdot\|P_{M}( \cdot|p_{1})-P_{M}(\cdot|p_{2})\|_{1}.$ (7) $$

###### Theorem 1 (Consistency of Watermark Effect) .

Let $p_{1}$ and $p_{2}$ be two different prompts satisfying the similarity condition in Equation 5.
Let $k_{1}$ and $k_{2}$ be two randomly sampled watermark keys from the key space $\mathcal{K}$.
The effect of applying these keys on the output distribution should be highly consistent across prompts:

$$ $\mathbb{E}_{k_{1},k_{2}\sim\mathcal{K}}[\text{Sim}(P_{M}^{F}(\cdot|p_{1},k_{1} )-P_{M}^{F}(\cdot|p_{1},k_{2}),P_{M}^{F}(\cdot|p_{2},k_{1})-P_{M}^{F}(\cdot|p_ {2},k_{2}))]\geq\rho$ (8) $$

where $P_{M}^{F}$ is the watermarked distribution, and $\text{Sim}(\cdot,\cdot)$ is a similarity measure (e.g., cosine similarity), and $\rho$ should be a constant significantly greater than 0.

Theorem 1 implies that for prompts with similar output distributions under the LLM, the expected differences introduced by the two watermark keys should also be similar. The formal proof is provided in Appendix A.

Based on Theorem 1, we calculate the average similarity using the estimated distributions from Step 2. To ensure stability across different sampling temperatures, we first apply a rank transformation. Specifically, for a token $y$, its rank is defined as the number of tokens with probability greater than or equal to that of $y$, denoted as $R(P)(y|x)=|\{y^{\prime}\in V:P(y^{\prime}|x)\geq P(y|x)\}|$. We then compute the expected similarity:

$$ $\bar{S}=\frac{1}{N}\sum_{P_{i}\neq P_{j}\in\mathcal{P}}\sum_{k_{m}\neq k_{n} \in\mathcal{K}}\text{Sim}(\Delta_{R}(P_{i},k_{m},k_{n}),\Delta_{R}(P_{j},k_{m} ,k_{n}))$ (9) $$

Here, $\mathcal{P}$ is the prompt set, $\mathcal{K}$ is the watermark key set, $N=|\mathcal{P}|(|\mathcal{P}|-1)|\mathcal{K}|(|\mathcal{K}|-1)$, and $\Delta_{R}(P,k_{m},k_{n})=R(\hat{P}_{M}^{F}(\cdot|P,k_{m}))-R(\hat{P}_{M}^{F}(
\cdot|P,k_{n}))$.
We verify the importance of rank transformation in Appendix G.

According to Theorem 1, if $M$ contains a watermark, the similarity obtained from Equation 9 should be significantly greater than 0. If $M$ does not contain a watermark, we assume $P_{M}^{F}=P_{M}$ for any $k$, so Equation 9 should represent the similarity between two random vectors with zero mean, which should be close to 0. A detailed analysis of the no-watermark case is provided in Appendix B.

Based on this, we design the following z-test to perform hypothesis testing on the average similarity:

$$ $z=(\bar{S}-\mu)/\sigma$ (10) $$

where $\bar{S}$ is the observed average similarity, $\sigma$ is the standard deviation of the $\bar{S}$, and $\mu$ is the mean of the $\bar{S}$ under the no-watermark case. Theoretically, $\mu$ should be 0 for an unwatermarked LLM. However, in practice, we may choose a value slightly greater than 0 to account for potential biases introduced by our prompt construction method or other factors. The standard deviation $\sigma$ is estimated through multiple experiments.

We reject the null hypothesis (no watermark) and conclude the LLM is likely watermarked if:
$z>z_{\alpha}$, where $z_{\alpha}$ is the chosen significance level $\alpha$.
In this work, we consider a z-score between 4 and 10 as moderate confidence, and above 10 as high confidence.

###### Assumption 1 (Lipschitz Continuity of Watermark Rule) .

###### Theorem 1 (Consistency of Watermark Effect) .

#### 4.3 Constructing Repeated Sampling With Same Watermark Key

In the previous section, we introduced the basic pipeline of our Water-Probe algorithm. A key challenge is constructing prompts that enable multiple samplings with the same watermark key. This varies for different watermarking algorithms. We’ll discuss approaches for n-gram based and fixed-key-list based methods.

For N-gram based watermarking (Definition 3), since the watermark key is derived from the previous N tokens, we can design prompts that make the LLM generate N irrelevant tokens before following the prompt. An example is provided below:

- Prompt 1: Example Prompt for Watermark-Probe-v1
Please generate abcd before answering the question.
Question: Name a country with a large population.
Answer: abcd India

In the example above, we assume generating abcd does not affect the distribution for answering the question. However, in practice, it’s challenging to ensure completely irrelevant tokens. Consequently, the $\bar{S}$ for an unwatermarked LLM constructed this way may be slightly above 0. We refer to the Watermark-Probe using prompts similar to the above table as Watermark-Probe-v1.

For fixed-key-set based watermarking, since the start watermark key is randomly selected each time, our approach is to approximate multiple samplings with the same watermark key by exploiting the correlation between the watermark key and the generated tokens. Specifically, we prompt the LLM to perform some quasi-random generation initially. Generally, the same watermark start key will only generate a few fixed sampling results, so we can assume that identical sampling result prefixes are generated by the same watermark key. Here’s a specific example of how we construct prompts for this approach:

- Prompt 2: Example Prompt for Watermark-Probe-v2
Please generate a sentence that satisfies the following conditions:
The first word is randomly sampled from A-Z. The second word is randomly sampled from zero to nine. The third word is randomly sampled from cat, dog, tiger and lion. Then answer the question: Name a country with a large population.
Answer: A one cat China

As shown in the example above, we first prompt the LLM to generate N (3 in this case) random tokens before answering the question. Different random token combinations typically correspond to a few watermark keys. Figure 2 illustrates the distribution of watermark key counts for varying numbers of random tokens. As evident from the figure, given a fixed prefix, the vast majority of cases utilize a specific key. So this approach thus approximates sampling with the same watermark key.
Similarly, we refer to the prompting method in the above table as Watermark-Probe-v2.

We provide the detailed steps of the Water-Probe algorithm in Algorithm 1 in the appendix.

Figure: Figure 2: Distribution of start keys for identical prefixes in KTH watermarking. Analysis based on prompts described in Section 4.3 for Watermark-Probe-v2. Each subplot represents a specific prefix(in title).
Refer to caption: x2.png

### 5 Experiment On Watermarked LLM Identification

#### 5.1 Experiment Setup

**Table 1: Detection similarities for various LLMs with and without different watermarks, calculated using Equation 9 and our two identification methods: Water-Probe-v1 and Water-Probe-v2. indicates high-confidence watermark identification and indicates low-confidence watermark identification while no color indicates no watermark identified. The corresponding z-scores can be found in Table 7 in the Appendix.**
| LLM | N-Gram | Fixed-Key-List |  |  |  |  |  |  |  |
| --- | --- | --- | --- | --- | --- | --- | --- | --- | --- |
| Non | KGW | Aar | KGW-Min | KGW-Skip | DiPmark | $\gamma$-Reweight | EXP-Edit | ITS-Edit |  |
| Water-Probe-v1 (w. prompt 4.3) |  |  |  |  |  |  |  |  |  |
| Qwen2.5-1.5B | 0.02 ± 0.02 | 0.37 ± 0.02 | 0.88 ± 0.06 | 0.37 ± 0.02 | 0.39 ± 0.01 | 0.55 ± 0.01 | 0.55± 0.01 | 0.01± 0.02 | 0.00± 0.04 |
| OPT-2.7B | 0.05 ± 0.01 | 0.47 ± 0.01 | 0.91 ± 0.01 | 0.42 ± 0.02 | 0.45 ± 0.01 | 0.60 ± 0.01 | 0.61± 0.01 | 0.08 ± 0.02 | 0.09± 0.01 |
| Llama-3.2-3B | 0.04 ± 0.02 | 0.53 ± 0.01 | 0.90 ± 0.01 | 0.48 ± 0.00 | 0.49 ± 0.01 | 0.61 ± 0.01 | 0.61± 0.01 | 0.03 ± 0.01 | 0.04± 0.01 |
| Qwen2.5-3B | 0.03 ± 0.01 | 0.33± 0.02 | 0.75± 0.05 | 0.33± 0.02 | 0.38 ± 0.00 | 0.51± 0.01 | 0.53± 0.01 | 0.03± 0.01 | 0.06± 0.02 |
| Llama2-7B | 0.02 ± 0.01 | 0.42 ± 0.01 | 0.87 ± 0.01 | 0.31 ± 0.01 | 0.42 ± 0.01 | 0.56 ± 0.01 | 0.56± 0.04 | 0.03 ± 0.02 | 0.02± 0.00 |
| Mixtral-7B | 0.01 ± 0.02 | 0.41 ± 0.01 | 0.85 ± 0.02 | 0.37 ± 0.01 | 0.41 ± 0.02 | 0.57 ± 0.01 | 0.58± 0.03 | 0.00± 0.00 | 0.02± 0.02 |
| Qwen2.5-7B | 0.07 ± 0.04 | 0.41 ± 0.02 | 0.82 ± 0.02 | 0.34 ± 0.03 | 0.38 ± 0.02 | 0.43 ± 0.03 | 0.43± 0.02 | 0.06± 0.01 | 0.04± 0.02 |
| Llama-3.1-8B | 0.01 ± 0.02 | 0.41 ± 0.02 | 0.85 ± 0.02 | 0.41 ± 0.01 | 0.39 ± 0.01 | 0.57 ± 0.02 | 0.58± 0.00 | 0.02± 0.02 | 0.00± 0.01 |
| Llama2-13B | 0.01 ± 0.03 | 0.41 ± 0.01 | 0.86 ± 0.01 | 0.31 ± 0.02 | 0.40 ± 0.02 | 0.58 ± 0.02 | 0.60± 0.01 | 0.02 ± 0.01 | 0.02± 0.03 |
| Average | 0.029 | 0.418 | 0.854 | 0.371 | 0.412 | 0.553 | 0.505 | 0.031 | 0.032 |
| Water-Probe-v2 (w. prompt 4.3) |  |  |  |  |  |  |  |  |  |
| Qwen2.5-1.5B | 0.02± 0.02 | 0.30± 0.01 | 0.83± 0.01 | 0.29± 0.01 | 0.27± 0.02 | 0.49± 0.02 | 0.52± 0.03 | 0.39± 0.03 | 0.60 ± 0.00 |
| OPT-2.7B | 0.04 ± 0.03 | 0.29 ± 0.02 | 0.88 ± 0.01 | 0.23 ± 0.01 | 0.19 ± 0.02 | 0.42 ± 0.01 | 0.43± 0.03 | 0.43 ± 0.01 | 0.62 ± 0.00 |
| Llama-3.2-3B | 0.00 ± 0.01 | 0.31 ± 0.01 | 0.89 ± 0.01 | 0.33 ± 0.00 | 0.24 ± 0.01 | 0.51 ± 0.01 | 0.54± 0.01 | 0.52 ± 0.01 | 0.84 ± 0.00 |
| Qwen2.5-3B | 0.03± 0.02 | 0.35± 0.04 | 0.78± 0.01 | 0.29± 0.02 | 0.28± 0.01 | 0.45± 0.02 | 0.45± 0.02 | 0.39± 0.02 | 0.71 ± 0.00 |
| Llama2-7B | 0.04 ± 0.02 | 0.34 ± 0.01 | 0.82 ± 0.02 | 0.33 ± 0.01 | 0.28 ± 0.01 | 0.50 ± 0.01 | 0.51± 0.02 | 0.48 ± 0.01 | 0.81 ± 0.00 |
| Mixtral-7B | 0.09 ± 0.01 | 0.34 ± 0.04 | 0.83 ± 0.01 | 0.29 ± 0.02 | 0.24 ± 0.01 | 0.51 ± 0.01 | 0.53± 0.00 | 0.42± 0.02 | 0.81 ± 0.00 |
| Qwen2.5-7B | -0.01 ± 0.04 | 0.26 ± 0.02 | 0.70 ± 0.00 | 0.28 ± 0.02 | 0.23 ± 0.01 | 0.32 ± 0.03 | 0.35± 0.02 | 0.32± 0.02 | 0.73 ± 0.00 |
| Llama-3.1-8B | 0.01 ± 0.00 | 0.31 ± 0.01 | 0.77 ± 0.01 | 0.29 ± 0.02 | 0.26 ± 0.00 | 0.50 ± 0.01 | 0.51 ± 0.01 | 0.43± 0.01 | 0.71 ± 0.00 |
| Llama2-13B | 0.01 ± 0.02 | 0.35 ± 0.01 | 0.82 ± 0.02 | 0.26 ± 0.02 | 0.26 ± 0.01 | 0.50 ± 0.01 | 0.53± 0.01 | 0.44 ± 0.02 | 0.73 ± 0.00 |
| Average | 0.026 | 0.317 | 0.813 | 0.288 | 0.250 | 0.467 | 0.486 | 0.424 | 0.729 |

Tested Watermarking Algorithms: We evaluated a diverse range of LLM watermarking algorithms, including N-Gram based watermarking and Fixed-Key-List based watermarking. For N-Gram based watermarking, we tested KGW (Kirchenbauer et al., ) ($\gamma=0.5$, $\delta=2$), Aar (Aaronson & Kirchner, ) ($N=1$), KGW-Min Kirchenbauer et al. () (window size of 4), KGW-Skip (Kirchenbauer et al., ) (window of 3), DiPMark (Wu et al., ) ($\alpha=0.45$), and $\gamma$ reweighting (Hu et al., ). For Fixed-Key-List based watermarking, we examined EXP-Edit (Kuditipudi et al., ) and ITS-Edit (Kuditipudi et al., ), both with a key length of 420. Details of these algorithms are provided in Appendix F.

Tested LLMs:
To comprehensively evaluate our algorithm’s effectiveness, we tested a diverse range of LLMs with varying parameter sizes, including Qwen2.5-1.5B (Hui et al., ), OPT-2.7B (Zhang et al., ), Llama3.2-3B (Meta AI, ), Qwen2.5-3B, Llama2-7B (Touvron et al., ), Mixtral-7B (Jiang et al., ), Qwen2.5-7B, Llama-3.1-8B (Dubey et al., ), and Llama2-13B (Touvron et al., ).We evaluated our Water-Probe algorithm on all LLMs, testing its performance under various watermarking schemes and in scenarios without watermarks.

Watermark-Probe Settings: For our Watermark-Probe algorithm, detailed prompts are provided in the appendix C. To calculate the z-score, we repeat each detection experiment 3 times to compute the standard deviation. We set $\mu=0.1$ for our experiments.

#### 5.2 Main Results

In Table 1, we present the average similarity and standard deviation obtained using Watermark-Probe-v1 and Watermark-Probe-v2 algorithms for identifying various LLMs under different watermarking conditions and non-watermarked scenarios. For all LLMs, the sampling temperature was set to 1, with the number of samples set to $10^{4}$. As evident from Table 1, Watermark-Probe-v1 demonstrates high effectiveness for N-gram based watermarking but is not applicable to fixed-key-list based watermarking. In contrast, the Watermark-Probe-v2 algorithm proves effective in identifying all watermarking algorithms tested.
Additionally, even watermarking algorithms claiming to be distortion-free, such as Aar (Aaronson & Kirchner, ), DiPMark (Wu et al., ), and $\gamma$-reweighting (Hu et al., ), they can be effectively identified by both versions of Watermark-Probe. Furthermore, our algorithm maintains low similarity for non-watermarked LLMs, ensuring minimal false positive rates.
Additionally, we calculated the average similarity for different watermarking algorithms in Table 1 to demonstrate their detection confidence. Among these, the Aar algorithm is the most easily detectable due to its pronounced perturbation for individual keys.
Lastly, given the same number of samples, the Watermark-Probe-v1 algorithm yields more significant identification results for N-gram based watermarking algorithms compared to the Watermark-Probe-v2 algorithm.

Figure: Figure 3: The left plot shows the variation of z-scores detected by Watermark-Probe-v1 and Watermark-Probe-v2 as a function of sampling temperature. The right plot illustrates the change in z-scores detected by Watermark-Probe-v1 and Watermark-Probe-v2 with different sampling numbers.
Refer to caption: x3.png

#### 5.3 Further Analysis

Influence of Sampling Temperature: The results in Table 1 are based on a sampling temperature of 1. To further validate the performance of Watermark-Probe under different sampling temperatures, we show the z-score changes across temperatures in the left plot of Figure 3. Using Llama2-7B as an example, as the sampling temperature increases from 0.1 to 1.5, both Watermark-Probe-v1 and Watermark-Probe-v2 can distinguish between watermarked and unwatermarked LLMs. However, at relatively low temperatures ($T<0.5$), detection of unwatermarked LLMs may show some fluctuations. Since deployed LLMs rarely use very low temperatures, our algorithm can be considered effective for detecting real-world LLM deployments.

Influence of Sampling Number: The right plot in Figure 3 illustrates the impact of the sampling number on the detected z-score. Specifically, we used Llama2-7B as the target LLM with a temperature of 1. We observed that Watermark-Probe-v2 requires more samples compared to Watermark-Probe-v1. With insufficient samples, Watermark-Probe-v2 lacks enough common prefixes to compute Equation 9. In our setting, Watermark-Probe-v1 can achieve stable detection with 1,000 samples, while Watermark-Probe-v2 requires at least $10^{4}$ samples. For cases where detection is successful, the z-score of watermarked LLMs tends to increase with the number of samples, although this trend exhibits fluctuations.

### 6 Enhancing the Imperceptibility of Watermarked LLMs

We have demonstrated that current watermarked LLMs can be identified by our Water-Probe method. In this section, we discuss how to improve the imperceptibility of watermarked LLMs. The core principle is to make it challenging to construct repeated sampling scenarios using two separate keys according to Equation 9.

One design is globally fixed watermark key (e.g., Unigram (Zhao et al., )). While it’s easy to construct repeated sampling scenarios with a single key, we cannot detect stable deviations between different keys as only one exists globally. In the appendix E, we provide an algorithm named Water-Contrast to identify watermarks by comparing the target LLM distribution and a prior distribution. While not theoretically guaranteed (it’s challenging to determine if this bias is from watermarking or inherent to the LLM), it shows practical effectiveness.
Meanwhile, Unigram watermarks are susceptible to cracking (Jovanović et al., ).

The second design aims to increase the randomness of watermark key selection, making it less dependent on N-grams. This approach makes it difficult to construct repeated sampling scenarios using the same key. For Fixed-Key-List Based watermarking, a viable strategy is to increase the length of the key list. Since the initial key position is random, increasing the key list length enhances the randomness of key selection.

Figure: Figure 4: Distribution of start keys for identical prefixes in Water-Bag strategy. Analysis based on prompts described in Section 4.3 for Water-Probe-v2. Each subplot represents a specific prefix (showed in title).
Refer to caption: x5.png

Additionally, for N-gram Based watermarking algorithms, we propose an enhanced strategy called Water-Bag, which combines multiple master watermark keys into a key-bag with a key inversion mechanism.

###### Definition 7 (Water-Bag Strategy) .

The Water-Bag strategy extends N-gram based watermarking by using a set of master keys $K=\{K_{1},K_{2},...,K_{n}\}$ and their inversions $\overline{K}=\{\overline{K_{1}},\overline{K_{2}},...,\overline{K_{n}}\}$. For each generation, a master key $K_{j}$ or its inversion $\overline{K_{j}}$ is randomly selected:

$$ $P_{M}^{WB}(y_{i}|x,y_{1:i-1},K,\overline{K})=F(P_{M}(y_{i}|x,y_{1:i-1}),k_{i}) ,\quad k_{i}=f(K_{j}^{*},y_{i-n:i-1}),\quad K_{j}^{*}\sim\text{Uniform}(K\cup \overline{K})$ (11) $$

where $P_{M}^{WB}$ is the modified probability distribution, $K_{j}^{*}$ is randomly sampled from the combined set of original and inverted keys, and $f$ is the watermark key derivation function. The inverted key $\overline{K_{j}}$ is defined as:

$$ $\frac{1}{2}(F(P_{M}(y_{i}|x,y_{1:i-1}),f(K_{j},y_{i-n:i-1}))+F(P_{M}(y_{i}|x,y _{1:i-1}),f(\overline{K_{j}},y_{i-n:i-1})))=P_{M}(y_{i}|x,y_{1:i-1})$ (12) $$

This ensures that the average effect of $K_{j}$ and $\overline{K_{j}}$ on the logits is equivalent to the original logits, which makes our Water-Probe-v1 ineffective against the Water-Bag strategy.

For the watermarked text detection for Water-Bag Strategy, we use the maximum detection score across all master keys in the bag. The text is considered watermarked if this maximum exceeds a threshold.

**Table 2: Performance comparison of Water-Bag Strategy and Exp-Edit algorithm in watermarked LLM identification and watermarked text detection. Water-Bag is evaluated with varying bag sizes, while Exp-Edit is tested with different key lengths. represents high-confidence watermark and represents low-confidence watermark. Detail of z-score could be seen in Appendix H.**
|  |  | KGW w. Water-Bag | Exp-Edit(Key-len) |  |  |  |  |  |
| --- | --- | --- | --- | --- | --- | --- | --- | --- |
|  | None | $|K\cup\overline{K}|=1$ | $|K\cup\overline{K}|=2$ | $|K\cup\overline{K}|=4$ | $|K\cup\overline{K}|=8$ | $|K|=420$ | $|K|=1024$ | $|K|=2048$ |
| Watermarked LLM Indentification |  |  |  |  |  |  |  |  |
| Water-Probe-v1(n=3) | 0.02 ± 0.01 | 0.42 ± 0.01 | 0.05 ± 0.01 | 0.02 ± 0.01 | 0.03 ± 0.02 | 0.03 ± 0.05 | 0.02 ± 0.01 | 0.02 ± 0.02 |
| Water-Probe-v2(n=3) | 0.04 ± 0.01 | 0.34 ± 0.01 | 0.34 ± 0.01 | 0.25 ± 0.01 | 0.16 ± 0.02 | 0.48 ± 0.01 | 0.33±0.01 | 0.23±0.02 |
| Water-Probe-v2(n=5) | 0.06 ± 0.06 | 0.32 ± 0.01 | 0.18 ± 0.01 | 0.12 ± 0.02 | 0.07 ± 0.01 | 0.64 ± 0.00 | 0.54 ± 0.01 | 0.44 ± 0.00 |
| Watermarked Text Detection |  |  |  |  |  |  |  |  |
| Detection-F1-score | - | 1.0 | 1.0 | 1.0 | 1.0 | 1.0 | 0.975 | 1.0 |
| PPL | 8.15 | 11.93 | 11.85 | 12.17 | 12.50 | 16.63 | 17.28 | 19.06 |
| Robustness (GPT3.5) | - | 0.843 | 0.849 | 0.748 | 0.696 | 0.848 | 0.854 | 0.745 |
| Detection-time (s) | - | 0.045 | 0.078 | 0.156 | 0.31 | 37.87 | 108.5 | 194.21 |

To validate the effectiveness of the two strategies for enhancing the imperceptibility of watermarked LLMs, we evaluated their performance in Table 2. We examined both watermarked LLM identification and watermarked text detection settings, assessing the new watermarking strategies’ detectability by our Water-Probe algorithm and their impact on watermarked text detection efficacy and performance.

For watermarked text detection, we used OPT-2.7B to generate texts on the C4 dataset (Raffel et al., ), using 30 tokens as prompts and generating 200 additional tokens with watermarks. PPL was calculated using Llama2-7B. To assess detection robustness, we computed the F1-score after rewriting texts using GPT-3.5. Detection time for single text was also recorded. We use the KGW algorithm as an implementation example for the water-bag strategy.
For watermarked LLM identification, we report results for both Water-Probe-v1 and Water-Probe-v2. For Water-Probe-v2, we present results for $n=3$ and $n=5$, where $n$ is the number of random tokens generated as described in Section 4.3. The $n=3$ setting matches Table 1, while prompts for $n=5$ are provided in Appendix C.

Table 2 demonstrates that Water-Probe-v1 fails to effectively identify both Water-Bag and Exp-Edit algorithms. For Water-Probe-v2, detection difficulty increases with larger bag sizes in Water-Bag and longer key lengths in Exp-Edit. However, Water-Bag proves more challenging to identify. Crucially, Water-Bag’s detectability remains stable as $n$ increases, while Exp-Edit becomes more easy to identify.
We analyzed the key distribution of Water-Bag under Water-Probe-v2 for different prefixes in Figure 4. The distribution is notably more uniform compared to Exp-Edit in Figure 2, explaining why Water-Bag is relatively harder to identify.
We observe that increasing watermark key randomness reduces watermark robustness for both strategies, with Exp-Edit also significantly increasing detection time. This highlights a trade-off in watermarking algorithms between key randomness and robustness (Liu et al., ). Future work should focus on developing algorithms that enhance randomness without compromising robustness.

###### Definition 7 (Water-Bag Strategy) .

### 7 Conclusion

In this paper, we pioneered the study of identifying watermarked LLMs. We first theoretically demonstrated the basis for identifying watermarked LLMs. We then designed the Water-Probe algorithm, which identifies watermarked LLMs by comparing distribution differences of similar prompts under different watermark keys. Our experiments showed that our algorithm is applicable to all N-Gram and Fixed-Key-List based Watermarking algorithms, independent of sampling temperature. We discussed scenarios where Water-Probe might fail and designed the WaterBag watermarking algorithm, which sacrifices some robustness of watermarked text detection to make watermarked LLMs harder to identify. Future work could focus on watermark concealment as a key research direction, designing more covert watermarking schemes.

## Part I Appendix

### Appendix A Proof of Theorem 1 and Additional Analyses

###### Proof.

Goal: Prove that under the Lipschitz continuity condition, the watermark modification differences for two similar prompts $p_{1}$ and $p_{2}$ have high similarity, with an expected similarity of at least $\rho$.

Step 1: Express the watermark modification difference

For a prompt $p$ and keys $k_{1},k_{2}$, define the watermark modification difference as:

$$ $\Delta_{p}(k_{1},k_{2})=P_{M}^{F}(\cdot|p,k_{1})-P_{M}^{F}(\cdot|p,k_{2})$ (13) $$

Step 2: Utilize the Lipschitz continuity condition

By the Lipschitz continuity condition, for any fixed key $k$:

$$ $\|P_{M}^{F}(\cdot|p_{1},k)-P_{M}^{F}(\cdot|p_{2},k)\|_{1}\leq L\cdot\|P_{M}( \cdot|p_{1})-P_{M}(\cdot|p_{2})\|_{1}$ (14) $$

Since $p_{1}$ and $p_{2}$ satisfy the similarity condition $\text{KL}(P_{M}(\cdot|p_{1})||P_{M}(\cdot|p_{2}))\leq\epsilon$ (Equation 5), by Pinsker’s inequality (Csiszár & Körner, ):

$$ $\|P_{M}(\cdot|p_{1})-P_{M}(\cdot|p_{2})\|_{1}\leq\sqrt{2\epsilon}$ (15) $$

Therefore, for any $k\in\{k_{1},k_{2}\}$:

$$ $\|P_{M}^{F}(\cdot|p_{1},k)-P_{M}^{F}(\cdot|p_{2},k)\|_{1}\leq L\cdot\sqrt{2\epsilon}$ (16) $$

Step 3: Analyze the difference of watermark modification differences

Consider the definition of $\Delta_{p}(k_{1},k_{2})$ and compute $\|\Delta_{p_{1}}(k_{1},k_{2})-\Delta_{p_{2}}(k_{1},k_{2})\|_{1}$:

$$ $\displaystyle\|\Delta_{p_{1}}(k_{1},k_{2})-\Delta_{p_{2}}(k_{1},k_{2})\|_{1}$ $\displaystyle=\|(P_{M}^{F}(\cdot|p_{1},k_{1})-P_{M}^{F}(\cdot|p_{1},k_{2}))-(P _{M}^{F}(\cdot|p_{2},k_{1})-P_{M}^{F}(\cdot|p_{2},k_{2}))\|_{1}$ $\displaystyle=\|(P_{M}^{F}(\cdot|p_{1},k_{1})-P_{M}^{F}(\cdot|p_{2},k_{1}))-(P _{M}^{F}(\cdot|p_{1},k_{2})-P_{M}^{F}(\cdot|p_{2},k_{2}))\|_{1}$ $\displaystyle\leq\|P_{M}^{F}(\cdot|p_{1},k_{1})-P_{M}^{F}(\cdot|p_{2},k_{1})\| _{1}+\|P_{M}^{F}(\cdot|p_{1},k_{2})-P_{M}^{F}(\cdot|p_{2},k_{2})\|_{1}$ $\displaystyle\leq L\cdot\sqrt{2\epsilon}+L\cdot\sqrt{2\epsilon}$ $\displaystyle=2L\cdot\sqrt{2\epsilon}$ $\displaystyle=\delta^{\prime}$ (17) $$

where $\delta^{\prime}=2L\cdot\sqrt{2\epsilon}$.

Step 4: Relate $L_{1}$ distance to similarity measure

Assume $\text{Sim}(\cdot,\cdot)$ is cosine similarity, which is negatively correlated with $L_{1}$ distance. Since $\|\Delta_{p_{1}}(k_{1},k_{2})-\Delta_{p_{2}}(k_{1},k_{2})\|_{1}\leq\delta^{\prime}$, and $\delta^{\prime}$ is a small positive number:

$$ $\text{Sim}(\Delta_{p_{1}}(k_{1},k_{2}),\Delta_{p_{2}}(k_{1},k_{2}))\geq\rho^{\prime}$ (18) $$

where $\rho^{\prime}$ is a lower bound dependent on $\delta^{\prime}$, approaching 1 as $\delta^{\prime}$ decreases.

Step 5: Calculate expected similarity

Since $\text{Sim}(\Delta_{p_{1}}(k_{1},k_{2}),\Delta_{p_{2}}(k_{1},k_{2}))\geq\rho^{\prime}$ for any $k_{1},k_{2}\in\mathcal{K}$, for randomly sampled $k_{1},k_{2}$:

$$ $\mathbb{E}_{k_{1},k_{2}\sim\mathcal{K}}\left[\text{Sim}(\Delta_{p_{1}}(k_{1},k _{2}),\Delta_{p_{2}}(k_{1},k_{2}))\right]\geq\rho^{\prime}$ (19) $$

Set $\rho=\rho^{\prime}$, and by choosing a sufficiently small $\epsilon$ (making $\delta^{\prime}$ small enough), we can ensure $\rho$ is close to 1.

Conclusion: Under the Lipschitz continuity condition, for two similar prompts $p_{1}$ and $p_{2}$, the expected similarity of their watermark modification differences under randomly sampled watermark keys $k_{1}$ and $k_{2}$ is at least $\rho$, where $\rho$ is a large positive number close to 1. This proves the high consistency of watermark effects in similar contexts, ensuring the detectability and robustness of the watermark.
∎

###### Proof.

### Appendix B Statistical Analysis of Unwatermarked LLMs Identification

When there’s no watermark present in the LLM, we expect the similarity measure in Equation 9 to be close to 0. This can be mathematically explained as follows:

For an unwatermarked model, $P_{M}^{F}=P_{M}$ for any $k$. Therefore:

$$ $\displaystyle\Delta_{R}(P_{i},k_{m},k_{n})$ $\displaystyle=R(\hat{P}_{M}^{F}(\cdot|P_{i},k_{m}))-R(\hat{P}_{M}^{F}(\cdot|P_ {i},k_{n}))$ (20) $\displaystyle=R(\hat{P}_{M}(\cdot|P_{i}))-R(\hat{P}_{M}(\cdot|P_{i}))$ $\displaystyle=0$ $$

Consequently, $\text{Sim}(\Delta_{R}(P_{i},k_{m},k_{n}),\Delta_{R}(P_{j},k_{m},k_{n}))\approx
\text{Sim}(0,0)=0$ for any $P_{i},P_{j},k_{m},k_{n}$. This results in the average similarity $\bar{S}$ in Equation 9 being approximately 0.

### Appendix C Detailed Prompts for Simulating Watermark Keys

In this section, we provide details of the prompt used for repeated sampling with the same key, as introduced in Section 4.3.

For the Water-Probe-v1 algorithm, we use the following prompt pair as shown in Prompt C and Prompt C.

- Prompt3: First prompt for Fixed Key List Based Watermarking(Water-Probe-v1)
Please generate some text based on the following instructions(no other words):
First generate the prefix D seven tiger.
Then answer the question: Name a country with a large population.
Example1: D seven tiger China
Example2: D seven tiger India
Example3:

- Prompt4: First prompt for Fixed Key List Based Watermarking(Water-Probe-v1)
Please generate some text based on the following instructions(no other words):
First generate the prefix D seven tiger.
Then answer the question: Name a country with a large area.
Example1: D seven tiger China
Example2: D seven tiger India
Example3:

Here, we use a fixed prefix D seven tiger as an example. In the actual experiment, we used 50 different prefixes to simulate 50 potentially different watermark keys. For the experiment in Table 1, we performed 10,000 samplings, with each prefix sampled 200 times. The complete list of 50 prefixes is shown below.

- Prompt5: First prompt for Fixed Key List Based Watermarking(Water-Probe-v2)
Please generate some text based on the following instructions(no other words):
The first word is randomly sampled from A-Z.
The second word is randomly sampled from zero to nine.
The third word is randomly sampled from cat, dog, tiger and lion.
Then add a separator | and answer the following question: Name a country with a large population.
Example1: D seven tiger | United States
Example2: J five dog | India
Example3: R six cat | China
Example4: T one tiger | Indonesia
Example5: P seven cat | Pakistan
Example6: G six cat | Russia
Example7: R five tiger | India
Example8: L nine cat | Mexico
Example9: T four tiger | United States
Example10: H three dog | Japan
Example11: B three lion | Germany
Example12:

- Prompt6: Second prompt for Fixed Key List Based Watermarking(Water-Probe-v2)
Please generate some text based on the following instructions(no other words):
The first word is randomly sampled from A-Z.
The second word is randomly sampled from zero to nine.
The third word is randomly sampled from cat, dog, tiger and lion.
Then add a separator | and answer the following question: Name a country with a large area.
Example1: L three tiger | United States
Example2: X three cat | India
Example3: A six tiger | China
Example4: W eight lion | Argentina
Example5: D five dog | France
Example6: P one cat | Russia
Example7: E six tiger | Australia
Example8: Z eight lion | Canada
Example9: Q two tiger | United States
Example10: A nine cat | Brazil
Example11: V three dog | Russia
Example12:

Similarly, for Prompt C and Prompt C, we assume that the answer spaces for the questions Name a country with a large population. and Name a country with a large area are highly correlated, i.e., countries with large areas tend to have relatively large populations.
In practice, other correlated prompt pairs can be chosen, as long as they satisfy the correlation requirement.

For the Water-Probe-v2 algorithm, we use the following prompt pair shown in Prompt C and Prompt C.

- Prompt7: First prompt for Fixed Key List Based Watermarking(Water-Probe-v2 N=5)
Please generate some text based on the following instructions(no other words):
The first word is randomly sampled from A-Z.
The second word is randomly sampled from zero to nine.
The third word is randomly sampled from cat, dog, tiger and lion.
The fourth word is randomly sampled from apple, banana and orange.
The fifth word is randomly sampled from car, bus and truck.
The sixth entry is the answer to the following question: Name a country with a large population.
Example1: D seven tiger apple car United States
Example2: J five dog banana bus India
Example3: R six cat orange truck China
Example4: T one tiger apple bus Indonesia
Example5: P seven cat orange car Pakistan
Example6: G six cat banana truck Russia
Example7: R five tiger apple bus India
Example8: L nine cat banana car Mexico
Example9: T four tiger orange truck United States
Example10: H three dog apple bus Japan
Example11: B three lion orange truck Germany
Example12:

- Prompt8: Second prompt for Fixed Key List Based Watermarking(Water-Probe-v2 N=5)
Please generate some text based on the following instructions(no other words):
The first word is randomly sampled from A-Z.
The second word is randomly sampled from zero to nine.
The third word is randomly sampled from cat, dog, tiger and lion.
The fourth word is randomly sampled from apple, banana and orange.
The fifth word is randomly sampled from car, bus and truck.
The sixth entry is the answer to the following question: Name a country with a large population.
Example1: D seven tiger apple car United States
Example2: J five dog banana bus India
Example3: R six cat orange truck China
Example4: T one tiger apple bus Argentina
Example5: P seven cat orange car France
Example6: G six cat banana truck Russia
Example7: R five tiger apple bus Australia
Example8: L nine cat banana car Canada
Example9: T four tiger orange truck United States
Example10: H three dog apple bus Brazil
Example11: B three lion orange truck Russia
Example12:

In this case, we selected the same question pair as in Water-Probe-v1. However, for Water-Probe-v2, all prefixes are randomly sampled by the LLM during the generation process. This random sampling by the LLM itself makes it easier to model the correlations between watermark keys at different positions.

Prompt C and Prompt C both generate prefixes of length 3. Since we further analyzed the case of generating prefixes of length 5 in Section 6, we also provide prompts for prefixes of length 5 in Prompt C and Prompt C. It is worth noting that the longer the required prefix, the more total sampling times are needed, as more samples are required to cover all actually occurring prefixes. In this work, for the case where the prefix length is 3, we typically sampled 10,000 times, and for the case where the prefix length is 5, we typically sampled 100,000 times.

### Appendix D Detailed Algorithm for Water-Probe

To provide a clearer presentation of our Water-Probe algorithm, we present here a complete algorithmic representation, corresponding to the algorithm pipeline process described in Section 4.2. This algorithm flow can be used for both Water-Probe-v1 and Water-Probe-v2, although the prompt construction process differs between them, as detailed in Appendix C.

Figure: Algorithm 1 Water-Probe Algorithm

### Appendix E Detection of Global-Fixed Key Watermarking in LLMs

In this section, we explore how to identify LLMs with global-fixed key based watermarking. As discussed in Section 6, since global-fixed key based watermarking uses only one key globally, we cannot compare differences between two different watermark keys. However, as the global-fixed key produces consistent bias across all prompts, we can calculate a prior distribution for each prompt and then verify if the differences from this prior distribution have high similarity across all prompt lists satisfying Equation 5.

Specifically, let $P_{1},P_{2},...,P_{N}$ be the constructed prompt list, and $P_{\text{prior}}$ be their prior distribution. We can modify the average similarity calculation in Equation 9 as follows:

**Table 3: Identification of global fixed key watermarking (e.g., Unigram watermarking) in LLMs using a prior distribution computed as the average output distribution of all eight LLMs. indicates high-confidence watermark identification and indicates low-confidence watermark identification while no color indicates no watermark identified.**
| LLM | Similarity | Z-Score |  |  |  |
| --- | --- | --- | --- | --- | --- |
|  | Unigram | Unwatermark | Unigram | Unwatermark |  |
| GPT2 | 0.59 ± 0.018 | 0.06 ± 0.37 | 32.49 | 1.61 |  |
| OPT1.3B | 0.6 ± 0.02 | 0.06 ± 0.029 | 28.77 | 2.07 |  |
| OPT2.7B | 0.65 ± 0.009 | 0.03 ± 0.048 | 74.81 | 0.63 |  |
| LLama2-7B | 0.48 ± 0.042 | 0.04 ± 0.043 | 11.31 | 0.92 |  |
| LLama-2-13B | 0.38 ± 0.037 | 0.08 ± 0.018 | 10.17 | 4.54 |  |
| LLama-3.1-8B | 0.68 ± 0.06 | 0.12 ± 0.05 | 12.09 | 2.42 |  |
| Mixtral-7B | 0.87 ± 0.056 | 0.038 ± 0.052 | 22.24 | 0.74 |  |
| Qwen2.5-7B | 0.53 ± 0.058 | 0.09 ± 0.028 | 8.96 | 3.29 |  |

**Table 4: Identification of global fixed key watermarking in LLMs using a single proxy model’s output as prior distribution. indicates high-confidence watermark identification and indicates low-confidence watermark identification while no color indicates no watermark identified.**
| LLM | GPT2 | OPT1.3B | OPT2.7B | LLama2-7B |  |  |  |  |  |  |  |  |  |  |  |  |
| --- | --- | --- | --- | --- | --- | --- | --- | --- | --- | --- | --- | --- | --- | --- | --- | --- |
| Similarity | Z-Score | Similarity | Z-Score | Similarity | Z-Score | Similarity | Z-Score |  |  |  |  |  |  |  |  |  |
| W | NW | W | NW | W | NW | W | NW | W | NW | W | NW | W | NW | W | NW |  |
| GPT2 | 0.48±0.009 | 0.005±0.025 | 49.68 | 0.21 | 0.58±0.004 | 0.07±0.02 | 138.24 | 3.46 | 0.65±0.008 | -0.07±0.015 | 77.60 | -0.49 | 0.42±0.017 | 0.034±0.015 | 25.37 | 2.25 |
| OPT1.3B | 0.71±0.07 | 0.09±0.04 | 9.433 | 2.31 | 0.59±0.007 | 0.04 ±0.02 | 81.44 | 1.84 | 0.67±0.007 | 0.05±0.015 | 92.46 | 3.43 | 0.45±0.008 | 0.11±0.04 | 53.21 | 2.55 |
| OPT2.7B | 0.7±0.03 | 0.1±0.05 | 23.89 | 1.70 | 0.59±0.01 | 0.07±0.03 | 53.05 | 2.41 | 0.64±0.007 | 0.025±0.025 | 88.79 | 0.99 | 0.46±0.04 | 0.03±0.01 | 11.16 | 2.47 |
| LLama2-7B | 0.66±0.015 | 0.1±0.007 | 44.03 | 14.08 | 0.57±0.025 | 0.07±0.02 | 22.38 | 3.32 | 0.67±0.008 | 0.028±0.023 | 79.72 | 1.21 | 0.43±0.02 | 0.06±0.02 | 18.35 | 2.60 |

- Prompt9: Used Prompt Set for Global-Fixed key Based Watermarking
1
: Please generate random number sequence between 0 to 9:2,3,9,8,0,4,7,5,6,1,5,8,7,1,2,4,6,0,9,3,2
: Please generate random number sequence between 0 to 9:3,8,0,7,4,5,6,1,9,2,1,3,7,9,2,0,4,6,8,5,3
: Please generate random number sequence between 0 to 9:5,8,7,1,2,4,6,0,9,3,2,3,9,8,0,4,7,5,6,1,4
: Please generate random number sequence between 0 to 9:0,7,2,3,6,5,1,9,8,4,3,8,0,7,4,5,6,1,9,2,5
: Please generate random number sequence between 0 to 9:1,3,7,9,2,0,4,6,8,5,0,7,2,3,6,5,1,9,8,4,6
: Please generate random number sequence between 0 to 9:6,0,9,3,2,3,9,8,0,4,7,5,6,1,5,8,7,1,2,4,7
: Please generate random number sequence between 0 to 9:4,6,0,9,3,2,3,9,8,0,4,7,5,6,1,5,8,7,1,2,8
: Please generate random number sequence between 0 to 9:9,8,0,4,7,5,6,1,5,8,7,1,2,4,6,0,9,3,1,3,9
: Please generate random number sequence between 0 to 9:7,4,5,6,1,9,2,1,3,7,9,2,0,4,6,8,5,0,7,2,10
: Please generate random number sequence between 0 to 9:8,0,7,4,5,6,1,9,2,5,8,7,1,2,4,6,0,9,3,2,

$$ $\bar{S}=\frac{1}{|\mathcal{P}|(|\mathcal{P}|-1)}\sum_{P_{i}\neq P_{j}\in \mathcal{P}}\text{Sim}(R(P_{M}(\cdot|P_{i})-P_{\text{prior}}(\cdot|P_{i})),R(P _{M}(\cdot|P_{j})-P_{\text{prior}}(\cdot|P_{j}))).$ (21) $$

For the prior distribution, we select N proxy LLMs and calculate the average output probability distribution under prompt $p_{i}$ as the prior distribution. Specifically:

$$ $P_{\text{prior}}(\cdot|p_{i})=\frac{1}{N}\sum_{j=1}^{N}P_{M_{j}}(\cdot|p_{i}),$ (22) $$

where $M_{j}$ represents the $j$-th proxy LLM, and $N$ is the total number of proxy LLMs used. Specifically, we used the following proxy LLMs: GPT2, OPT1.3B, OPT2.7B, Llama2-7B, Llama2-13B, Llama3-8.1B, Mixtral-7B, and Qwen2.5-7B.

Additionally, we utilized 10 distinct prompts for computation to validate our method as shown in Prompt E.

**Table 5: Details of watermarking algorithms tested in our work.**
| Algorithm Name | Category | Methodology |
| --- | --- | --- |
| KGW (Kirchenbauer et al., ) | N-Gram | Separate the vocabulary set into two lists: a red list and a green list based on the preceding token, then add bias to the logits of green tokens so that the watermarked text exhibits preference of using green tokens. |
| KGW-Min (Kirchenbauer et al., ) | N-Gram | Similar to KGW, this approach partitions the vocabulary set based on the minimum token ID within a window of N-gram preceding tokens. |
| KGW-Skip (Kirchenbauer et al., ) | N-Gram | Similar to KGW, this approach partitions the vocabulary set based on the left-most token ID within a window of N-gram preceding tokens. |
| Aar (Aaronson & Kirchner, ) | N-Gram | Generate a pseudo-random vector $r_{t}$ based on the N-gram preceding tokens to guide sampling at position $t$, and choose the token $i$ that maximize $r_{t}(i)^{1/p_{t}(i)}$ (exponential minimum sampling), where $p_{t}$ is the probability produced by LLM. |
| $\gamma$-Reweight (Hu et al., ) | N-Gram | Randomly shuffle the probability vector using a seed based on the preceding N-gram tokens. Discard the left half of the vector, doubling the remaining probabilities. Conduct further sampling using this reweighted distribution. |
| DiPmark (Wu et al., ) | N-Gram | Similar to $\gamma$-Reweight, after shuffling, discard the left $\alpha$ portion of the vector and amplify the remaining probabilities by $1/(1-\alpha)$. |
| EXP-Edit (Kuditipudi et al., ) | Fixed-Key-List | Based on the Aar concept, construct a fixed pseudo-random vector list. When generating watermarked text, randomly select a start index in the list. For each watermarked token generation, sequentially use the pseudo-random vectors from this index for exponential minimum sampling. During detection, employ edit distance to calculate the correlation between the pseudo-random vector list and the text. |
| ITS-Edit (Kuditipudi et al., ) | Fixed-Key-List | Similar to EXP-Edit, this method also uses a fixed pseudo-random vector list. However, it uses inverse transform sampling instead of exponential minimum sampling during token selection. |

The intuition behind this method is that if $M$ is watermarked, the differences between its output distributions and those of the proxy model should exhibit consistent patterns across different prompts, resulting in a high correlation. Conversely, for an unwatermarked model, we assume that the differences in bias between different language models are relatively small, and the correlation of these differences should be low.

Table 3 presents the identification results for global-fixed key watermarking using a prior distribution. Our method effectively identifies models employing global-fixed key watermarking, while yielding low z-scores for unwatermarked LLMs.

To further validate the key factors in using prior distribution for testing, we conducted experiments using a single proxy model as the prior distribution, as shown in Table 4. We performed cross-experiments with different LLMs.
These results demonstrate that global fixed-key watermarking can still be detected when using a single LLM as the prior distribution. However, the z-score detection for unwatermarked LLMs exhibits greater fluctuation.
This is primarily due to the significant bias in a single proxy model as the prior distribution, affecting the variance of identification and resulting in small z-score.

Although our method using prior distribution achieved good results in our experiments, this identification approach has a limitation in that it can only detect stable biases in LLMs, assuming that a stable bias indicates a watermark. However, this assumption may not hold in real-world scenarios, as it is challenging to distinguish whether the bias is caused by watermarking or inherent to the LLM itself. This is particularly problematic in cases where LLMs are known to have inherent biases. Future work could investigate more interpretable detection methods.

**Table 6: Z-scores of waterbag method and Exp-Edit method. represents high-confidence watermark and represents low-confidence watermark, while no color means no watermark. This table provides supplementary information on the similarity content in Table 2.**
|  |  | KGW w. Water-Bag | Exp-Edit(Key-len) |  |  |  |  |  |
| --- | --- | --- | --- | --- | --- | --- | --- | --- |
|  | None | $|K\cup\overline{K}|=1$ | $|K\cup\overline{K}|=2$ | $|K\cup\overline{K}|=4$ | $|K\cup\overline{K}|=8$ | 420 | 1024 | 2048 |
| Watermarked LLM Indentification |  |  |  |  |  |  |  |  |
| Water-Probe-v1(n=3) | -8.20 | 11.67 | -4.22 | -7.84 | -4.34 | -3.07 | -5.87 | -5.17 |
| Water-Probe-v2(n=3) | -2.87 | 24.87 | 23.49 | 16.19 | 3.08 | 47.74 | 18.70 | 6.17 |
| Water-Probe-v2(n=5) | -100.43 | 28.56 | 13.68 | 1.38 | -4.41 | 131.30 | 78.63 | 69.40 |

Figure: Figure 5: The variation of z-scores at different temperatures when calculating similarity without using rank transformation in Equation 9.
Refer to caption: x6.png

**Table 7: Z-scores for watermark detection on various LLMs with and without watermarks using Watermark-Probe-1 and Watermark-Probe-2. indicates high-confidence watermark identification, indicates low-confidence watermark identification, while no color indicates no watermark identified. This table provides supplementary information on the similarity content in Table 1.**
| LLM | N-Gram | Fixed-Key-Set |  |  |  |  |  |  |  |
| --- | --- | --- | --- | --- | --- | --- | --- | --- | --- |
| Non | KGW | Aar | KGW-Min | KGW-Skip | DiPmark | $\gamma$-Reweight | EXP-Edit | ITS-Edit |  |
| Watermark-Probe-1 (w. prompt 4.3) |  |  |  |  |  |  |  |  |  |
| Qwen2.5-1.5B | -5.02 | 43.32 | 123.57 | 14.70 | 24.29 | 33.35 | 55.90 | -5.21 | -25.04 |
| OPT-2.7B | -5.99 | 49.21 | 117.95 | 16.69 | 35.78 | 93.25 | 49.98 | -1.32 | -1.27 |
| Llama-3.2-3B | -4.52 | 49.39 | 79.33 | 80.94 | 71.11 | 87.17 | 76.35 | -6.70 | -4.29 |
| Qwen2.5-3B | -6.70 | 48.20 | 127.07 | 14.73 | 625.36 | 56.15 | 42.96 | -6.18 | -1.97 |
| Llama2-7B | -8.20 | 30.01 | 109.00 | 25.51 | 29.75 | 44.35 | 106.37 | -3.07 | -28.05 |
| Mixtral-7B | -3.80 | 25.03 | 40.21 | 35.51 | 19.99 | 36.30 | 137.39 | -22.01 | -3.31 |
| Qwen2.5-7B | -1.16 | 38.25 | 30.03 | 22.85 | 50.34 | 47.31 | 50.48 | -3.39 | -15.92 |
| Llama-3.1-8B | -6.44 | 20.40 | 143.52 | 29.05 | 28.19 | 29.01 | 125.46 | -3.79 | -12.70 |
| Llama2-13B | -3.62 | 29.23 | 79.42 | 11.74 | 18.01 | 30.40 | 49.23 | -6.63 | -2.78 |
| Watermark-Probe-2 (w. prompt 4.3) |  |  |  |  |  |  |  |  |  |
| Qwen2.5-1.5B | -5.06 | 34.55 | 55.97 | 16.39 | 8.79 | 19.45 | 14.28 | 10.73 | 1840.44 |
| OPT-2.7B | -1.95 | 42.59 | 67.93 | 15.17 | 4.61 | 32.40 | 11.04 | 26.44 | 1073.13 |
| Llama-3.2-3B | -12.42 | 29.91 | 96.14 | 50.00 | 19.91 | 80.04 | 67.07 | 34.99 | 7702.12 |
| Qwen2.5-3B | -3.48 | 6.35 | 108.44 | 11.29 | 25.92 | 39.88 | 18.84 | 18.06 | 8209.12 |
| Llama2-7B | -2.87 | 24.87 | 40.05 | 32.68 | 15.62 | 35.50 | 25.11 | 47.74 | 6885.04 |
| Mixtral-7B | -0.87 | 6.09 | 54.39 | 11.14 | 13.49 | 49.02 | 111.12 | 14.83 | 1812.12 |
| Qwen2.5-7B | -2.48 | 8.90 | 185.88 | 10.50 | 13.06 | 7.64 | 12.40 | 13.04 | 1982.74 |
| Llama-3.1-8B | -64.31 | 25.24 | 104.77 | 10.03 | 49.23 | 38.47 | 81.36 | 31.35 | 12701.95 |
| Llama2-13B | -3.98 | 20.72 | 38.26 | 10.36 | 16.60 | 58.10 | 83.27 | 47.74 | 333.37 |

**Table 8: Identification p-value for various LLMs with different watermarks and without watermarks, using our two methods: Watermark-Probe-v1 and Watermark-Probe-v2. represents high-confidence watermark and represents low-confidence watermark, while no color means no watermark. This table provides supplementary information on the similarity content in Table 1.**
| LLM | N-Gram | Fixed-Key-Set |  |  |  |  |  |  |  |
| --- | --- | --- | --- | --- | --- | --- | --- | --- | --- |
| Non | KGW | Aar | KGW-Min | KGW-Skip | DiPmark | $\gamma$-reweight | EXP-Edit | ITS-Edit |  |
| Watermark-Probe-v1 (w. prompt 4.3) |  |  |  |  |  |  |  |  |  |
| Qwen2.5-1.5B | 1 | 2.9e-410 | 5.9e-3319 | 3.2e-49 | 1.3e-130 | 3.6e-244 | 2.0e-681 | 1 | 1 |
| OPT-2.7B | 1 | 1.1e-528 | 3.4e-3024 | 7.7e-63 | 1.1e-280 | 2.6e-1891 | 2.9e-545 | 1-9.3e-2 | 1-1.0e-1 |
| Llama-3.2-3B | 1 | 1.6e-532 | 1.4e-1369 | 1.3e-1425 | 5.2e-1101 | 4.4e-1653 | 7.9e-1269 | 1 | 1 |
| Qwen2.5-3B | 1 | 2.7e-507 | 1.8e-3509 | 2.1e-49 | 8.3e-84925 | 1.7e-687 | 1.6e-403 | 1 | 1-2.4e-2 |
| Llama2-7B | 1 | 3.6e-198 | 4.3e-2583 | 7.6e-144 | 8.7e-195 | 7.0e-430 | 4.4e-2460 | 1-1.1e-3 | 1 |
| Mixtral-7B | 1 | 1.4e-138 | 8.0e-354 | 1.7e-276 | 3.4e-89 | 8.1e-289 | 3.9e-4102 | 1 | 1 |
| Qwen2.5-7B | 1-1.2e-1 | 2.1e-320 | 2.0e-198 | 7.3e-116 | 4.2e-553 | 7.9e-489 | 3.6e-556 | 1 | 1 |
| Llama-3.1-8B | 1 | 8.4e-93 | 4.4e-4476 | 7.7e-186 | 3.9e-175 | 2.5e-185 | 3.6e-3421 | 1 | 1 |
| Llama2-13B | 1 | 4.0e-188 | 1.1e-1372 | 4.0e-32 | 8.1e-73 | 2.7e-203 | 4.3e-529 | 1 | 1-2.7e-3 |
| Watermark-Probe-v2 (w. prompt 4.3) |  |  |  |  |  |  |  |  |  |
| Qwen2.5-1.5B | 1 | 7.1e-262 | 4.1e-683 | 1.1e-60 | 7.5e-19 | 1.5e-84 | 1.5e-46 | 3.7e-27 | 9.8e-735530 |
| OPT-2.7B | 1-2.6e-2 | 1.2e-396 | 5.6e-1005 | 2.8e-52 | 2.0e-6 | 1.4e-230 | 1.2e-28 | 2.4e-154 | 1.2e-250072 |
| Llama-3.2-3B | 1 | 7.3e-197 | 3.5e-2010 | 1.1e-545 | 1.7e-88 | 3.7e-1394 | 9.2e-980 | 1.6e-268 | 2.5e-12881755 |
| Qwen2.5-3B | 1 | 1.1e-10 | 1.2e-2556 | 7.4e-30 | 2.0e-148 | 4.4e-348 | 1.8e-79 | 3.3e-73 | 7.3e-14633482 |
| Llama2-7B | 1-2.1e-3 | 7.9e-137 | 4.9e-351 | 1.5e-234 | 2.7e-55 | 2.5e-276 | 1.9e-139 | 1.0e-497 | 4.3e-10293604 |
| Mixtral-7B | 1-1.9e-1 | 5.6e-10 | 3.1e-645 | 4.0e-29 | 9.0e-42 | 1.3e-524 | 2.0e-2684 | 4.7e-50 | 6.5e-713068 |
| Qwen2.5-7B | 1-6.6e-3 | 2.8e-19 | 3.9e-7506 | 4.3e-26 | 2.8e-39 | 1.1e-14 | 1.3e-35 | 3.6e-39 | 3.1e-853666 |
| Llama-3.1-8B | 1 | 7.3e-141 | 1.0e-2386 | 5.6e-24 | 4.3e-529 | 4.5e-324 | 2.0e-1440 | 4.9e-216 | 7.5e-35034440 |
| Llama2-13B | 1 | 1.1e-95 | 1.4e-320 | 1.9e-25 | 3.5e-62 | 6.8e-736 | 1.0e-1508 | 1.0e-497 | 2.0e-24136 |

### Appendix F Details of Tested Watermarking Algorithms

To help understand the watermarking algorithms related to the experiments in this paper, we provide detailed information for all watermarking algorithms in Table 5, including their names, references, types, and brief descriptions. All our experiments were conducted using the MarkLLM (Pan et al., ) framework.

### Appendix G Ablation of Rank Transformation

To illustrate the importance of the rank transformation mentioned in Equation 9, we present in Figure 5 the variation of z-scores at different temperatures without using rank transformation. It can be observed that without rank transformation, the z-scores for Unwatermarked LLMs are significantly higher, especially at lower temperatures. Comparing the left plots in Figures 3 and 5, we can see that rank transformation effectively reduces the z-scores of Unwatermarked LLMs, making the identification and detection more stable.

### Appendix H Supplementary Z-Scores and P-values

To facilitate a better understanding of the statistical methods used in identifying watermarked LLMs, we provide detailed information including z-scores and p-values for Table 1 and Table 2 in this section.

Specifically, Table 6 provides supplementary z-score information for Table 2, Table 7 provides supplementary z-score information for Table 1, and Table 8 provides supplementary p-value information for Table 1.

For all experiments, we consider a z-score below 4 to indicate no watermark, between 4 and 10 to indicate a watermark with relatively low confidence, and above 10 to indicate a watermark with high confidence.

## Peer Reviews

confidence: 4
contribution: 4
initial_rating: 8
presentation: 4
questions: How about the blackbox detection on LLMs in real world? What is the difficulty?
soundness: 4
strengths: 1 This work is the first study on the imperceptibility of watermarked LLMs. 
2 This paper is well organized and written, make it easy to follow.
3 The experiments are conducted across different LLMs and with different sampling methods and temperature settings. The conclusion and discussion based on the evaluation results are clear.
summary: Considering that “even if individual watermarked texts are imperceptible, the distribution of numerous watermarked texts may reveal whether the LLM is watermarked, especially when repeatedly sampling with the same watermark key”, this paper proposed Water-Probe to detect if an LLM service contains watermarks through well-designed prompts then introduced Water-Bag strategy to improve watermark imperceptibility by merging multiple watermark keys.
weaknesses: The threat model should be further described, especially in terms of the prior knowledge  assumptions of the detector.

---

confidence: 3
contribution: 3
initial_rating: 6
presentation: 3
questions: 1. In Table 1, why Water-Probe-V2 has low detection confidence for KGW series watermarks?
2. Could you provide some experiments on watermark detection on closed-source models such as ChatGPT using your tool? Since the proposed method is a black-box setting, this should be achievable.
3. In the method for enhancing the key pseudorandomness of the Water-Bag Strategy, your use of the "set of master keys and their inversions" approach lacks theoretical support. Does "inversions" refer to bit flips, or does it refer to certain keys that satisfy the constraints of equation (12)? If it’s the former, then the claim of enhanced pseudorandomness has no basis; if it’s the latter, the paper does not explain how to derive the inversion key space that satisfies equation (12), and it seems that satisfying (12) does not have a direct relationship with the pseudorandomness of $K^*_j$. The experiments in Table 2 test methods against the attacks proposed by the authors, without directly testing the pseudorandomness of the sequences in the key space using suites like NIST SP 800-22. Ultimately, this seems to only indicate that the specific method enhances defense capabilities against the proposed attacks, rather than demonstrating that the enhancement of pseudorandomness leads to improved defense capabilities, which is inconsistent with the authors' previous claims.
soundness: 3
strengths: 1. It is reasonable to use repeated sampling to detect whether an LLM has been watermarked. 
2. Prompts have been designed to reveal the connection between generated text and watermark keys in a black-box setting.
3. The evaluation is comprehensive, and many different watermarking methods have been tested. Experimental results also show that the proposed method works.
summary: The paper proposed a method called Water-Probe to detect whether a LLM is watermarked through well-designed prompts. Experiments show the method is effective to almost all mainstream watermarking algorithms. The authors also proposed a strategy to improves watermark imperceptibility by multiple keys.
weaknesses: 1. Symbol reuse. In Section 3 and Section 4, the symbol $P$ represents both the model distribution in $P_M$ and the prompt in $P_1, P_2, ..., P_N$, which can be confusing. Too many $P$ across line 222 to line 227.
2. Line 289, Figure 2. ‘KTH’ does not refer to anything else in other contexts. Is it Exp-Edit or ITS-Edit?

---

confidence: 4
contribution: 2
initial_rating: 5
presentation: 3
questions: - **Concept Confusion**: the authors mislead watermarking concept.
LLM Watermarking is the process of injecting a invisible identifier into LLM, track unauthorized distribution, or indicate authenticity, withch contains two steps **injection** and **detection**. The author just talk about how to identify LLM output consistency, I never see any discussion about watermark injection in paper. The authors should clarify how their work relates to the injection and detection steps they have outlined, or explain why their approach focuses solely on identification without addressing injection.

- **Unknown Key Sampling Strategy**: the author propose the watermark identification method using setected prompts.
Step 1-3 only discuss the object of identification key sampling, never talk about the detail of identification key sampling.

- **Method Designed**: The author says “Research on the imperceptibility of watermarked LLMs should investigate whether watermarked and non-watermarked LLMs can be distinguished.”. However, in this paper, I never see the sampled prompts and identification method can solve this problem. The authors should clearly illustrate how the proposed method differentiates between watermarked and non-watermarked LLMs. I recommend including a specific experiment or analysis that directly addresses this assertion.
soundness: 3
strengths: - **Accuracy and Robustness**: The experimental results demonstrate the high accuracy and robustness of the Water-Probe algorithm across various LLMs, watermarking methods, and generation settings. The low false positive rate for non-watermarked LLMs further strengthens the algorithm’s reliability.
- **Practical Solution**: The proposed Water-Bag strategy offers a practical solution to improve the imperceptibility of watermarks, which is a critical concern for LLM providers.
summary: This paper addresses the imperceptibility of watermarking techniques used in Large Language Models (LLMs). It highlights that while existing watermarking techniques are effective in terms of detectability, minimal impact on text quality, and robustness to editing, they have not been thoroughly examined for their imperceptibility to users. The authors introduce an identification algorithm, Water-Probe, which is capable of detecting watermarks through carefully designed prompts. To enhance watermark imperceptibility, the authors propose a Water-Bag strategy, which merges multiple watermark keys to increase randomness.
weaknesses: - **Concept Confusion**: The manuscript mislead concept of watermarking and fingerprinting, the proposed method should be categorized into fingerprinting instead of watermarking. see Question 1.
- **Lack of Controbution**: The authors inputs prompts to see the response of watermarked LLM and non-watermarked LLM, which is so called identification algorithm. The manuscript just defeines some concept, samples prompt to see similarity of the inspected models. 
- **Limited Scope of Water-Probe**: The Water-Probe algorithm might be limited in its application to only certain types of watermarking schemes. The paper does not sufficiently explore how the algorithm would perform against a wider range of watermarking techniques, especially those that are more sophisticated or less predictable.

---

confidence: 3
contribution: 3
initial_rating: 6
presentation: 3
questions: Why is the similarity of distribution differences between prompts in the no-watermark case low?
soundness: 3
strengths: The paper identifies watermarks in a black-box setting. Also, it proposes an effective method to improve watermark imperceptibility.
summary: The paper proposes an identification algorithm to detect watermarks through well-designed prompts to LLMs, as current watermarked LLMs have consistent biases under the same watermark key.
weaknesses: The primary concern I have with the paper is the similarity of distribution differences between prompts in the no-watermark case. First, cosine similarity measures the similarity between two non-zero vectors; thus, the equation Sim(0, 0) in the proof in Appendix B is confusing. Second, if we examine the distances of the elements directly rather than using the Sim function, the distances of the elements in the no-watermark case are approximately zero, suggesting high similarity.

The selection of prompts appears empirical. The paper could be strengthened by providing guidance on generating prompts that produce similar output distributions.

Meta-review: This paper focuses on the imperceptibility of watermarking techniques in Large Language Models (LLMs). While existing methods ensure detectability, robustness, and minimal text quality degradation, their imperceptibility to users remains underexplored. The authors introduce Water-Probe, an identification algorithm that detects watermarks by using carefully designed prompts, proving effective against most mainstream watermarking methods. To address imperceptibility, the authors propose the Water-Bag strategy, which enhances randomness by merging multiple watermark keys. Experiments demonstrate the effectiveness of Water-Probe for watermark detection and the ability of Water-Bag to improve imperceptibility, advancing the evaluation and design of watermarking techniques for LLMs.

Overall, the problem is new and the approach is interesting. Since all the reviewers recommend acceptance; I also vote for acceptance.

Decision: Accept (Spotlight)