Title: Segmenting Watermarked Texts From Language Models
Venue: NeurIPS 2024
Accepted: yes
Keywords: Large language models

Randomization test

Segmentation

Watermark
Abstract: Watermarking is a technique that involves embedding nearly unnoticeable statistical signals within generated content to help trace its source. This work focuses on a scenario where an untrusted third-party user sends prompts to a trusted language model (LLM) provider, who then generates a text from their LLM with a watermark. This setup makes it possible for a detector to later identify the source of the text if the user publishes it. The user can modify the generated text by substitutions, insertions, or deletions. Our objective is to develop a statistical method to detect if a published text is LLM-generated from the perspective of a detector. We further propose a methodology to segment the published text into watermarked and non-watermarked sub-strings. The proposed approach is built upon randomization tests and change point detection techniques. We demonstrate that our method ensures Type I and Type II error control and can accurately identify watermarked sub-strings by finding the corresponding change point locations. To validate our technique, we apply it to texts generated by several language models with prompts extracted from Google's C4 dataset and obtain encouraging numerical results. We release all code publicly at https://github.com/doccstat/llm-watermark-cpd.

Segmenting Watermarked Texts From Language Models

Abstract

Watermarking is a technique that involves embedding nearly unnoticeable statistical signals within generated content to help trace its source. This work focuses on a scenario where an untrusted third-party user sends prompts to a trusted language model (LLM) provider, who then generates a text from their LLM with a watermark. This setup makes it possible for a detector to later identify the source of the text if the user publishes it. The user can modify the generated text by substitutions, insertions, or deletions. Our objective is to develop a statistical method to detect if a published text is LLM-generated from the perspective of a detector. We further propose a methodology to segment the published text into watermarked and nonwatermarked sub-strings. The proposed approach is built upon randomization tests and change point detection techniques. We demonstrate that our method ensures Type I and Type II error control and can accurately identify watermarked sub-strings by finding the corresponding change point locations. To validate our technique, we apply it to texts generated by several language models with prompts extracted from Google's C4 dataset and obtain encouraging numerical results. 1

1 Introduction

With the increasing use of large language models in recent years, it has become essential to differentiate between text generated by these models and text written by humans. Some of the most advanced LLMs, such as GPT-4, Llama 3, and Gemini, are very good at producing human-like texts, which could be challenging to distinguish from human-generated texts, even for humans. However, it is crucial to distinguish between human-produced texts and machine-produced texts to prevent the spread of misleading information, improper use of LLM-based tools in education, model extraction attacks through distillation, and the contamination of training datasets for future language models.

Watermarking is a principled method for embedding nearly unnoticeable statistical signals into text generated by LLMs, enabling provable detection of LLM-generated content from its human-written

counterpart. This work focuses on a scenario where an untrusted third-party user sends prompts to a trusted large language model (LLM) provider, who then generates a text from their LLM with a watermark. This makes it possible for a detector to later identify the source of the text if the user publishes it. The user is allowed to modify the generated text by making substitutions, insertions, or deletions before publishing it. We aim to develop a statistical method to detect if a published text is LLM-generated from the perspective of a detector. When the text is declared to be watermarked, we study a problem that has been relatively less investigated in the literature: to separate the published text into watermarked and non-watermarked sub-strings. In particular, we divide the texts into a sequence of moving sub-strings with a pre-determined length and sequentially test if each sub-string is watermarked based on the p -values obtained from a randomization test. Given the p -value sequence, we examine if there are structural breaks in the underlying distributions. For non-watermarked sub-strings, the p -values are expected to be uniformly distributed over [0 , 1] , while for watermarked sub-strings, the corresponding p -values are more likely to concentrate around zero. By identifying the change points in the distributions of the p -values, we can segment the texts into watermarked and non-watermarked sub-strings.

In theory, we demonstrate that our method ensures Type I and Type II error control and can accurately identify watermarked sub-strings by finding the corresponding change point locations. Specifically, we obtain the convergence rate for the estimated change point locations. To validate our technique, we apply it to texts generated by different language models with prompts extracted from the real news-like data set from Google's C4 dataset, followed by different modifications/attacks that can introduce change points in the text.

1.1 Related works and contributions

Digital watermarking is a field that focuses on embedding a signal in a medium, such as an image or text, and detecting it using an algorithm. Early watermarking schemes for natural language processing-generated text were presented by Atallah et al. [2001]. Hopper et al. [2007] formally defined watermarking and its desired properties, but their definitions were not specifically tailored to LLMs. More recently, Abdelnabi and Fritz [2021] and Munyer and Zhong [2023] proposed schemes that use machine learning models in the watermarking algorithm. These schemes take a passage of text and use a model to produce a semantically similar altered passage. However, there is no formal guarantee for generating watermarked text with desirable properties when using machine learning.

The current work is more related to Aaronson [2023], Kirchenbauer et al. [2023a], Kuditipudi et al. [2023]. In particular, Kirchenbauer et al. [2023a] introduced a 'red-green list" watermarking technique that splits the vocabulary into two lists based on hash values of previous tokens. This technique slightly increases the probability of embedding the watermark into 'green" tokens. Other papers that discuss this type of watermarking include Kirchenbauer et al. [2023b], Cai et al. [2024], Liu and Bu [2024], and Zhao et al. [2023]. However, this watermarking technique is biased, as the next token prediction (NTP) distributions have been modified, leading to a performance degradation of the LLM. Aaronson [2023] describes a technique for watermarking LLMs using exponential minimum sampling to sample tokens from an LLM, where the inputs to the sampling mechanism are also a hash of the previous tokens. This approach is closely related to the so-called Gumbel trick in machine learning. Kuditipudi et al. [2023] proposed an inverse transform watermarking method that can be made robust against potential random edits. Other unbiased watermarks in this fast-growing line of research include Zhao et al. [2024], Fernandez et al. [2023], Hu et al. [2023], Wu et al. [2023].

However, less attention has been paid to understanding the statistical properties of watermark generation and detection schemes. The paper by Huang et al. [2023] considers watermark detection as a problem of composite dependence testing. The authors aim to understand the minimax Type II error and the most powerful test that achieves it. However, Huang et al. [2023] assumes that the NTP distributions remain unchanged from predicting the first token to the last token, which can be unrealistic. On the other hand, Li et al. [2024] introduced a flexible framework for determining the statistical efficiency of watermarks and designing powerful detection rules. This framework reduces the problem of determining the optimal detection rule to solving a minimax optimization program.

Compared to the existing literature, we make two contributions:

(a) We rigorously study the Type I and Type II errors of the randomization test to test the existence of a watermark; see Theorems 1-2. We apply these results to the recently proposed inverse transform watermark and Gumbel watermark schemes; see Corollary 1.

(b) We develop a systematic statistical method to segment texts into watermarked and nonwatermarked sub-strings. We have also investigated the theoretical and finite sample performance of this methodology. As far as we know, this problem has not been studied in recent literature on generating and detecting watermarked texts from LLMs.

2 Problem setup

2.1 Watermarked text generation

Denote by V the vocabulary (a discrete set), and let P be an autoregressive LLM which maps a string y -n 0 : t -1 = y -n 0 y -n 0 +1 · · · y t -1 ∈ V t + n 0 to a distribution over the vocabulary, with p ( ·| y -n 0 : t -1 ) being the distribution of the next token y t . Here y -n 0 :0 denotes the prompt provided by the user. Set V = |V| be the vocabulary size, and let ξ 1: t = ξ 1 ξ 2 · · · ξ t be a watermark key sequence with ξ i ∈ Ξ for each i , where Ξ is a general space. Given a prompt sent from a third-party user, the LLM provider calls a generator to autoregressitvely generate text from an LLM using a decoder function Γ , which maps ξ t and a distribution p t over the next token to a value in V . The watermarking scheme should preserve the original text distribution, i.e., P (Γ( ξ t , p t ) = y ) = p t ( y ) . A watermark text generation algorithm recursively generates a string y 1: n by

where n is the number of tokens in the text y 1: n generated by the LLM, and ξ i 's are assumed to be independently generated from some distribution ν over Ξ . In other words, given p ( ·| y -n 0 : i -1 ) , y i is completely determined by ξ i and y -n 0 : i -1 .

For ease of presentation, we associate each token in the vocabulary with a unique index from [ V ] := { 1 , 2 , . . . , V } , and we remark that the generator should be invariant to this assignment.

Example 1 (Inverse transform sampling) . In this example, we describe a watermarked text generation method developed in Kuditipudi et al. [2023] and discuss an alternative formulation of their approach. Write µ i ( k ) = p ( k | y -n 0 : i -1 ) for 1 ≤ k ≤ V and 1 ≤ i ≤ n . To generate the i th token, we consider a permutation π i on [ V ] and u i ∼ Unif [0 , 1] (which denotes the uniform distribution over [0 , 1] ), which jointly act as the key ξ i . Let

We note that Γ(( π i , u i ) , µ i ) = k if min { π i ( l ) : µ i ( j : π i ( j ) ≤ π i ( l )) ≥ u i } = π i ( k ) , which implies that

As the length of this interval is µ i ( k ) , P (Γ(( π i , u i ) , µ i ) = k ) = µ i ( k ) . An alternate way to describe the same generator is as follows: Given a partition of the interval [0 , 1] into V sub-intervals denoted by I 1 , . . . , I V , we can order them in such a way that each interval I i is adjacent to its immediate right neighbor I i +1 . Now let I ( k ; i ) 's be V intervals with the length |I ( k ; i ) | = p ( k | y -n 0 : i -1 ) for 1 ≤ k ≤ V . Given the permutation π i , we can order the V intervals through π i , i.e., {I π i ( k ) ( k ; i ) : k = 1 , 2 , . . . , V } . Define ξ ik = I { u i ∈ I π i ( k ) ( k ; i ) } and set y i = k if ξ ik = 1 . Clearly P ( y i = k ) = P ( ξ ik = 1) = P ( u i ∈ I π i ( k ) ( k ; i )) = p ( k | y -n 0 : i -1 ) .

Now, for a string ˜ y 1: n (with the same length as the key) that is possibly watermarked, Kuditipudi et al. [2023] suggest the following metric to quantify the dependence between the watermark key and the string:

Observe that if ˜ y i is generated using the above scheme with the key ξ i = ( π i , u i ) , then u i and π i ( ˜ y i ) are positively correlated. Thus, a large value of M indicates that ˜ y 1: n is potentially watermarked.

Example 2 (Exponential minimum sampling) . Wedescribe another watermarking technique proposed in Aaronson [2023]. To generate each token of a text, we first sample ξ ik ∼ Unif [0 , 1] independently for 1 ≤ k ≤ V . Let

where E ik := -log( ξ ik ) /p ( k | y -n 0 : i -1 ) ∼ Exp ( p ( k | y -n 0 : i -1 )) with Exp ( a ) denoting an exponential random variable with the rate a . For two exponential random variables X ∼ Exp ( a ) and Y ∼ Exp ( b ) , we have two basic properties: (i) min( X,Y ) ∼ Exp ( a + b ) ; (ii) P ( X < Y ) = E [1 -exp( -aY )] = a/ ( a + b ) . Using (i) and (ii), it is straightforward to verify that

̸

Hence, this generation scheme preserves the original text distribution.

Aaronson [2023] proposed to measure the dependence between a string ˜ y 1: n and the key sequence ξ 1: n using the metric

The idea behind the definition of this function is that if ˜ y i was generated using the key ξ i , then ξ i, ˜ y i tends to have a higher value than the other components of ξ i . Therefore, a larger value of the metric indicates that the string ˜ y 1: n is more likely to be watermarked.

2.2 Watermarked text detection

We now consider the detection problem, which involves determining whether a given text is watermarked or not. Consider the case where a string ˜ y 1: m is published by the third-party user and a key sequence ξ 1: n is provided to a detector. The detector calls a detection method to test

by computing a p -value with respect to a test statistic ϕ ( ξ 1: n , ˜ y 1: m ) . It is important to note that the text published by the user can be quite different from the text initially generated by the LLM using the key ξ 1: n , which we refer to as y 1: n . To account for this difference, we can use a transformation function E that takes y 1: n as the input and produces the published text ˜ y 1: m as the output, i.e., ˜ y 1: m = E ( y 1: n ) . This transformation can involve substitutions, insertions, deletions, paraphrases, or other edits to the input text.

The test statistic ϕ measures the dependence between the text ˜ y 1: m and the key sequence ξ 1: n . Throughout our discussions, we will assume that a large value of ϕ provides evidence against the null hypothesis (e.g., stronger dependence between ˜ y 1: m and ξ 1: n ). To obtain the p -value, we consider a randomization test. In particular, we generate ξ ( t ) i ∼ ν independently over 1 ≤ i ≤ n and 1 ≤ t ≤ T , and ξ ( t ) i s are independent with ˜ y 1: m . Then the randomization-based p -value is given by

Theorem 1. For the randomization test, we have the following results.

(i) Under the null, P ( p T ≤ α ) = ⌊ ( T +1) α ⌋ / ( T +1) ≤ α , where ⌊ a ⌋ denotes the greatest integer that is less than or equal to a ;

(ii) Suppose the following three conditions hold:

(a) max { Var( ϕ ( ξ 1: n , ˜ y 1: m ) |F m ) , Var( ϕ ( ξ ′ 1: n , ˜ y 1: m ) |F m ) } ≤ C v /n with C v > 0 ;

(c) lim n →∞ n E [ ϕ ( ξ 1: n , ˜ y 1: m ) |F m ] = ∞ .

(b) E [ ϕ ( ξ ′ 1: n , ˜ y 1: m ) |F m ] = O ( n -1 / 2 ) ; √

Here F m = [ y -n 0 :0 , ˜ y 1: m ] and ξ ′ 1: n is a key sequence generated in the same way as ξ 1: n but is independent of ˜ y 1: m . Given any ϵ > 0 , when T > 2 /ϵ -1 ,

We now apply the results in Theorem 1 to Examples 1-2 with m = n and ϕ ( ξ 1: n , ˜ y 1: n ) = M ( ξ 1: n , ˜ y 1: n ) for M defined in (1) and (2).

Corollary 1. If ˜ y 1: n = y 1: n and

then Conditions (a)-(c) in Theorem 1 are satisfied for Examples 1-2. Consequently, the power of the randomization test converges to 1 in these examples as T → + ∞ .

However, as the published text can be modified, it is not expected that every token in ˜ y 1: m will be related to the key sequence. Instead, we expect certain sub-strings of ˜ y 1: m to be correlated with the key sequence under the alternative hypothesis H a . To measure the dependence, we use a scanning method that looks at every segment/sub-string of ˜ y 1: m and a segment of ξ 1: n with the same length B . We use a measure M ( ξ a : a + B -1 , ˜ y b : b + B -1 ) to quantify the dependence between ξ a : a + B -1 and ˜ y b : b + B -1 , chosen based on the watermarked text generation method described above. Given M and the block size B , we can define the maximum test statistic as

Theorem 2. Consider the maximum statistic defined in (5), where the dependence measure takes the form of M ( ξ a : a + B -1 , ˜ y b : b + B -1 ) = B -1 ∑ B -1 i =0 h i ( ξ a + i , ˜ y b + i ) and h i s are independent conditional on [ ˜ y 1: m , y -n 0 : n ] . Under the setting of Example 1, max i | h i | ≤ 1 / 4 . In this case, suppose

where N = max { n, m } and C N,B = √ log( N ) /B . Then, (3) holds true. Under the setting of Example 2, h i s are exponentially distributed conditional on [ ˜ y 1: m , y -n 0 : n ] . In this case, (3) is still true under (6) with C N,B = log( N ) /B.

3 Sub-string identification

In this section, we aim to address the following question, which seems less explored in the existing literature: given that the global null hypothesis H 0 is rejected, how can we identify the sub-strings from the modified text ˜ y 1: m that are machine-generated?

To describe the setup, we suppose the text published by the third-party user has the following structure:

in which case the sub-strings ˜ y τ 1 +1 · · · ˜ y τ 2 and ˜ y τ 3 +1 · · · ˜ y τ 4 are watermarked. We emphasize that the orders of the watermarked and non-watermarked sub-strings can be arbitrary and do not affect our method described below. The goal here is to separate the text into watermarked and nonwatermarked sub-strings accurately. Our key insight to tackling this problem is translating it into a change point detection problem. To describe our method, we define a sequence of moving windows I i = [( i -B/ 2) ∨ 1 , ( i + B/ 2) ∧ m ] with B being the window size which is assumed to be an even number (for the ease of presentation) and 1 ≤ i ≤ m . For each sub-string, we define a randomization-based p -value given by

where we let ϕ ( ξ 1: n , ˜ y I i ) = max 1 ≤ k ≤ n M ( ξ J k , ˜ y I i ) with J k = [( k -B/ 2) ∨ 1 , ( k + B/ 2) ∧ n ] .

We have now transformed the text into a sequence of p -values: p 1 , . . . , p m . Under the null, the p -values are roughly uniformly distributed, while under the alternatives, the p -values will concentrate around zero. Consider a simple setup where the published text can be divided into two halves, with the first half watermarked and the second half non-watermarked (or vice versa). Then, we can identify the change point location through

where F a : b ( t ) denotes the empirical cumulative distribution function (cdf) of { p i } b i = a .

We shall use the block bootstrap-based approach to determine if ˆ τ is statistically significant. Note that conditional on F m , the p -value sequence is B -dependent in the sense that p i and p j are independent only if | i -j | > B . Hence, the usual bootstrap or permutation methods for independent data may not work as they fail to capture the dependence from neighboring p -values. Instead, we can employ the so-called moving block bootstrap for time series data [Kunsch, 1989, Liu et al., 1992]. Given a block size, say B ′ , we create m -B ′ +1 blocks given by { p i , . . . , p i + B ′ -1 } for 1 ≤ i ≤ m -B ′ +1 . We randomly sample m/B ′ (assuming that m/B ′ is an integer for simplicity) blocks with replacement and paste them together. Denote the resulting resampled p -values by p ∗ 1 , . . . , p ∗ m . We then compute F ∗ a,b ( t ) based on the bootstrapped p -values and define

Repeat this procedure T ′ times and denote the statistics by S ∗ , ( t ) 1: m ( τ ) for t = 1 , 2 , . . . , T ′ . Define the corresponding bootstrap-based p -value as

We claim that there is a statistically significant change point if ˜ p T ′ ≤ α .

Remark 1. Alternatively, one can consider the Cramér-von Mises type statistic max τ C 1: m ( τ ) with

to examine the existence of a change point, where w ( · ) is a non-negative weight function defined over [0 , 1] . If a change point exists, we can estimate its location through ˜ τ = arg max 1 ≤ τ<m C 1: m ( τ ) . Other methods to quantify the distributional shift for change point detection include the distance and kernel-based two-sample metrics [Matteson and James, 2014, Chakraborty and Zhang, 2021] and graph-based test statistics [Chen and Zhang, 2015].

Consider the case where there is a single change point located at τ ∗ . Without loss of generality, let us assume that before the change, the p -values are uniformly distributed, i.e., p 1 , . . . , p τ ∗ ∼ F 0 with F 0 ( t ) = t (the cdf of Unif [0 , 1] ), while after the change, the p -values follow different distributions that concentrate around zero. We assume that lim inf m → + ∞ D ( F 0 , E [ F τ ∗ +1: m ( t )]) > 0 , where D ( F 0 , E [ F τ ∗ +1: m ( t )]) := sup t ∈ [0 , 1] | F 0 ( t ) -E [ F τ ∗ +1: m ( t )] | is the Kolmogorov-Smirnov distance. In other words, the empirical cdf of the p -values from the watermarked sub-strings converges to a distribution that is different from the uniform distribution. This mild assumption allows for the detection of the structure break. As discussed before, we consider the scan statistic max 1 ≤ τ<m S 1: m ( τ ) with S 1: m ( τ ) defined in (9) for examining the existence of a change point.

Proposition 1. Assuming that B → + ∞ and B/m → 0 , we have

Next, we shall establish the consistency of the change point estimator ˆ τ = arg max 1 ≤ τ<m S 1: m ( τ ) . In particular, we obtain the following result, which establishes the convergence rate of the change point estimate.

Theorem 3. Under the Assumptions in Proposition 1, we have

Remark 2. Our scenario differs from the traditional nonparametric change point detection problem in a few ways: (i) Rather than testing the homogeneity of the original data sequence (which is the setup typically considered in the change point literature), we convert the string into a sequence of

p-values, based on which we conduct the change-point analysis; (ii) In the classical change point literature, the observations (in our case, the p-values) within the same segment are assumed to follow the same distribution. In contrast, for the watermark detection problem, the p-values from the watermarked segment could follow different distributions, adding a layer of difficulty to the analysis; (iii) The p-value sequence is dependent (where the strength of dependence is controlled by B ), making our setup very different from the one in Carlstein [1988], which assumed the underlying data sequence to be independent; (iv) The technical tool used in our analysis must account for the particular dependence structure within the p-value sequence.

3.1 Binary segmentation

In this section, we describe an algorithm to separate watermarked and non-watermarked sub-strings by identifying multiple change point locations. There are two main types of algorithms for identifying multiple change points in the literature: (i) exact or approximate optimization by minimizing a penalized cost function [Harchaoui and Lévy-Leduc, 2010, Truong et al., 2020, R. Killick and Eckley, 2012, Li and Zhang, 2024, Zhang and Dawn, 2023] and (ii) approximate segmentation algorithms. Our proposed algorithm is based on the popular binary segmentation method, a top-down approximate segmentation approach for finding multiple change point locations. Initially proposed by Vostrikova [1981], binary segmentation identifies a single change point in a dataset using a CUSUM-like procedure and then employs a divide-and-conquer approach to find additional change points within sub-segments until a stopping condition is reached. However, as a greedy algorithm, binary segmentation can be less effective with multiple change points. Wild binary segmentation (WBS) [Fryzlewicz, 2014] and seeded binary segmentation (SeedBS) [Kovács et al., 2022] improve upon this by defining multiple segments to identify and aggregate potential change points. SeedBS additionally addresses the issue of overly long sub-segments in WBS that may contain several change points. SeedBS uses multiple layers of intervals, each with a fixed number of intervals of varying lengths and shifts, to enhance the search for change points. When comparing multiple candidates for the next change point, the narrowest-over-threshold (NOT) method [Baranowski et al., 2019] prioritizes narrower sub-segments. Built upon these ideas, we develop an effective algorithm to identify the change points that separate watermarked and non-watermarked sub-strings. The details are described in Algorithm 1 below.

Algorithm 1 SeedBS-NOT for change point detection in potentially partially watermarked texts

Require: Sequence of p -values { p i } i m =1 , decay parameter a ∈ [1 / 2 , 1) for SeedBS, threshold ζ for NOT.

Ensure: Locations of the change points.

Define

.

, m

]

for

1

I

= (0

1

/a do

2

, . . . , log

m

k

⌈

⌉

Number of intervals in the

k

Length of intervals in the

-th layer:

-th layer:

Shift of intervals in the

k

k

-th layer intervals:

I

end for

Define all seeded intervals for

i

do

1

, . . . ,

|I|

Define the

i

Define

-th interval

S

i

i

k

n

l

-th layer:

k

=

s

⋃

(

{

⌊

I

k

n

i

=1

=

log

⌈

k

= (

I

s

i

r

+1:

(

τ

Let

i

) := sup

ˆ

τ

Obtain

end for

Define the set of potential change point locations O = { i : ˜ p i < ζ } and the final set of change point locations S = ∅ .

while

=

O ̸

∅

do

Select

= arg min

i

S ← S ∪ {

i

=1

,...,

|O|

ˆ

i

τ

;

}

end while return

S

.

O ← {

i

I

|}

{|

j

≤ |O|

= arg min

: ˆ

τ

i

̸∈

I

j

}

.

1

=1

⋃

r

t

i

, s

(

,

k

= 2

k

k

(

i

/a

]

=

⌈

ma

= (

m

-

1)

m

.

τ

-

(

r

s

i

⌉

k

(1

/a

k

-

s

k

I

)(

i

i

[0

1]

∈

-

s

r

i

r

i

<τ

s

r

+1:

s

S

˜

p

i

i

i

i

.

(

τ

)

through block bootstrap (10).

.

i

-

)

3

τ

/

1

-

l

⌋

,

)

2

)

.

k

)

⌈

|

(

F

/

i

r

i

(

n

-

+1:

k

1

-

k

1)

⌉ -

-

1

1)

s

τ

(

k

t

.

.

+

)

-

k

l

F

τ

i

=1

,...,

|O|

{

s

i

-

r

⌉

.

]

}

+1:

s

i

}

.

i

(

t

)

|

.

▷ Start of SeedBS

▷ End of SeedBS ▷ Start of NOT

▷ End of NOT

0

2

4

6

8

0.001

0.005

0.01

0.05

Change Point Count

Setting 1, EMS

Setting 1, EMSL

Setting 1, ITS

Setting 1, ITSL

ITS

ITSL

EMS

EMSL

0

100

200

300

400

500

0

100

200

300

400

500

0.00

0.25

0.50

0.75

0.00

0.25

0.50

0.75

P-Value

Threshold

Token Index

4 Numerical experiments

We conduct extensive real-data-based experiments following a similar empirical setting in Kirchenbauer et al. [2023a], where we generate watermarked text based on the prompts sampled from the news-like subset of the colossal clean crawled corpus (C4) dataset [Raffel et al., 2020]. We utilized three LLMs, namely openai-community/gpt2 [Radford et al., 2019], facebook/opt-1.3b [Zhang et al., 2022] and Meta-Llama-3-8B [AI@Meta, 2024], to evaluate the effectiveness of the proposed method. We consider the following four watermark generation and detection methods:

ITS : The inverse transform sampling method with the dependence measure defined in (1).

ITSL : The inverse transform sampling method with the dependence measure defined in (B.2), which is based on the Levenshtein cost (B.1) with base alignment cost (B.3).

EMS : The exponential minimum sampling method with the dependence measure defined in (2).

EMSL : The exponential minimum sampling method with the dependence measure defined in (B.3), which is based on the Levenshtein cost (B.1) with base alignment cost (B.4).

The details of the Levenshtein cost are deferred to Appendix B.1. For each of the experiment settings, 100 prompts were used to generate the watermarked text. We fix the length of text m = 500 , the size of sliding window B = 20 , and the block size used in the block bootstrap-based test B ′ = 20 . Results for other choices of B are shown in Appendix C. In Algorithm 1, we set the decay parameter a = √ 2 and the minimum length of the intervals generated by SeedBS to be 50 such that the block bootstrapbased test is meaningful, and the threshold ζ ∈ { 0 . 05 , 0 . 01 , 0 . 005 , 0 . 001 } . We present the results for openai-community/gpt2 in the main text and defer the results for facebook/opt-1.3b and Meta-Llama-3-8B to Appendix B.3 and Appendix B.4, respectively.

4.1 False positive analysis

We begin by analyzing the false discoveries of the change point detection method. We will generate watermarked text with a length of m = 500 , where no change points exist.

Setting 1 (no change point): Generate 500 tokens with a watermark.

The results for openai-community/gpt2 are showed in Figure 1. The left panel illustrates that the two exponential minimum sampling methods result in fewer false discoveries compared to the two inverse transform sampling methods. Indeed, the existence of false discoveries highly depends on the quality of the obtained sequence of p -values. In the right panel, we fixed one prompt and plotted the sequence of p -values for the four methods. Most of the p -values are near 0 . However, certain sub-strings have relatively high p -values for the two inverse transform methods, indicating that these methods failed to detect the watermark in these segments, resulting in false discoveries for change point detection.

0.2

0.4

0.6

0.8

1.0

0.001

0.005

0.01

0.05

Threshold

Rand Index

Setting 2, EMS

Setting 2, EMSL

Setting 2, ITS

Setting 2, ITSL

Setting 3, EMS

Setting 3, EMSL

Setting 3, ITS

Setting 3, ITSL

Setting 4, EMS

Setting 4, EMSL

Setting 4, ITS

Setting 4, ITSL

4.2 Change point analysis

When users modify the text generated by LLM, there may be some sub-strings with watermarks and others without. Our goal is to accurately separate the text into watermarked and non-watermarked sub-strings. In this section, we will focus on two types of attacks: insertion and substitution. To demonstrate, we will consider the following three settings:

Setting 2 (insertion attack): Generate 250 tokens with watermarks, then append with 250 tokens without watermarks. In this setting, there is a single change point at the index 251 .

Setting 3 (substitution attack): Generate 500 tokens with watermarks, then substitute the token with indices ranging from 201 to 300 with non-watermarked text of length 100 . In this setting, there are two change points at the indices 201 and 301 , respectively.

Setting 4 (insertion and substitution attacks): Generate 400 tokens with watermarks, substitute the token with indices ranging from 101 to 200 with non-watermarked text of length 100 , and then insert 100 tokens without watermarks at the index 300 . In this setting, there are four change points located at the indices 101 , 201 , 301 , and 401 , respectively.

For more complex simulation settings, please refer to Appendix B.5.

We compare the clusters identified through the detected change points with the true clusters separated by the true change points using the Rand index [Rand, 1971]. A higher Rand index indicates better performance of different methods.

Figure 2 shows the Rand index for four methods in Settings 2-4 corresponding to different thresholds ζ . For each method, their performance in Settings 2-3 is better than that of Setting 4 when the threshold ζ ≤ 0 . 01 . This is because Setting 4 includes two types of attacks, making the problem more difficult than in Settings 2 and 3. In all cases, the two exponential minimum sampling methods outperform the two inverse transform sampling methods, and EMS delivers the highest Rand index value. We want to emphasize again that the performance of the change point detection method highly depends on the quality of the obtained sequence of p -values. Figure 3 shows the p -value sequence for all methods given one fixed prompt in Setting 4. The change points detected by the EMS and EMSL methods are closer to the true change points compared to those detected by the ITS and ITSL methods. Additionally, the sequence of p -values for all methods in Settings 1-4 with the first 10 prompts extracted Google C4 dataset are shown in Figure B.1 in the Appendix.

5 Discussions

In this study, we have introduced a method for detecting whether a text is generated using LLM through randomization tests. We have demonstrated that our method effectively controls Type I and Type II errors under appropriate assumptions. Additionally, we have developed a technique to partition the published text into watermarked and non-watermarked sub-strings by treating it as a change point detection problem. Our proposed method accurately identifies watermarked sub-strings by determining the locations of change points. Simulation results using real data indicate that the EMS method outperforms the other methods.

ITS

ITSL

EMS

EMSL

0

100

200

300

400

500

0

100

200

300

400

500

0.00

0.25

0.50

0.75

1.00

0.00

0.25

0.50

0.75

1.00

Token Index

P-Value

The performance of the segmentation algorithm depends crucially on the quality of the randomizationbased p -values from each sub-string. Intuitively, a more significant discrepancy between the p -value distributions under the null and alternative will lead to better segmentation results. Thus, a powerful watermark detection algorithm is crucial to the success of the segmentation procedure. Motivated by Condition (4), an interesting future direction is to develop an adaptive watermark generation and detection procedure where the LLM provider adaptively embeds the key according to NTP, and the detector uses a corresponding adaptive procedure to detect the watermark.

Another interesting direction to explore is extending the algorithm to handle scenarios where the published text combines watermarked texts from different LLMs with varying watermark generation schemes. In this case, the goal is to separate the texts into sub-strings from different LLMs. This scenario involves multiple sequences of keys from different LLMs, each producing a sequence of p -values and change points. Figuring out how to aggregate these results to separate different LLMs and user-modified texts would be an intriguing problem.

Acknowledgments and Disclosure of Funding

GL and XZ were supported by the National Institutes of Health under Grant R01GM144351. Part of the research was conducted using the Arseven Computing Cluster at the Department of Statistics, Texas A&M University.

References

Scott Aaronson. Watermarking of large language models, 2023. URL https://simons.berkeley. edu/talks/scott-aaronson-ut-austin-openai-2023-08-17 .

Sahar Abdelnabi and Mario Fritz. Adversarial watermarking transformer: Towards tracing text provenance with data hiding. In 2021 IEEE Symposium on Security and Privacy (SP) , pages 121-140. IEEE, 2021. URL https://doi.org/10.1109/SP40001.2021.00083 .

AI@Meta. Llama 3 model card. https://github.com/meta-llama/llama3/blob/main/ MODEL_CARD.md , 2024. URL https://github.com/meta-llama/llama3/blob/main/ MODEL_CARD.md .

Mikhail J Atallah, Victor Raskin, Michael Crogan, Christian Hempelmann, Florian Kerschbaum, Dina Mohamed, and Sanket Naik. Natural language watermarking: Design, analysis, and a proof-of-concept implementation. In Information Hiding: 4th International Workshop, IH 2001 Pittsburgh, PA, USA, April 25-27, 2001 Proceedings 4 , pages 185-200. Springer, 2001. URL https://doi.org/10.1007/3-540-45496-9_14 .

Rafal Baranowski, Yining Chen, and Piotr Fryzlewicz. Narrowest-Over-Threshold Detection of Multiple Change Points and Change-Point-Like Features. Journal of the Royal Statistical Society Series B: Statistical Methodology , 81(3):649-672, 05 2019. ISSN 1369-7412. doi: 10.1111/rssb. 12322. URL https://doi.org/10.1111/rssb.12322 .

Zhongze Cai, Shang Liu, Hanzhao Wang, Huaiyang Zhong, and Xiaocheng Li. Towards better statistical understanding of watermarking llms. arXiv preprint arXiv:2403.13027 , 2024. URL https://doi.org/10.48550/arXiv.2403.13027 .

Edward Carlstein. Nonparametric change-point estimation. The Annals of Statistics , 16(1):188-197, 1988. URL https://doi.org/10.1214/aos/1176350699 .

Shubhadeep Chakraborty and Xianyang Zhang. High-dimensional change-point detection using generalized homogeneity metrics. arXiv preprint arXiv:2105.08976 , 2021. URL https://doi. org/10.48550/arXiv.2105.08976 .

Hao Chen and Nancy Zhang. Graph-based change-point detection. Annals of Statistics , 2015. URL https://doi.org/10.1214/14-AOS1269 .

Pierre Fernandez, Antoine Chaffin, Karim Tit, Vivien Chappelier, and Teddy Furon. Three bricks to consolidate watermarks for large language models. In 2023 IEEE International Workshop on Information Forensics and Security (WIFS) , pages 1-6. IEEE, 2023. URL https://doi.org/ 10.48550/arXiv.2308.00113 .

Piotr Fryzlewicz. Wild binary segmentation for multiple change-point detection. The Annals of Statistics , 42(6):2243 - 2281, 2014. doi: 10.1214/14-AOS1245. URL https://doi.org/10. 1214/14-AOS1245 .

Zaıd Harchaoui and Céline Lévy-Leduc. Multiple change-point estimation with a total variation penalty. Journal of the American Statistical Association , 105(492):1480-1493, 2010.

Nicholas Hopper, David Molnar, and David Wagner. From weak to strong watermarking. In Theory of Cryptography: 4th Theory of Cryptography Conference, TCC 2007, Amsterdam, The Netherlands, February 21-24, 2007. Proceedings 4 , pages 362-382. Springer, 2007. URL https: //doi.org/10.1007/978-3-540-70936-7_20 .

Zhengmian Hu, Lichang Chen, Xidong Wu, Yihan Wu, Hongyang Zhang, and Heng Huang. Unbiased watermark for large language models. arXiv preprint arXiv:2310.10669 , 2023. URL https: //doi.org/10.48550/arXiv.2310.10669 .

Baihe Huang, Banghua Zhu, Hanlin Zhu, Jason D Lee, Jiantao Jiao, and Michael I Jordan. Towards optimal statistical watermarking. arXiv preprint arXiv:2312.07930 , 2023. URL https://doi. org/10.48550/arXiv.2312.07930 .

John Kirchenbauer, Jonas Geiping, Yuxin Wen, Jonathan Katz, Ian Miers, and Tom Goldstein. A watermark for large language models. In Andreas Krause, Emma Brunskill, Kyunghyun Cho, Barbara Engelhardt, Sivan Sabato, and Jonathan Scarlett, editors, Proceedings of the 40th International Conference on Machine Learning , volume 202 of Proceedings of Machine Learning Research , pages 17061-17084. PMLR, 23-29 Jul 2023a. URL https://proceedings.mlr. press/v202/kirchenbauer23a.html .

John Kirchenbauer, Jonas Geiping, Yuxin Wen, Manli Shu, Khalid Saifullah, Kezhi Kong, Kasun Fernando, Aniruddha Saha, Micah Goldblum, and Tom Goldstein. On the reliability of watermarks for large language models. arXiv preprint arXiv:2306.04634 , 2023b. URL https://doi.org/ 10.48550/arXiv.2306.04634 .

S Kovács, P Bühlmann, H Li, and A Munk. Seeded binary segmentation: a general methodology for fast and optimal changepoint detection. Biometrika , 110(1):249-256, 10 2022. ISSN 1464-3510. doi: 10.1093/biomet/asac052. URL https://doi.org/10.1093/biomet/asac052 .

Rohith Kuditipudi, John Thickstun, Tatsunori Hashimoto, and Percy Liang. Robust distortionfree watermarks for language models. arXiv preprint arXiv:2307.15593 , 2023. URL https: //doi.org/10.48550/arXiv.2307.15593 .

Hans R Kunsch. The jackknife and the bootstrap for general stationary observations. The annals of Statistics , pages 1217-1241, 1989. URL https://doi.org/10.1214/aos/1176347265 .

Soumendra N Lahiri. Theoretical comparisons of block bootstrap methods. Annals of Statistics , pages 386-404, 1999.

Xiang Li, Feng Ruan, Huiyuan Wang, Qi Long, and Weijie J Su. A statistical framework of watermarks for large language models: Pivot, detection efficiency and optimal rules. arXiv preprint arXiv:2404.01245 , 2024. URL https://doi.org/10.48550/arXiv.2404.01245 .

Xingchi Li and Xianyang Zhang. fastcpd: Fast change point detection in r. arXiv preprint arXiv:2404.05933 , 2024. URL https://doi.org/10.48550/arXiv.2404.05933 .

Regina Y Liu, Kesar Singh, et al. Moving blocks jackknife and bootstrap capture weak dependence. Exploring the limits of bootstrap , 225:248, 1992. URL https://www.wiley.com/en-us/ Exploring+the+Limits+of+Bootstrap-p-9780471536314 .

Yepeng Liu and Yuheng Bu. Adaptive text watermark for large language models. arXiv preprint arXiv:2401.13927 , 2024. URL https://doi.org/10.48550/arXiv.2401.13927 .

David S Matteson and Nicholas A James. A nonparametric approach for multiple change point analysis of multivariate data. Journal of the American Statistical Association , 109(505):334-345, 2014. URL https://doi.org/10.1080/01621459.2013.849605 .

Travis Munyer and Xin Zhong. Deeptextmark: Deep learning based text watermarking for detection of large language model generated text. arXiv preprint arXiv:2305.05773 , 2023. URL https: //doi.org/10.48550/arXiv.2305.05773 .

Fearnhead R. Killick and I. A. Eckley. Optimal detection of changepoints with a linear computational cost. Journal of the American Statistical Association , 107(500):1590-1598, 2012. doi: 10.1080/01621459.2012.737745. URL https://doi.org/10.1080/01621459.2012.737745 .

Alec Radford, Jeffrey Wu, Rewon Child, David Luan, Dario Amodei, Ilya Sutskever, et al. Language models are unsupervised multitask learners. OpenAI blog , 1(8):9, 2019. URL https://openai. com/index/better-language-models/ .

Colin Raffel, Noam Shazeer, Adam Roberts, Katherine Lee, Sharan Narang, Michael Matena, Yanqi Zhou, Wei Li, and Peter J. Liu. Exploring the limits of transfer learning with a unified text-to-text transformer. Journal of Machine Learning Research , 21(140):1-67, 2020. URL http://jmlr.org/papers/v21/20-074.html .

William M. Rand. Objective criteria for the evaluation of clustering methods. Journal of the American Statistical Association , 66(336):846-850, 1971. doi: 10.1080/01621459.1971.10482356. URL https://www.tandfonline.com/doi/abs/10.1080/01621459.1971.10482356 .

Sidney Resnick. A probability path . Springer, 2019. URL https://doi.org/10.1007/ 978-0-8176-8409-9 .

Charles Truong, Laurent Oudre, and Nicolas Vayatis. Selective review of offline change point detection methods. Signal Processing , 167:107299, 2020. ISSN 0165-1684. doi: https://doi.org/ 10.1016/j.sigpro.2019.107299. URL https://www.sciencedirect.com/science/article/ pii/S0165168419303494 .

Lyudmila Yur'evna Vostrikova. Detecting 'disorder' in multidimensional random processes. In Doklady akademii nauk , volume 259, pages 270-274. Russian Academy of Sciences, 1981. URL https://www.mathnet.ru/eng/dan44582 .

Yihan Wu, Zhengmian Hu, Hongyang Zhang, and Heng Huang. Dipmark: A stealthy, efficient and resilient watermark for large language models. arXiv preprint arXiv:2310.07710 , 2023. URL https://doi.org/10.48550/arXiv.2310.07710 .

Susan Zhang, Stephen Roller, Naman Goyal, Mikel Artetxe, Moya Chen, Shuohui Chen, Christopher Dewan, Mona Diab, Xian Li, Xi Victoria Lin, Todor Mihaylov, Myle Ott, Sam Shleifer, Kurt Shuster, Daniel Simig, Punit Singh Koura, Anjali Sridhar, Tianlu Wang, and Luke Zettlemoyer. Opt: Open pre-trained transformer language models, 2022. URL https://doi.org/10.48550/ arXiv.2205.01068 .

Xianyang Zhang and Trisha Dawn. Sequential gradient descent and quasi-newton's method for change-point analysis. In International Conference on Artificial Intelligence and Statistics , pages 1129-1143. PMLR, 2023.

Xuandong Zhao, Yu-Xiang Wang, and Lei Li. Protecting language generation models via invisible watermarking. In International Conference on Machine Learning , pages 42187-42199. PMLR, 2023. URL https://doi.org/10.48550/arXiv.2302.03162 .

Xuandong Zhao, Lei Li, and Yu-Xiang Wang. Permute-and-flip: An optimally robust and watermarkable decoder for llms. arXiv preprint arXiv:2402.05864 , 2024. URL https://doi.org/10. 48550/arXiv.2402.05864 .

Appendix

A Proofs of the main results

Proof of Theorem 1.

(i) For simplicity of notation, denote φ := ϕ ( ξ 1: n , ˜ y 1: m ) and φ ( t ) := ϕ ( ξ ( t ) 1: n , ˜ y 1: m ) for t = 1 , · · · , T . Under the null hypothesis, we know that ξ 1: n is independent of ˜ y 1: m , which implies that the pairs ( ξ 1: n , ˜ y 1: m ) , ( ξ (1) 1: n , ˜ y 1: m ) , . . . , ( ξ ( T ) 1: n , ˜ y 1: m ) follow the same distribution. Hence φ, φ (1) , . . . , φ ( T ) are exchangeable. The exchangeability ensures that the rank of φ relative to { φ, φ (1) , . . . , φ ( t ) } is uniformly distributed. Denote the order statistics φ (1) ≤ · · · ≤ φ ( T +1) . Then we have

Hence, for j = 1 , . . . , T +1 , we have

Then, we have P ( p T ≤ α ) = ⌊ ( T +1) α ⌋ / ( T +1) ≤ α . (ii) Denote E ξ ′ := E [ ϕ ( ξ ′ 1: n , ˜ y 1: m |F m )] . By Chebyshev's inequality and Condition (a), we get

for all ϵ ≥ 0 . Thus ϕ ( ξ ′ 1: n , ˜ y 1: m ) -E ξ ′ = O p ( n -1 / 2 ) , which together with Condition (b) implies that ϕ ( ξ ′ 1: n , ˜ y 1: m ) = O p ( n -1 / 2 ) given F m .

Denote the distribution of ϕ ( ξ ′ 1: n , ˜ y 1: m ) conditional on F m by F , and the empirical distribution of { φ ( t ) } T t =0 by F T , where we set φ (0) = φ . Let q 1 -α,T = φ ( T +2 -j α ) with j α = ⌊ ( T +1) α ⌋ . Note that F T ( q 1 -α,T ) = 1 -( j α -1) / ( T + 1) . Our test rejects the null whenever p T ≤ α , which is equivalent to rejecting the null if φ ≥ φ ( T +2 -j α ) . By the Dvoretzky-Kiefer-Wolfowitz inequality, we have

for some constant C 1 > 0 , which implies that with probability greater than 1 -C 1 exp( -2 Tϵ 2 ) , F ( q 1 -α,T ) < 1 -( j α -1) / ( T +1)+2 ϵ . Define F -1 ( t ) = inf { s : F ( s ) ≥ t } and the event A T = { q 1 -α,T < F -1 (1 -( j α -1) / ( T +1)+2 ϵ ) } . Then we have P ( A T |F m ) ≥ 1 -C 1 exp( -2 Tϵ 2 ) . In addition, as ϕ ( ξ ′ 1: n , ˜ y 1: m ) = O p ( n -1 / 2 ) , we have F -1 ( s ) = O ( n -1 / 2 ) for any s < 1 . By Condition (a), we have √ n ( ϕ ( ξ 1: n , ˜ y 1: m ) -E [ ϕ ( ξ 1: n , ˜ y 1: m ) |F m ] ) = O p (1) . Hence, for T > 2 /ϵ -1 ,

where we have used Condition (c) and the fact that √ nF -1 (1 -α + 3 ϵ ) = O (1) to get the convergence.

We state the following Lemma, which is useful for the proof of Corollary 1.

Lemma 1. { ξ i } n i =1 are conditionally independent given y -n 0 : n .

Proof of Lemma 1. Recall that y i = Γ( ξ i , p ( ·| y -n 0 : i -1 )) = Γ i ( ξ i ) , where Γ i depends on y -n 0 : i -1 . We note that

Hence the conditional distribution of ξ 1: n given y -n 0 : n is equal to

which implies that { ξ i } n i =1 are conditionally independent given y -n 0 : n .

Proof of Corollary 1.

(i) Recall in Example 1 that the test statistic is given by:

where ξ i = ( u i , π i ) . Wefirst note that h i ( ξ i , y i ) is bounded and thus Condition (a) of Theorem 1 holds. Since ξ ′ 1: n is independent of y -n 0 : n , we have u ′ i | y 1: n ∼ Unif [0 , 1] . Thus, E [ ϕ ( ξ ′ 1: n , y 1: n ) | y 1: n ] = 0 , which implies that Condition (b) of Theorem 1 is fulfilled.

Denote µ i ( · ) = p ( ·| y -n 0 : i -1 ) . Conditional on y -n 0 : i and π i ( y i ) , we know that u i follows the uniform distribution over the interval [ µ i ( y : π i ( y ) < π i ( y i )) , µ i ( y : π i ( y ) ≤ π i ( y i ))] . As a result, we can calculate the expected value of u i given y -n 0 : i and π i ( y i ) as

where the third equality is because given π i ( y i ) = k , π i follows the uniform distribution over the permutation space with the restriction π i ( y i ) = k . Then, we have

Given y i , ( π i ( y i ) -1) / ( V -1) follows the uniform distribution over the discrete space { 0 , 1 / ( V -1) , . . . , 1 } . Thus we have E [( π i ( y i ) -1) / ( V -1)] = 1 / 2 and Var(( π i ( y i ) -1) / ( V -1)) = C , which is a constant less than 1 / 12 (i.e., the variance of Unif [0 , 1] ). Hence, E [ ϕ ( ξ 1: n , y 1: n )] = Cn -1 ∑ n i =1 ( 1 -p ( y i | y 1: i -1 ) ) and Condition (c) of Theorem 1 is satisfied due to (4).

(ii) In Example 2, the test statistic is given by

Observe that E ik := -log( ξ ik ) /p ( k | y -n 0 : i -1 ) ∼ Exp ( p ( k | y -n 0 : i -1 )) . Since ξ ′ 1: n is independent of y 1: n , we have -log( ξ ′ i,y i ) | y -n 0 : n ∼ Exp (1) . Hence, conditional on y -n 0 : n , we have

Condition (b) of Theorem 1 is satisfied.

Given y -n 0 : n , we know E i,y i = min 1 ≤ k ≤ V E ik , which implies -log( ξ i,y i ) /p ( y i | y -n 0 : i -1 ) | y -n 0 : n ∼ Exp (1) . It is worth noting that

That is -log( ξ i,y i ) | y -n 0 : n ∼ Exp (1 /p ( y i | y -n 0 : i -1 )) . According to Lemma 1, ξ i s are conditionally independent given y -n 0 : n . Thus, we have

Therefore, Conditions (a) and (c) of Theorem 1 are satisfied.

Proof of Theorem 2. Weonly prove the result under the setting of Example 1 as the proof for Example 2 is similar with the help of Bernstein's inequality. Because | h i | ≤ 1 / 4 , by Hoeffding's inequality, we have

By the union bound, we have

Integrating out the strings in y 1: n that are not contained in ˜ y 1: m , we obtain

where F m = [ ˜ y 1: m , y -n 0 :0 ] . Thus, conditional on F m , max a,b {| E [ M ( ξ a : a + B -1 , ˜ y b : b + B -1 ) |F m ] -M ( ξ a : a + B -1 , ˜ y b : b + B -1 ) |} = O ( C N,B ) . Note that

On the other hand, for a randomly generated key ξ ′ 1: n , E [ M ( ξ ′ a : a + B -1 , ˜ y b : b + B -1 ) |F m ] = 0 for all a, b . By the same argument, we get

which suggests that F -1 ( s ) = O ( C N,B ) with F being the distribution of ϕ ( ξ ′ 1: n , ˜ y 1: m ) conditional on F m and F -1 ( t ) = inf { s : F ( s ) ≥ t } . The rest of the arguments are similar to those in the proof of Theorem 1. We skip the details.

Proof of Proposition 1. Note that conditional on F m , the p -value sequence is B -dependent in the sense that p i and p j are independent only if | i -j | > B . Under the assumption on B , we have

for | b -a | ≍ m, which implies that F a +1: b ( t ) -E [ F a +1: b ( t )] → p 0 for any given t ∈ [0 , 1] . Using similar arguments as in the proof of Theorem 7.5.2 of Resnick [2019], we can strengthen the result to allow uniform convergence over t ∈ [0 , 1] :

Therefore, we obtain

Proof of Theorem 3. We begin the proof by noting that

Let us focus on the case where ˆ τ ≤ τ ∗ . The other case where ˆ τ > τ ∗ can be proved in a similar way. By the definition of ˆ τ , we have

For any 1 ≤ a < b ≤ m and there is no change point between [ a, b ] , define W a : b ( t ) = ∑ b i = a 1 { p i ≤ t } and ¯ W a : b ( t ) = ∑ b i = a { 1 { p i ≤ t } -E [ F a : b ( t )] } . For any τ ≤ τ ∗ , we have

where R ( τ ) is a reminder term satisfying that

Hence, we have

which implies that

Notice that

It remains to analyze sup τ ≤ m sup t ∈ [0 , 1] ∣ ∣ ¯ W 1: τ ( t ) ∣ ∣ . We first present a result that slightly modifies the Ottaviani's inequality. For the ease of notation, we write ∥ ¯ W 1: τ ∥ = sup t ∈ [0 , 1] ∣ ∣ ¯ W 1: τ ( t ) ∣ ∣ . For any u > 0 and v > B , we have

To see this, let A k be the event that ∥ ¯ W 1: k ∥ is the first ∥ ¯ W 1: j ∥ (for j = 1 , 2 , . . . , m ) that is strictly greater than u + v . The event on the LHS is the disjoint union of A 1 , . . . , A m . Note that ∥ ¯ W k +1+ B : m ∥ is independent of ∥ ¯ W 1 ∥ , . . . , ∥ ¯ W k ∥ (and hence is independent of A k ). We have

where we have usede the fact ∥ ¯ W k +1: m ∥ ≤ ∥ ¯ W k +1: k + B ∥ + ∥ ¯ W k + B +1: m ∥ ≤ u + B . Summing over k leads to

which gives the desired result.

Next, we study P ( ∥ ¯ W τ +1: m ∥ > u ) for any τ = 0 , . . . , m -1 . Note that P ( ∥ ¯ W τ +1: m ∥ > u ) ≤ P ( ∥ ¯ W τ +1: τ ∗ ∥ > u/ 2) + P ( ∥ ¯ W τ ∗ +1: m ∥ > u/ 2) . Thus, without loss of generality, let us focus our analysis on the probability P ( ∥ ¯ W a +1: b ∥ > u ) , where the corresponding segment does not contain a change point. Assume b -a = 2 KB . (Notice that for general a < b , we have an additional interval whose length is smaller than B . Hence, a similar analysis below can be used.) We divide the index set { a +1 , a +2 , . . . , b } into 2 K consecutive blocks, denoted by J 1 , . . . , J 2 K , with equal block size B . Then we have

with ¯ F J i = ¯ W J i /B , where ∑ K i =1 ¯ F J 2 i -1 and ∑ K i =1 ¯ F J 2 i are both sums of independent bounded random variables. Let us analyze the second term on the RHS. Define

Let t v,L = G -1 ( v/L ) for v = 1 , 2 , . . . , L . Following the argument in Theorem 7.5.2 of Resnick [2019], we can show that

where G m ( t -) = ∑ K i =1 ∑ i ∈ J 2 i 1 { p i < t } / ( KB ) and G ( t -) is defined similarly. Thus we have

where the third inequality follows from Hoeffding's inequality. For any ϵ > 0 , we can set u = C 3 ( b -a ) √ log( K ) / √ K = 2 C 3 B √ K log( K ) and L = √ K for some large enough C 3 such that

Now back to (A.2), set u = C 4 √ mB log( m/B ) and v = C 5 √ mB log( m/B ) . We can make the RHS of (A.2) arbitrarily small with large enough C 4 and C 5 . It thus gives max τ ≤ m ∥ ¯ S 1: τ ∥ = O ( √ mB log( m/B )) and sup τ | R ( τ ) | = O ( √ B log( m/B ) /m ) . Hence, we deduce that

A similar argument applies to the other direction, which gives

B Additional numerical results

B.1 The Levenshtein cost

Recall that we use V to denote the vocabulary and Ξ to represent the space of watermark keys. Let V ∗ be the space of strings, where ∗ can be any positive integer; for example, ˜ y 1: m ∈ V m . Similarly, we define Ξ ∗ as the space of watermark key sequences. Given a string ˜ y , let ˜ y a : be the string from the a th token to the end; for example, if ˜ y = ˜ y 1: m , then ˜ y 2: = ˜ y 2: m . Denote the length of a string ˜ y as len ( ˜ y ) .

Definition 1 (Simple Levenshtein cost) . Let γ ∈ R and base alignment cost d 0 : V × Ξ → R . Given a string ˜ y ∈ V ∗ and a watermark key sequence ξ ∈ Ξ ∗ , the simple Levenshtein cost d γ ( ˜ y, ξ ) is defined by

with d γ ( ˜ y, ξ ) := γ · len ( ˜ y ) if ξ is empty and d γ ( ˜ y, ξ ) := γ · len ( ξ ) if ˜ y is empty.

We use the following metric to quantify the dependence between the watermark key and the string:

in inverse transform sampling method, while

in exponential minimum sampling method.

B.2 Sequences of p -values from OpenAI-Community/GPT2

Figure B.1 presents the p -values for 500 text tokens generated from 10 prompts. The language model used is openai-community/gpt2 obtained from https://huggingface.co/ openai-community/gpt2 . The p -value sequences are organized into four groups, namely four settings corresponding to the numerical experiments section in the main paper (Section 4):

Setting 1 (no change point): Generate 500 tokens with a watermark.

Setting 2 (insertion attack): Generate 250 tokens with watermarks, then append with 250 tokens without watermarks. In this setting, there is a single change point at the index 251 .

Setting 3 (substitution attack): Generate 500 tokens with watermarks, then substitute the token with indices ranging from 201 to 300 with non-watermarked text of length 100. In this setting, there are two change points at the indices 201 and 301 .

Setting 4 (insertion and substitution attacks): Generate 400 tokens with watermarks, substitute the token with indices ranging from 101 to 200 with non-watermarked text of length 100, and then insert 100 tokens without watermarks at the index 300 . In this setting, there are four change points located at the indices 101 , 201 , 301 , and 401 .

Within each group, the rows represent p -values calculated using four different distance metrics: EMS , EMSL , ITS , and ITSL . It is easy to see that EMS performs the best, followed by EMSL .

B.3 Results for Facebook/OPT-1.3b

Figure B.2 shows the results for false discoveries under Setting 1, where the text tokens are all watermarked and generated with facebook/opt-1.3b . A lower threshold leads to fewer false discoveries.

Figure B.4 shows the boxplots of the Rand index for the four methods under different settings. EMS demonstrates the best performance across all settings, achieving its best performance at a threshold of ζ = 0 . 005 .

where γ = 0 . 4 , and

Prompt 1

Prompt 2

Prompt 3

Prompt 4

Prompt 5

Prompt 6

Prompt 7

Prompt 8

Prompt 9

Prompt 10

EMS

Setting 1

EMSL

Setting 1

ITS

Setting 1

ITSL

Setting 1

EMS

Setting 2

EMSL

Setting 2

ITS

Setting 2

ITSL

Setting 2

EMS

Setting 3

EMSL

Setting 3

ITS

Setting 3

ITSL

Setting 3

EMS

Setting 4

EMSL

Setting 4

ITS

Setting 4

ITSL

Setting 4

0

100

200

300

400

500

0

100

200

300

400

500

0

100

200

300

400

500

0

100

200

300

400

500

0

100

200

300

400

500

0

100

200

300

400

500

0

100

200

300

400

500

0

100

200

300

400

500

0

100

200

300

400

500

0

100

200

300

400

500

0.00

0.25

0.50

0.75

1.00

0.00

0.25

0.50

0.75

1.00

0.00

0.25

0.50

0.75

1.00

0.00

0.25

0.50

0.75

1.00

0.00

0.25

0.50

0.75

1.00

0.00

0.25

0.50

0.75

1.00

0.00

0.25

0.50

0.75

1.00

0.00

0.25

0.50

0.75

1.00

0.00

0.25

0.50

0.75

1.00

0.00

0.25

0.50

0.75

1.00

0.00

0.25

0.50

0.75

1.00

0.00

0.25

0.50

0.75

1.00

0.00

0.25

0.50

0.75

1.00

0.00

0.25

0.50

0.75

1.00

0.00

0.25

0.50

0.75

1.00

0.00

0.25

0.50

0.75

1.00

Token Index

P-Value

0.0

2.5

5.0

7.5

0.001

0.005

0.01

0.05

Threshold

Change Point Count

Setting 1, EMS

Setting 1, EMSL

Setting 1, ITS

Setting 1, ITSL

Prompt 1

Prompt 2

Prompt 3

Prompt 4

Prompt 5

Prompt 6

Prompt 7

Prompt 8

Prompt 9

Prompt 10

EMS

Setting 1

EMSL

Setting 1

ITS

Setting 1

ITSL

Setting 1

EMS

Setting 2

EMSL

Setting 2

ITS

Setting 2

ITSL

Setting 2

EMS

Setting 3

EMSL

Setting 3

ITS

Setting 3

ITSL

Setting 3

EMS

Setting 4

EMSL

Setting 4

ITS

Setting 4

ITSL

Setting 4

0

100

200

300

400

500

0

100

200

300

400

500

0

100

200

300

400

500

0

100

200

300

400

500

0

100

200

300

400

500

0

100

200

300

400

500

0

100

200

300

400

500

0

100

200

300

400

500

0

100

200

300

400

500

0

100

200

300

400

500

0.00

0.25

0.50

0.75

1.00

0.00

0.25

0.50

0.75

1.00

0.00

0.25

0.50

0.75

1.00

0.00

0.25

0.50

0.75

1.00

0.00

0.25

0.50

0.75

1.00

0.00

0.25

0.50

0.75

1.00

0.00

0.25

0.50

0.75

1.00

0.00

0.25

0.50

0.75

1.00

0.00

0.25

0.50

0.75

1.00

0.00

0.25

0.50

0.75

1.00

0.00

0.25

0.50

0.75

1.00

0.00

0.25

0.50

0.75

1.00

0.00

0.25

0.50

0.75

1.00

0.00

0.25

0.50

0.75

1.00

0.00

0.25

0.50

0.75

1.00

0.00

0.25

0.50

0.75

1.00

Token Index

P-Value

0.2

0.4

0.6

0.8

1.0

0.001

0.005

0.01

0.05

Threshold

Rand Index

Setting 2, EMS

Setting 2, EMSL

Setting 2, ITS

Setting 2, ITSL

Setting 3, EMS

Setting 3, EMSL

Setting 3, ITS

Setting 3, ITSL

Setting 4, EMS

Setting 4, EMSL

Setting 4, ITS

Setting 4, ITSL

The sequence of p -values from the first 10 prompts extracted from the Google C4 dataset in all settings is presented in Figure B.3. The p -value sequences are organized into four groups, corresponding to the four settings in the numerical experiments section of the main text. Within each group, each row corresponds to one method.

By comparing with the p -value sequences obtained using openai-community/gpt2 , we claim that the segmentation algorithm's performance depends on the quality of the p -values obtained for each sub-string and this relies on not only the watermark generation schemes but also the language models from which the texts are generated.

B.4 Results for meta-llama/Meta-Llama-3-8B

Figure B.6 shows the results for false discoveries under Setting 1, where the text tokens are all watermarked and generated with meta-llama/Meta-Llama-3-8B . A lower threshold leads to fewer false discoveries.

Prompt 1

Prompt 10

Prompt 2

Prompt 3

Prompt 4

Prompt 5

Prompt 6

Prompt 7

Prompt 8

Prompt 9

EMS

Setting 1

EMSL

Setting 1

ITS

Setting 1

ITSL

Setting 1

EMS

Setting 2

EMSL

Setting 2

ITS

Setting 2

ITSL

Setting 2

EMS

Setting 3

EMSL

Setting 3

ITS

Setting 3

ITSL

Setting 3

EMS

Setting 4

EMSL

Setting 4

ITS

Setting 4

ITSL

Setting 4

0

100 200 300 400 500

0

100 200 300 400 500

0

100 200 300 400 500

0

100 200 300 400 500

0

100 200 300 400 500

0

100 200 300 400 500

0

100 200 300 400 500

0

100 200 300 400 500

0

100 200 300 400 500

0

100 200 300 400 500

0.00

0.25

0.50

0.75

1.00

0.00

0.25

0.50

0.75

1.00

0.00

0.25

0.50

0.75

1.00

0.00

0.25

0.50

0.75

1.00

0.00

0.25

0.50

0.75

1.00

0.00

0.25

0.50

0.75

1.00

0.00

0.25

0.50

0.75

1.00

0.00

0.25

0.50

0.75

1.00

0.00

0.25

0.50

0.75

1.00

0.00

0.25

0.50

0.75

1.00

0.00

0.25

0.50

0.75

1.00

0.00

0.25

0.50

0.75

1.00

0.00

0.25

0.50

0.75

1.00

0.00

0.25

0.50

0.75

1.00

0.00

0.25

0.50

0.75

1.00

0.00

0.25

0.50

0.75

1.00

Index

P-Value

0

2

4

6

8

0.001

0.005

0.01

0.05

Threshold

Change Point Count

Setting 1, EMS

Setting 1, EMSL

Setting 1, ITS

Setting 1, ITSL

0.2

0.4

0.6

0.8

1.0

0.001

0.005

0.01

0.05

Threshold

Rand Index

Setting

Setting 2, EMS

Setting 2, EMSL

Setting 2, ITS

Setting 2, ITSL

Setting 3, EMS

Setting 3, EMSL

Setting 3, ITS

Setting 3, ITSL

Setting 4, EMS

Setting 4, EMSL

Setting 4, ITS

Setting 4, ITSL

B.5 Other settings

We focus on the Meta-Llama-3-8B model to conduct simulation studies under some more difficult scenarios. Specifically, we increase the number of change points to 4 , 8 , and 12 and vary the segment lengths accordingly. The results are presented in Figure B.8. For scenarios with 4 and 8 change points, the proposed method successfully identifies all change points. As the number of change points increases, the change point detection problem becomes much more challenging. In a scenario with 12 change points, our method was able to identify 9 of them, showing its robust performance in handling more difficult situations. Generally, the difficulty of a change point detection problem depends on the distance between the change points and the magnitudes of the changes, as indicated by the theoretical results in Proposition 1.

C Choices of the tuning parameters

We investigate the impact of the window size B and the number of permutations T on the proposed method. In particular, we focus on the EMS method in simulation Setting 4 of the main text, using the Meta-Llama-3-8B model.

0.00

0.25

0.50

0.75

1.00

0

100

200

300

400

500

Index

P-Value

Four change points

0.00

0.25

0.50

0.75

1.00

0

100

200

300

400

500

Index

P-Value

Eight change points

0.00

0.25

0.50

0.75

1.00

0

100

200

300

400

500

Index

P-Value

Twelve change points

First, to study the impact of the window size B , we fix T = 999 and vary the value of B in the set { 10 , 20 , 30 , 40 , 50 } . Table C.1 shows the rand index value for each setting, with a higher rand index indicating better performance.

It is crucial to select an appropriate value for B . If B is too small, the corresponding window may not contain enough data to reliably detect watermarks, as longer strings generally make the watermark more detectable. Conversely, if B is excessively large, it might prematurely shift the detected change point locations, thus reducing the rand index. For instance, let us consider a scenario with 200 tokens where the first 100 tokens are non-watermarked, and the subsequent 100 are watermarked, with the true change point at index 101. Assuming our detection test is highly effective, then it will yield a p-value uniformly distributed over [0 , 1] over a non-watermarked window and a p-value around zero over a window containing watermarked tokens. When B = 50 , the window beginning at the 76th token contains one watermarked token, which can lead to a small p-value and thus erroneously indicate a watermark from the 76th token onwards. In contrast, if B = 20 , the window starting at the 91st token will contain the first watermarked token, leading to a smaller error in identifying the change point location. The above phenomenon is the so-called edge effect, which will diminish as B gets smaller.

The trade-off in the choice of window size is recognized in the time series literature. For instance, a common recommendation for the window size in time series literature is to set B = Cn 1 / 3 , where n is the sample size, as seen in Corollary 1 of Lahiri [1999]. Based on our experience, setting B = ⌊ 3 n 1 / 3 ⌋ (for example, when n = 500 , B = 23 ) often results in good finite sample performance. A more thorough investigation of the choice of B is deferred to future research.

We next examine how our method is affected by the choice of the number of permutations T . To this end, we set B to 20 and consider T ∈ { 99 , 249 , 499 , 749 , 999 } . The results are summarized in Table C.2, including the rand index value and computation times for each setting. As expected, the computation time increases almost linearly with the number of permutations T . We also note that the rand index remains consistent across different values of T , indicating a level of stability in our method.

NeurIPS Paper Checklist

1. Claims

Question: Do the main claims made in the abstract and introduction accurately reflect the paper's contributions and scope?

Answer: [Yes]

Justification: We claim our contributions in the abstract and introduction. (i) We rigorously study the Type I and Type II errors of the randomization test for detecting the presence of a watermark. We then apply these findings to the inverse transform and Gumbel watermark schemes. (ii) We develop a systematic statistical method for segmenting texts into watermarked and non-watermarked sub-strings, and we investigate the theoretical and finite sample performance of this methodology, which has yet to be thoroughly explored in recent literature.

Guidelines:

The answer NA means that the abstract and introduction do not include the claims made in the paper.

The abstract and/or introduction should clearly state the claims made, including the contributions made in the paper and important assumptions and limitations. A No or NA answer to this question will not be perceived well by the reviewers.

The claims made should match theoretical and experimental results, and reflect how much the results can be expected to generalize to other settings.

It is fine to include aspirational goals as motivation as long as it is clear that these goals are not attained by the paper.

2. Limitations

Question: Does the paper discuss the limitations of the work performed by the authors?

Answer: [Yes]

Justification: We have emphasized that the segmentation algorithm's performance depends crucially on the quality of the simulation-based p -values from each sub-string. Therefore, a powerful watermark detection algorithm is essential for the segmentation procedure's success.

Guidelines:

The answer NA means that the paper has no limitation while the answer No means that the paper has limitations, but those are not discussed in the paper.

The authors are encouraged to create a separate "Limitations" section in their paper.

The paper should point out any strong assumptions and how robust the results are to violations of these assumptions (e.g., independence assumptions, noiseless settings, model well-specification, asymptotic approximations only holding locally). The authors should reflect on how these assumptions might be violated in practice and what the implications would be.

The authors should reflect on the scope of the claims made, e.g., if the approach was only tested on a few datasets or with a few runs. In general, empirical results often depend on implicit assumptions, which should be articulated.

The authors should reflect on the factors that influence the performance of the approach. For example, a facial recognition algorithm may perform poorly when image resolution is low or images are taken in low lighting. Or a speech-to-text system might not be used reliably to provide closed captions for online lectures because it fails to handle technical jargon.

The authors should discuss the computational efficiency of the proposed algorithms and how they scale with dataset size.

If applicable, the authors should discuss possible limitations of their approach to address problems of privacy and fairness.

While the authors might fear that complete honesty about limitations might be used by reviewers as grounds for rejection, a worse outcome might be that reviewers discover limitations that aren't acknowledged in the paper. The authors should use their best

judgment and recognize that individual actions in favor of transparency play an important role in developing norms that preserve the integrity of the community. Reviewers will be specifically instructed to not penalize honesty concerning limitations.

3. Theory Assumptions and Proofs

Question: For each theoretical result, does the paper provide the full set of assumptions and a complete (and correct) proof?

Answer: [Yes]

Justification: In the main text, we formally state essential assumptions for our theoretical results and provide rigorous proof in the appendix.

Guidelines:

The answer NA means that the paper does not include theoretical results.

All the theorems, formulas, and proofs in the paper should be numbered and crossreferenced.

All assumptions should be clearly stated or referenced in the statement of any theorems.

The proofs can either appear in the main paper or the supplemental material, but if they appear in the supplemental material, the authors are encouraged to provide a short proof sketch to provide intuition.

Inversely, any informal proof provided in the core of the paper should be complemented by formal proofs provided in appendix or supplemental material.

Theorems and Lemmas that the proof relies upon should be properly referenced.

4. Experimental Result Reproducibility

Question: Does the paper fully disclose all the information needed to reproduce the main experimental results of the paper to the extent that it affects the main claims and/or conclusions of the paper (regardless of whether the code and data are provided or not)?

Answer: [Yes]

Justification: We include all experimental details, such as LLM and hyperparameter values, in the main text and provide code for reproducibility in the supplementary materials.

Guidelines:

The answer NA means that the paper does not include experiments.

If the paper includes experiments, a No answer to this question will not be perceived well by the reviewers: Making the paper reproducible is important, regardless of whether the code and data are provided or not.

If the contribution is a dataset and/or model, the authors should describe the steps taken to make their results reproducible or verifiable.

Depending on the contribution, reproducibility can be accomplished in various ways. For example, if the contribution is a novel architecture, describing the architecture fully might suffice, or if the contribution is a specific model and empirical evaluation, it may be necessary to either make it possible for others to replicate the model with the same dataset, or provide access to the model. In general. releasing code and data is often one good way to accomplish this, but reproducibility can also be provided via detailed instructions for how to replicate the results, access to a hosted model (e.g., in the case of a large language model), releasing of a model checkpoint, or other means that are appropriate to the research performed.

While NeurIPS does not require releasing code, the conference does require all submissions to provide some reasonable avenue for reproducibility, which may depend on the nature of the contribution. For example

(a) If the contribution is primarily a new algorithm, the paper should make it clear how to reproduce that algorithm.

(b) If the contribution is primarily a new model architecture, the paper should describe the architecture clearly and fully.

(c) If the contribution is a new model (e.g., a large language model), then there should either be a way to access this model for reproducing the results or a way to reproduce the model (e.g., with an open-source dataset or instructions for how to construct the dataset).

(d) We recognize that reproducibility may be tricky in some cases, in which case authors are welcome to describe the particular way they provide for reproducibility. In the case of closed-source models, it may be that access to the model is limited in some way (e.g., to registered users), but it should be possible for other researchers to have some path to reproducing or verifying the results.

5. Open access to data and code

Question: Does the paper provide open access to the data and code, with sufficient instructions to faithfully reproduce the main experimental results, as described in supplemental material?

Answer: [Yes]

Justification: We have provided the code for reproducibility in the supplementary materials. The code has already been uploaded to GitHub. Due to the double-blind policy, the GitHub repository can't be accessed now, but we will make it open access if the paper is accepted.

Guidelines:

The answer NA means that paper does not include experiments requiring code.

Please see the NeurIPS code and data submission guidelines ( https://nips.cc/ public/guides/CodeSubmissionPolicy ) for more details.

While we encourage the release of code and data, we understand that this might not be possible, so 'No' is an acceptable answer. Papers cannot be rejected simply for not including code, unless this is central to the contribution (e.g., for a new open-source benchmark).

The instructions should contain the exact command and environment needed to run to reproduce the results. See the NeurIPS code and data submission guidelines ( https: //nips.cc/public/guides/CodeSubmissionPolicy ) for more details.

The authors should provide instructions on data access and preparation, including how to access the raw data, preprocessed data, intermediate data, and generated data, etc.

The authors should provide scripts to reproduce all experimental results for the new proposed method and baselines. If only a subset of experiments are reproducible, they should state which ones are omitted from the script and why.

At submission time, to preserve anonymity, the authors should release anonymized versions (if applicable).

Providing as much information as possible in supplemental material (appended to the paper) is recommended, but including URLs to data and code is permitted.

6. Experimental Setting/Details

Question: Does the paper specify all the training and test details (e.g., data splits, hyperparameters, how they were chosen, type of optimizer, etc.) necessary to understand the results?

Answer: [Yes]

Justification: We include all experimental details, such as LLM and hyperparameter values, in the main text and provide code for reproducibility in the supplementary materials.

Guidelines:

The answer NA means that the paper does not include experiments.

The experimental setting should be presented in the core of the paper to a level of detail that is necessary to appreciate the results and make sense of them.

The full details can be provided either with the code, in appendix, or as supplemental material.

7. Experiment Statistical Significance

Question: Does the paper report error bars suitably and correctly defined or other appropriate information about the statistical significance of the experiments?

Answer: [Yes]

Justification: We do not need to display the error bar in our experimental results; in other words, the experiment results don't involve an error bar. We do include boxplots as a way to show the performance of the algorithm. The statistical significance is obtained through bootstrap-based test for each and every single change point, and comparisons have been made between performances using different p -value thresholds.

Guidelines:

The answer NA means that the paper does not include experiments.

The authors should answer "Yes" if the results are accompanied by error bars, confidence intervals, or statistical significance tests, at least for the experiments that support the main claims of the paper.

The factors of variability that the error bars are capturing should be clearly stated (for example, train/test split, initialization, random drawing of some parameter, or overall run with given experimental conditions).

The method for calculating the error bars should be explained (closed form formula, call to a library function, bootstrap, etc.)

The assumptions made should be given (e.g., Normally distributed errors).

It should be clear whether the error bar is the standard deviation or the standard error of the mean.

It is OK to report 1-sigma error bars, but one should state it. The authors should preferably report a 2-sigma error bar than state that they have a 96% CI, if the hypothesis of Normality of errors is not verified.

For asymmetric distributions, the authors should be careful not to show in tables or figures symmetric error bars that would yield results that are out of range (e.g. negative error rates).

If error bars are reported in tables or plots, The authors should explain in the text how they were calculated and reference the corresponding figures or tables in the text.

8. Experiments Compute Resources

Question: For each experiment, does the paper provide sufficient information on the computer resources (type of compute workers, memory, time of execution) needed to reproduce the experiments?

Answer: [Yes]

Justification: We provide information on our computer resources in the supplementary materials, specifically, the accompanied README.md file in the reproducing materials.

Guidelines:

The answer NA means that the paper does not include experiments.

The paper should indicate the type of compute workers CPU or GPU, internal cluster, or cloud provider, including relevant memory and storage.

The paper should provide the amount of compute required for each of the individual experimental runs as well as estimate the total compute.

The paper should disclose whether the full research project required more compute than the experiments reported in the paper (e.g., preliminary or failed experiments that didn't make it into the paper).

9. Code Of Ethics

Question: Does the research conducted in the paper conform, in every respect, with the NeurIPS Code of Ethics https://neurips.cc/public/EthicsGuidelines ?

Answer: [Yes]

Justification: Our research conducted in the paper conforms with the NeurIPS Code of Ethics in every respect.

Guidelines:

The answer NA means that the authors have not reviewed the NeurIPS Code of Ethics.

If the authors answer No, they should explain the special circumstances that require a deviation from the Code of Ethics.

The authors should make sure to preserve anonymity (e.g., if there is a special consideration due to laws or regulations in their jurisdiction).

10. Broader Impacts

Question: Does the paper discuss both potential positive societal impacts and negative societal impacts of the work performed?

Answer: [Yes]

Justification: In the introduction, we emphasize the importance of distinguishing between human and LLM-produced texts. This is essential to prevent the spread of misleading information, improper use of LLM-based tools in education, model extraction attacks through distillation, and the contamination of training datasets for future language models.

Guidelines:

The answer NA means that there is no societal impact of the work performed.

If the authors answer NA or No, they should explain why their work has no societal impact or why the paper does not address societal impact.

Examples of negative societal impacts include potential malicious or unintended uses (e.g., disinformation, generating fake profiles, surveillance), fairness considerations (e.g., deployment of technologies that could make decisions that unfairly impact specific groups), privacy considerations, and security considerations.

The conference expects that many papers will be foundational research and not tied to particular applications, let alone deployments. However, if there is a direct path to any negative applications, the authors should point it out. For example, it is legitimate to point out that an improvement in the quality of generative models could be used to generate deepfakes for disinformation. On the other hand, it is not needed to point out that a generic algorithm for optimizing neural networks could enable people to train models that generate Deepfakes faster.

The authors should consider possible harms that could arise when the technology is being used as intended and functioning correctly, harms that could arise when the technology is being used as intended but gives incorrect results, and harms following from (intentional or unintentional) misuse of the technology.

If there are negative societal impacts, the authors could also discuss possible mitigation strategies (e.g., gated release of models, providing defenses in addition to attacks, mechanisms for monitoring misuse, mechanisms to monitor how a system learns from feedback over time, improving the efficiency and accessibility of ML).

11. Safeguards

Question: Does the paper describe safeguards that have been put in place for responsible release of data or models that have a high risk for misuse (e.g., pretrained language models, image generators, or scraped datasets)?

Answer: [NA]

Justification: The paper poses no such risks as no pretrained models are created. The models used in the paper are publicly available and peer-reviewed and thus have a minimum risk for misuse.

Guidelines:

The answer NA means that the paper poses no such risks.

Released models that have a high risk for misuse or dual-use should be released with necessary safeguards to allow for controlled use of the model, for example by requiring that users adhere to usage guidelines or restrictions to access the model or implementing safety filters.

Datasets that have been scraped from the Internet could pose safety risks. The authors should describe how they avoided releasing unsafe images.

We recognize that providing effective safeguards is challenging, and many papers do not require this, but we encourage authors to take this into account and make a best faith effort.

12. Licenses for existing assets

Question: Are the creators or original owners of assets (e.g., code, data, models), used in the paper, properly credited and are the license and terms of use explicitly mentioned and properly respected?

Answer: [Yes]

Justification: Two pretrained language models were used and properly cited, as well as the GNU Parallel used in the experiments.

Guidelines:

The answer NA means that the paper does not use existing assets.

The authors should cite the original paper that produced the code package or dataset.

The authors should state which version of the asset is used and, if possible, include a URL.

The name of the license (e.g., CC-BY 4.0) should be included for each asset.

For scraped data from a particular source (e.g., website), the copyright and terms of service of that source should be provided.

If assets are released, the license, copyright information, and terms of use in the package should be provided. For popular datasets, paperswithcode.com/datasets has curated licenses for some datasets. Their licensing guide can help determine the license of a dataset.

For existing datasets that are re-packaged, both the original license and the license of the derived asset (if it has changed) should be provided.

If this information is not available online, the authors are encouraged to reach out to the asset's creators.

13. New Assets

Question: Are new assets introduced in the paper well documented and is the documentation provided alongside the assets?

Answer: [Yes]

Justification: The code and algorithm are documented properly and provided alongside the code.

Guidelines:

The answer NA means that the paper does not release new assets.

Researchers should communicate the details of the dataset/code/model as part of their submissions via structured templates. This includes details about training, license, limitations, etc.

The paper should discuss whether and how consent was obtained from people whose asset is used.

At submission time, remember to anonymize your assets (if applicable). You can either create an anonymized URL or include an anonymized zip file.

14. Crowdsourcing and Research with Human Subjects

Question: For crowdsourcing experiments and research with human subjects, does the paper include the full text of instructions given to participants and screenshots, if applicable, as well as details about compensation (if any)?

Answer: [NA]

Justification: The paper does not involve crowdsourcing nor research with human subjects.

Guidelines:

The answer NA means that the paper does not involve crowdsourcing nor research with human subjects.

Including this information in the supplemental material is fine, but if the main contribution of the paper involves human subjects, then as much detail as possible should be included in the main paper.

According to the NeurIPS Code of Ethics, workers involved in data collection, curation, or other labor should be paid at least the minimum wage in the country of the data collector.

15. Institutional Review Board (IRB) Approvals or Equivalent for Research with Human Subjects

Question: Does the paper describe potential risks incurred by study participants, whether such risks were disclosed to the subjects, and whether Institutional Review Board (IRB) approvals (or an equivalent approval/review based on the requirements of your country or institution) were obtained?

Answer: [NA]

Justification: The paper does not involve crowdsourcing nor research with human subjects.

Guidelines:

The answer NA means that the paper does not involve crowdsourcing nor research with human subjects.

Depending on the country in which research is conducted, IRB approval (or equivalent) may be required for any human subjects research. If you obtained IRB approval, you should clearly state this in the paper.

We recognize that the procedures for this may vary significantly between institutions and locations, and we expect authors to adhere to the NeurIPS Code of Ethics and the guidelines for their institution.

For initial submissions, do not include any information that would break anonymity (if applicable), such as the institution conducting the review.

Peer reviews:
{"IS_META_REVIEW": false, "comments": "SUMMARY:\nThe work makes two contributions: (1) A randomized-based test, including a theoretical analysis of a substring-based watermark detection for two popular distribution-preserving watermarks. The key idea is similar to the base detection methods in their respective base papers but shows some additional theoretical properties (e.g., convergence). (2) Based on the substring-based detection, an approach to segment the given text into both watermarked and non-watermarked sections by detecting points of change in the observed p-values on substrings. The proposed algorithm SeedBS-NOT is then evaluated in 4 settings and 2 models.\n\nSTRENGTHS:\n- The idea of differentiating between sections of text that are watermarked and the ones that are not is interesting and has many potential downstream applications (such as adaptive attacks). \n- The theoretical analysis seems sound and does provide asymptotical guarantees that are new to the reviewer.\n- In the evaluations, the change-points of watermarked and not-watermarked texts can be detected quite accurately.\n\nWEAKNESSES:\n- The method seems to rely heavily on the hyper-parameter B (substring block size) and would most likely profit from some ablation on the parameter in practical settings.\n- The introduced scenarios for evaluation feel very clean, with large gaps between change points. However, I do not believe this accurately reflects real-world perturbations in the watermarked text.\n- Similarly the evaluation is only one two quite oudated LLMs (gpt2 being 5 years old now) despite the paper claiming that the segementation algorithms performance \"depends [on]  the language model with which the texts are generated\". Given this I believe an extended evaluation on more realistic models (that people would actually watermark) is important to evaluate the empirical effectiveness of the approach (i.e., at least 7B -  the reviewer found in their own work that there can be significant changes between larger models when it comes to watermark detection).\n\n## Minor \n\n- The methods and analysis is restricted two specific types of watermarks, which are generally praised for their theoretical guarantees, but arguably not as popular as red-green type schemes for which such segmentations would also be interesting. Notably practical implementations of these schemes can, e.g., contain repeating key patterns whose effect would be interesting to investigate/mention in this context.\n- Given this the results too me only feel partially encouraging in that we (despite theoretical asymptotic error bounds) do observe some false-positives in some of the settings (which qualitatively is confirmed by, e.g., Fig. B.1 where some of the results tend to be quite noisy especially for ITS(L)).\n\n## Typos/Nits\n\n- L.27 Large Language Model\n- L.291 400 -> 500. According to the plots and the fact that there are 4 change-points it should be 500 tokens (same in L. 579)\n\nQUESTIONS:\n- How does the proposed method fare on more realistic text scenarios with shorter sections of watermarked / non-watermarked texts?\n- Can you provide a practical ablation over B (both w.r.t. runtime and effectiveness of the detection)?\n- Can you provide some experiments on larger and more realistic models (As well as the costs associated with running tests on such models).\n- Do you have any ideas about the effect of repeating key-patterns (which happens in practice) on your analysis?\n\nLIMITATIONS:\nThe paper addresses several limitations; however, as pointed out above, I think some more limitations of the current setting could be discussed.", "TITLE": "Official Review", "RECOMMENDATION": 6, "REVIEWER_CONFIDENCE": 3, "SOUNDNESS_CORRECTNESS": 3, "DATE": "2024-07-08"}

{"IS_META_REVIEW": false, "comments": "SUMMARY:\nThis paper presents a statistical method for detecting and segmenting watermarked text generated by large language models (LLMs). The key contributions are:\n\nA rigorous analysis of Type I and Type II errors for a randomization test to detect the presence of watermarks in generated text. The authors apply their findings to two specific watermarking schemes: inverse transform sampling and exponential minimum sampling (Gumbel watermark).\nA novel statistical approach to segment text into watermarked and non-watermarked substrings. This method is based on change point detection techniques and is designed to handle scenarios where users may modify LLM-generated text through insertions, deletions, or substitutions.\nTheoretical analysis of the proposed segmentation method, including convergence rates for estimated change point locations.\nEmpirical evaluation of the proposed methods using texts generated by GPT-2 and OPT-1.3b models, with prompts from Google's C4 dataset. The experiments demonstrate the effectiveness of the approach under various modification scenarios.\n\nThe paper provides a theoretical foundation for watermark detection and segmentation in LLM-generated text, addressing an important problem in the context of distinguishing between human-written and machine-generated content.\n\nSTRENGTHS:\nOriginality:\n\nA novel statistical approach for watermarked text segmentation, is very interesting and useful.\n\nCreative application of change point detection to watermarking.\n\nQuality:\n\nRigorous theoretical analysis with proofs.\n\nThorough experimental design across multiple scenarios.\n\nClarity:\n\nWell-structured paper with clear progression.\n\nReadable mathematical concepts with intuitive explanations.\n\nSignificance:\n\nAddresses key problem of distinguishing AI-generated text in \"segmented text\" scenario.\n\nWEAKNESSES:\nWeaknesses:\n\nLimited model diversity: Experiments focus on GPT-2 and OPT-1.3b. Testing on larger language models would strengthen findings.\n\nWhile I appreciate the intent of this paper, the attack scenario set up in this article is still rather simple. In a real-world application,\n1) Users may make changes to the text by means of cross-lingual attack and GPT rewrite, etc., the authors did not analyze these scenarios.\n2) The precautions users may take to prevent possible watermarking are more complex. I would like to see more change points and performance ratings of methods for scenarios with different segment lengths.\n\nIn addition, better visualization for selected scenarios could better reflect the value of the accuracy of the proposed methodology.\n\nIt should be noted that I will raise my score when I see appropriate rebuttals.\n\nQUESTIONS:\n1. I am interested in the number of $T$ selected for watermarked text detection. Is there a solution for balancing performance and speed?\n\nLIMITATIONS:\nThe authors have adequately addressed the limitations in the checklist.", "TITLE": "Official Review", "RECOMMENDATION": 5, "REVIEWER_CONFIDENCE": 2, "SOUNDNESS_CORRECTNESS": 3, "DATE": "2024-07-08"}

comments: SUMMARY:
This paper considers the problem of segmenting a watermarked text into watermarked and unwatermarked subsequences. This is achieved using change point detection methods. There is a nice statistical analysis of the single change point detection problem, and a heuristic algorithm (from Kovács et al., 2022) with experimental results for multiple change point detection.

STRENGTHS:
The paper is easy to read.

Section 2 (Problem setup) presents a result (Theorem 1) on the power of the watermark test that is more rigorous than previous presentations (to my knowledge).

The connection to change-point detection literature with analysis using the block-bootstrap is interesting (Section 3). While I am not deeply familiar with the change-point literature, the connections and application of these methods appears correct and well-cited.

The experiments appear well-executed.

WEAKNESSES:
This paper could be criticized for novelty, in the sense that it mostly applies standard tools from the change point detection literature to watermarking. I find Remark 2 somewhat cryptic; perhaps it speaks to some novelty of the analysis in Section 3, in which case this should be clarified.

Unbiased watermarks work best for high entropy text; small models like gpt2/opt-1.3b tend to produce high-entropy text. It would be informative to see whether change-point detection remains practical for the text generated by larger models (e.g., Mistral, LLaMA).

QUESTIONS:
Why was block size B=20 chosen? Is it important that B = B'? How do these and other parameters affect the performance of SeedBS in this setting?

LIMITATIONS:
Limitations are adequately addressed.
TITLE: Official Review
SOUNDNESS_CORRECTNESS: 4
DATE: 2024-07-10