Title: Unbiased Watermark for Large Language Models
Venue: NeurIPS 2023
Accepted: no
Keywords: watermark

language model
Abstract: The recent advancements in large language models (LLMs) have sparked a growing apprehension regarding the potential misuse. One approach to mitigating this risk is to incorporate watermarking techniques into LLMs, allowing for the tracking and attribution of model outputs. This study examines a crucial aspect of watermarking: how significantly watermarks impact the quality of model-generated outputs. Previous studies have suggested a trade-off between watermark strength and output quality. However, our research demonstrates that it is possible to integrate watermarks without affecting the output probability distribution with appropriate implementation. We refer to this type of watermark as an unbiased watermark. This has significant implications for the use of LLMs, as it becomes impossible for users to discern whether a service provider has incorporated watermarks or not. Furthermore, the presence of watermarks does not compromise the performance of the model in downstream tasks, ensuring that the overall utility of the language model is preserved. Our findings contribute to the ongoing discussion around responsible AI development, suggesting that unbiased watermarks can serve as an effective means of tracking and attributing model outputs without sacrificing output quality.

Abstract

The recent advancements in large language models (LLMs) have sparked a growing apprehension regarding the potential misuse. One approach to mitigating this risk is to incorporate watermarking techniques into LLMs, allowing for the tracking and attribution of model outputs. This study examines a crucial aspect of watermarking: how significantly watermarks impact the quality of model-generated outputs. Previous studies have suggested a trade-off between watermark strength and output quality. However, our research demonstrates that it is possible to integrate watermarks without affecting the output probability distribution with appropriate implementation. We refer to this type of watermark as an unbiased watermark . This has significant implications for the use of LLMs, as it becomes impossible for users to discern whether a service provider has incorporated watermarks or not. Furthermore, the presence of watermarks does not compromise the performance of the model in downstream tasks, ensuring that the overall utility of the language model is preserved. Our findings contribute to the ongoing discussion around responsible AI development, suggesting that unbiased watermarks can serve as an effective means of tracking and attributing model outputs without sacrificing output quality.

1 Introduction

In recent years, large language models (LLMs) [19, 39, 40] have become an indispensable tool for a wide range of tasks, including text generation [27, 10], translation [7, 5], summarization [36], etc. With the escalating misuse of LLMs, such as plagiarism, tracking the usage of text generated by machines has become increasingly important. One viable method to monitor the usage of LLMs is watermarking [20, 32, 59], which embeds imperceptible information within the generated text, thereby allowing for efficient detection and tracking of the model's potential abuse.

Watermarking techniques can serve multiple purposes, such as embedding ownership information 25 within the generated text to protect the intellectual property rights of the model. It can also help 26 mitigate potential harm caused by LLMs by monitoring where the model is being used and whether it 27 is being misused or abused. 28

A good watermarking method should not adversely affect the normal usage of the language model or 29 degrade the quality of the generated text. However, a prevailing belief holds that there is an inevitable 30 trade-off between the strength of the watermark and the quality of the output text. For instance, 31 recent work by Kirchenbauer et al. [32] introduced a method that augmented the logits of a randomly 32 selected set of "green" tokens. By tuning the 'magnitude of logits adjustment', they demonstrated a 33 trade-off between watermark strength and text quality. 34

Our primary contribution is to challenge this conventional wisdom. We show that with the right 35 implementation, watermarking can be accomplished without affecting the output quality. We refer to 36

this particular type of watermark as an unbiased watermark . We approach the problem of output 37 quality degradation from the perspective of watermark detection. We posit that if the watermark 38

39

40

41

42

43

44

45

46

47

48

49

50

51

52

53

54

55

56

57

58

59

60

61

62

63

64

65

66

67

68

69

70

71

72

73

74

75

76

77

78

79

80

81

82

83

84

causes a decline in output quality, there should be a method to guess the presence of the watermark based on the quality. Conversely, if the watermark cannot be detected, it implies that the output quality remains unaffected. Specifically, we provide a proof that with a suitable implementation, watermarking does not affect the output probability distribution. This has significant implications, as users who do not have the private key are unable to discern whether a service provider has applied watermarking to the model. Furthermore, the addition of watermarking does not affect the performance of the generated text in any downstream tasks. Our main contributions can be summarized as follows:

We introduce unbiased watermark , an innovative family of watermark methods that guarantee the non-degradation of text quality. In addition, we offer a comprehensive framework that facilitates the design and detection of unbiased watermarks.

We propose two innovative and practical watermarking techniques known as δ -reweight and γ -reweight. Through extensive experimentation, we demonstrate that these techniques preserve output quality in machine translation and text summarization tasks.

We develop an advanced maximin variant of the original log-likelihood ratio test for watermark detection. This novel detection method comes with theoretical guarantees, specifically an upper bound on type I error, thus enhancing the reliability of watermark detection in language models.

2 Preliminary

In this section, we delve into the problem of watermarking in the context of LLMs. We begin by setting up the problem and defining essential concepts.

Problem Modeling: We first introduce several notations to formalize the problem. Let Σ denote the vocabulary set, which is the set of all possible tokens an LLM can generate in a single step. We then define the set Σ ∗ as the collection of all possible strings of any length, including those of length zero.

An LLM generates a sequence of tokens conditioned on a given context. In a single step, the probability of generating the next token x n +1 ∈ Σ given the current context, x 1 , x 2 , ..., x n , can be denoted as P M ( x n +1 | x 1 , x 2 , ..., x n ) . The LLM operates in an autoregressive fashion, which means the joint probability of generating multiple tokens x n +1 , . . . , x n + m can be written as:

For simplicity, we use the following notation: P M ( x n +1: n + m | x 1: n ) , where x n +1: n + m = ( x n +1 , . . . , x n + m ) ∈ Σ ∗ .

In the context of watermarking, we introduce a service provider that holds a private key k from the key space K . The key k ∈ K is chosen at random from the prior distribution P K ( k ) . The watermarked output of the LLM follows distribution P M,w ( x n +1 | x 1 , x 2 , ..., x n ; k ) , which is conditioned on both the key k and the context x 1: n . Similarly, we use the notation P M,w ( x n +1: n + m | x 1: n ; k ) for the probability of generating a sequence of tokens in a watermarked model.

Objective. Our goal is to devise a watermarking scheme that: a) is efficiently detectable by the service provider; b) can't be detected by users and does not negatively impact the quality of the output.

The reason we focus on the detection of watermarks by users is that it is closely related to the output quality. If the watermark causes a degradation in the output quality, there should exist a method to infer the presence of the watermark by examining the quality. Conversely, if the watermark is undetectable, it implies that it does not impact the output quality.

From a statistical testing perspective, a watermark is considered strictly undetectable if the probability distributions of the watermarked and non-watermarked outputs are identical. To capture this notion, we define several desirable properties of watermarking schemes.

Definition 1 ( n -shot-undetectable) . For a fixed input sequence a ∈ Σ ∗ , we say that watermarked LLM and key prior pair ( P M,w , P K ) is n -shot-undetectable compared to original LLM P M if

85

86

87

88

89

90

91

92

93

94

95

96

97

98

99

100

101

102

103

104

105

106

107

108

109

110

111

112

113

114

115

116

117

118

119

120

121

122

123

124

125

126

127

128

129

130

131

132

133

Definition 2 (downstream-invariant) . We say the watermarked LLM and key prior pair ( P M,w , P K ) are invariant compared to original LLM P M on downstream tasks iff

for any strings x , a ∈ Σ , and for any metric f : Σ → R

Note that the one-shot-undetectable property implies the downstream invariant property. Interestingly, this implication does not require the n -shot-undetectable property for n > 1 , which means a watermarking scheme that is one-shot-undetectable can still maintain the output quality for downstream tasks even if the user might discern the existence of the watermark through multiple generation requests.

In summary, we have outlined the preliminary concepts and objectives for developing a watermarking scheme for LLMs. We highlight the desired properties of n -shot-undetectability and downstream invariance, as they provide a rigorous theoretical guarantee of quality preservation and integrity in the deployment of watermark schema. In Section 4, we will present a watermark framework that is provably n -shot-undetectable for any given integer n ≥ 1 .

3 Warm up: undetectability in a simplified toy environment

In this subsection, we aim to prove the feasibility of undetectability in a highly simplified toy environment. This preliminary analysis serves as a foundation for understanding the more complex scenarios that follow.

Settings. Consider a service provider that offers a random number generation service. The service outputs a uniformly distributed random number in the set { 0 , 1 } . The clean generation process can be represented as P M ( x ) = 1 / 2 , ∀ x ∈ { 0 , 1 } . We assume that the key k belongs to the set { 0 , 1 } and is selected with equal probability. With the watermark added, the probability of the new output can be expressed as: P M,w ( x | k ) = δ k ( x ) .

Recall that the one-shot-undetectable property can be represented as P M ( x ) = ∑ k ∈ K P M,w ( x | k ) P K ( k ) . Suppose that a user can only make a single request to the service. If the user is unaware of the key, the user will be unable to distinguish whether the received result is watermarked or not. Therefore, in this simplified scenario, the undetectability of the watermark is achieved.

However, there is a considerable gap between this toy example and the practical implementation of watermarking in LLMs. Firstly, the symbol set Σ in LLMs is far more complex than the binary set { 0 , 1 } , and the probability distribution is not uniform. Besides, the generation process in LLMs is autoregressive, which means that more than one symbol are generated iteratively. Furthermore, the toy example does not satisfy the n -shot-undetectable property for n > 1 .

Despite these differences, this simple example provides essential insights that help in understanding the following sections where we address these challenges. The underlying principles of undetectability remain constant, while their application becomes more intricate in a more complex environment.

4 Watermarking with unbiased reweighting

In this section, we build upon the intuition from the previous section and extend the approach to LLMs' generation. The section is structured as follows: Section 4.1 introduces a fundamental mathematical tool for addressing the reweighting problem in general discrete probability distributions. Section 4.2 applies the reweighting technique to LLMs. Section 4.3 presents the final framework.

4.1 Distribution reweighting

In its most general form, we consider a random watermark code E and a reweight function R E : ∆ Σ → ∆ Σ , which depends on the random watermark code E . The set of all possible probability distributions on the symbol set Σ is denoted as ∆ Σ , which forms a simplex.

Definition 3. A reweighting function is a tuple ( E , P E , R ) where E is called the watermark code space, P E is a probability distribution on space E , and R is a function R : E × ∆ Σ → ∆ Σ . For a specific watermark code E ∈ E , we denote the partially evaluated reweighting function as R E : ∆ Σ → ∆ Σ .

Definition 4. Given a random watermark code E and a reweighting function R E : ∆ Σ → ∆ Σ , we say that R is an unbiased reweighting function if and only if for all P ∈ ∆ Σ , E E [ R E ( P )] = P .

134

135

136

137

138

139

140

141

142

143

144

145

146

147

148

149

150

151

152

153

154

155

156

157

158

159

160

161

162

163

164

165

166

167

168

169

170

4.1.1 Existing reweighting methods

Kirchenbauer et al. [32] essentially comprise two reweighting methods in their work, but neither of them satisfies the unbiased property.

Both methods have E as the set of mappings f : Σ → { red , green } , such that f maps half of the tokens in Σ to 'red' and the other half to 'green', and P E as a uniform distribution. Therefore, the random watermark code E assigns each symbol to either red or green . The 'Hard Red List' method sets the probability of all red symbols to zero and renormalizes the probabilities of the remaining vocabulary. The second method is 'Soft Red List' blocking, where they randomly select the same 'Red List' as the first method and decrease the corresponding probability for red symbols by adding a constant δ to the logits of the green symbols, then apply softmax to obtain the final probabilities.

4.1.2 Unbiased reweighting methods

In this section, we present two reweighting methods that satisfy the unbiased property.

δ -reweight: Let the watermark code space E be the interval [0 , 1] , and let P E be the uniform probability on E . Leveraging Inverse Transform Sampling 1 [14], we can sample from distribution P ∈ ∆ Σ using a uniformly distributed random number in [0 , 1] . Therefore, we have a mapping sampling P : E → Σ . The δ -reweight just returns a delta distribution R E ( P ) = δ sampling P ( E ) .

It is important to note that while the reweighted distribution for each individual random event E is a delta distribution, the mean output token probabilities remain the original distribution P when considering the randomness of E .

γ -reweight: Let the watermark code space E be the set of all bijective function between vocabularies set Σ and a set of indices [ | Σ | ] = { 1 , . . . , | Σ |} , where | Σ | is the size of vocabularies set Σ . Essentially, any watermark code E is an indexing function for vocabularies set Σ , and is also equivalent to a total order on Σ . Let P E be the uniform probability on E , it is easy to sample a watermark code E by randomly shuffling the symbol list.

Assume the original distribution is P T ( t ) ∈ ∆ Σ , ∀ t ∈ Σ . Given the watermark code E : Σ → [ | Σ | ] , we construct auxiliary functions F I ( i ) = ∑ t ∈ Σ 1 ( E ( t ) ≤ i ) P T ( t ) , F S ( s ) = max(2 s -1 , 0) , F I ′ ( i ) = F S ( F I ( i )) . The γ -reweight yields new distribution P T ′ ( t ) = F I ′ ( E ( t )) -F I ′ ( E ( t ) -1) .

...

'

but"

...

'ized"

...

E

Reweight

'ized"

0

1

...

'

but"

...

'ized"

...

...

'ized"

...

'

but" ...

0

1/2

1

Shuffle

Reweight

'ized"

...

'

but"

...

0

1

We provide illustrations of the δ -reweight and γ -reweight methods in Figures 1 and 2. Each block represents a token, and the width represents the probability of that token, so the total length is 1 The left panel shows the δ -reweight method, where each individual random watermark code E ∈ [0 , 1] uniformly sampled from interval [0 , 1] corresponds to a specific token according to the horizontal axis, and the reweighted distribution is just a δ distribution on that token, such that the selected token has 1 probability, and all other vocabulary tokens have a probability of 0 . The right panel demonstrates the γ -reweight method. First, the symbol set is shuffled. Then, the left half of the regions are rejected, and the remaining regions are amplified with a factor of 2.

Both methods are unbiased 1 when considering the randomness of the watermark code E . For δ -reweight, we can see that by noticing that the probability of returning a δ distribution on a token is

171

172

173

174

175

176

177

178

179

180

181

182

183

184

185

186

187

188

189

190

191

192

193

194

195

196

197

198

199

200

201

202

203

204

205

206

207

208

209

210

211

212

213

214

215

216

217

218

219

220

221

just the original probability on that token, therefore the weighted average of all delta distributions is still the original probability. In the case of γ -reweight, although certain regions are rejected and the other regions are amplified, every token has the same probability to be in the rejected or amplified region, thus ensuring the unbiased property.

4.2 Reweighting for autoregressive model

The reweighting methods presented in the previous section can be applied to single token-generation directly. Given a prefix x 1: n , the probability distribution for generating a new token without a watermark is denoted as P M ( ·| x 1: n ) ∈ ∆ Σ . For a random watermark code E , we sample from a new distribution P M,w ( ·| x 1: n ) = R E ( P M ( ·| x 1: n )) ∈ ∆ Σ . If the reweighting function is unbiased, we have E E [ R E ( P M ( ·| x 1: n ))] = P M ( ·| x 1: n ) . This ensures that, for an individual unaware of the watermark code, it is impossible to determine whether a new token is sampled directly from P M ( ·| x 1: n ) or from P M,w ( ·| x 1: n ; E ) for a random watermark E . However, if the watermark code is known, one can perform statistical hypothesis testing to determine the likelihood of a token being sampled from either distribution.

The main challenge now is constructing the watermark code E . Since the LLM generation task is autoregressive, multiple reweighting steps are required, with each step needing a watermark code E i for reweighting the distribution of token x i .

4.2.1 Independence of watermark codes

It is crucial that E i values are independent to ensure the unbiased nature of the entire sequence, rather than just the single-token generation process.

Theorem 5. Given an unbiased reweighting function ( E , P E , R ) , if E i values are i.i.d. with the distribution P E , we have: E E 1 ,...,E n [ P M,w ( x 1: n | a 1: m )] = P M ( x 1: n | a 1: m ) .

If the E i values are not independent, we cannot guarantee that the generation probability of the entire sequence remains unbiased. As an extreme example, consider a case where all E i values are identical. Referring to the random bit example in the previous section, assume that the correct distribution is a sequence where each token is a random 0 or 1 with equal probability. Identical E i values would result in identical token outputs, ultimately producing sequences consisting solely of 0's or 1's, which is clearly biased.

4.2.2 Context code

To construct a large number of independent watermark codes E i during watermarking and to know the used E i values during watermark detection, we follow an approach similar to Kirchenbauer et al. [32] by combining the information from the prefix and a secret key to construct E i .

For a single token generation process, given a prefix x 1 , x 2 , ..., x n , we consider an abstract context code space C and an abstract context code generation function cc : Σ ∗ → C . Based on the prefix, we construct the context code c n +1 = cc ( x 1 , x 2 , ..., x n ) . Specific examples include using the entire prefix c n +1 = ( x 1 , x 2 , ..., x n ) , and using the m most recent prefixes c n +1 = ( x n -m +1 , ..., x n ) . Our comprehensive framework accommodates diverse context code generation approaches, particularly those that integrate error-correcting mechanisms to augment watermark resilience in the face of text manipulation attacks. Nevertheless, we refrain from delving into these strategies within the confines of this paper and consider it a subject for subsequent investigation.

The final watermark code is defined as E i = ˆ E ( c i , k ) , using a watermark code generation function ˆ E : C × K →E .

Definition 6. Given an unbiased reweighting function ( E , P E , R ) and a context code space C , an unbiased watermark code generation function is a tuple ( E , P E , R, C, K, P K , ˆ E ) that satisfies:

Unbiasedness: E k ∼ P K [ R ˆ E ( c,k ) ( P )] = P, ∀ P ∈ ∆ Σ , ∀ c ∈ C .

Independence: For any n distinct c 1 , . . . , c n ∈ C , the values R ˆ E ( c i ,k ) ( P ) are mutually independent.

Theorem 7. For any unbiased reweighting function and context code space, an unbiased watermark code generation function always exists.

In practice, pseudorandom numbers can be used to implement the unbiased watermark code generation function in the above theorem. Specifically, the hash value hash( c, k ) can be used as a random seed

222

223

224

225

226

227

228

229

230

231

232

233

234

235

236

237

238

239

240

241

242

243

244

245

246

247

248

249

250

251

252

253

to sample E from P E as an implementation of E = ˆ E ( c, k ) . In this paper, we employ SHA-256 for hash function and a 1024-bit random bitstring as the key k .

An unbiased watermark code generation function ensures that watermark codes E i are independent with each other if only their context codes are different. During the generation of a sequence, context codes may be repeated, although this is a rare event in practice. If c i and c j are equal, then E i and E j are also equal, violating the independence of E i . A simple workaround is to skip reweighting for a token when encountering a previously used context code. In other words, we set P M,w ( ·| a 1: m , x 1: i -1 ) = P M ( ·| a 1: m , x 1: i -1 ) if the context code has appeared before.

4.3 Framework

Algorithm 1 Watermarking framework

1: Input: key for watermark k ∈ K , prompt a 1: m ∈ Σ ∗ , generate length n ∈ N , initial code history cch ∈ 2 C , context code function cc : Σ ∗ → C , watermark code generation function ˆ × →E , and reweighting function E × → .

Integrating the tools discussed earlier, we present a general framework for watermarking here. The algorithm for this framework is outlined in Algorithm 1.

We note that our abstract framework requires the specification of two key components in order to be practically implemented: the unbiased reweight function R E and the context code function cc .

5 Statistical hypothesis testing for watermark detection

In the previous section, we discussed the process of adding a watermark to a text based on a secret key k and a given prompt a 1: m . The watermark-embedded text can be sampled from the distribution P M,w ( x 1: n | a 1: m ; k ) . In this section, we focus on the watermark detection task, which is the inverse problem of watermark embedding.

Given a text x 1: n , the goal of watermark detection is to infer whether it is more likely to be generated from the unmarked distribution P M ( x 1: n | a 1: m ) or the marked distribution P M,w ( x 1: n | a 1: m ; k ) . This problem can be formulated as a statistical hypothesis test between two competing hypotheses: H 0 , which posits that x 1: n follows the unmarked distribution, and H 1 , which posits that x 1: n follows the marked distribution.

5.1 Score-based tesing

We focus on a particular kind of score-based testing, which assigns a score to each token in the text. The score can be interpreted as the confidence that the token was generated by the watermark model rather than the original model. Scores s i can be computed based on x 1: i , in accordance with the autoregressive manner of the generation process.

The total score S is given by S = ∑ n i =1 s i . A threshold ˆ S is set such that if S < ˆ S , the null hypothesis H 0 is accepted, indicating insufficient evidence to conclude that the text contains a watermark. Otherwise, the null hypothesis is rejected. There are two types of error probabilities associated with this decision process: Type I error, which is the probability of incorrectly rejecting

254

255

256

257

258

259

260

261

262

263

264

265

266

267

268

269

270

271

272

273

274

275

276

277

278

279

280

281

282

283

284

285

286

287

288

289

290

291

292

293

294

295

296

297

298

299

the null hypothesis under H 0 , denoted as P H 0 ( S ≥ ˆ S ) , and Type II error, which is the probability of incorrectly accepting the null hypothesis under H 1 , denoted as P H 1 ( S < ˆ S ) .

To derive theoretical results, we require the scores to have a specific property: under the null hypothesis H 0 , the exponential momentum of s i is bounded, conditioned on the preceding context x 1 ,i -1 . This requirement leads to an upper bound on α , the Type I error probability.

To derive theoretical results, we require that the scores have a particular property: the exponential moment of s i under H 0 should be bounded, conditioned on the previous text x 1 ,i -1 . This requirement leads to an upper bound on the Type I error rate.

Theorem 8. Given a probability space (Ω , A , P ) and a Σ -valued stochastic process x i : 1 ≤ i ≤ n , as well as an R -valued stochastic process s i : 1 ≤ i ≤ n , let F x i := σ ( x j | 1 ≤ j ≤ i ) and F s i := σ ( s j | 1 ≤ j ≤ i ) be the corresponding filtrations, where σ ( · ) denotes the σ -algebra generated by random variables. If F s i ⊆ F x i and E [exp( s i ) |F x i -1 ] ≤ 1 , then P ( ∑ n i =1 s i ≥ t ) ≤ e -t .

Therefore, to ensure that the Type I error probability has an upper bound α , we can set the threshold ˆ S as ˆ S = -log( α ) . In the following, we discuss two special scores.

5.2 Log likelihood ratio (LLR) score

According to the Neyman-Pearson lemma, the likelihood ratio test is the most powerful test among all tests with the same Type I error rate. Specifically, the log-likelihood ratio (LLR) score is defined as s i = log P M,w ( x i | a 1: m , x 1: i -1 ; k ) P M ( x i | a 1: m , x 1: i -1 ) , and the total score becomes S = log P M,w ( x 1: n | a 1: m ; k ) P M ( x 1: n | a 1: m ) .

We now provide an optimization derivation of the above s i to gain intuition and set the foundation for the maximin variant of the LLR score in the next section. Let P i = P M ( ·| a 1: m , x 1: i -1 ) , Q i = P M,w ( ·| a 1: m , x 1: i -1 ; k ) , and let s i = S i ( x i ) ∈ R denote the score corresponding to different x i . Note that P i , Q i , and S i are all functions with signature Σ → R , therefore equivalent to vectors of dimension | Σ | . We can define the inner product as ⟨ P i , S i ⟩ = ∑ x ∈ Σ P i ( x ) S i ( x ) .

The requirement E [exp( s i ) |F x i -1 ] ≤ 1 can be reformulated as ⟨ P i , exp( S i ) ⟩ ≤ 1 , where the exponential function is applied element-wise. Instead of minimizing the Type II error directly, we aim to maximize the average score under H 1 , i.e., ⟨ Q i , S i ⟩ .

The optimization problem becomes max S i ⟨ Q i , S i ⟩ , s.t. ⟨ P i , exp( S i ) ⟩ ≤ 1 . The optimal solution is given by S i ( x ) = log Q i ( x ) P i ( x ) , which recovers the optimal log likelihood ratio score.

5.3 Maximin variant of LLR score

One major limitation of the LLR score described in the previous section is that when Q i ( x ) = 0 , S i ( x ) = -∞ . This means that as long as a single token does not come from the watermark model P M,w , the score becomes negative infinity, making it impossible to reject the null hypothesis H 0 .

A more general reason for this issue is that the watermark model P M,w used in the detection process may not exactly match the true distribution of the watermarked text. In practice, potential sources of discrepancy include editing (e.g., a text sampled from P M,w may undergo some degree of editing before being watermark detection) and imperfect estimation of the generation process (e.g., due to lack of knowledge of the exact prompt and temperature used during generation).

To address this problem, we consider a perturbed generation distribution. Instead of the original hypothesis H 1 , where x 1: n follows the watermark distribution P M,w , we now assume that x 1: n follows a distribution P ′ M,w , which is similar to but not identical to P M,w . Specifically, during the generation of each token, the total variation (TV) distance between Q ′ i and Q i is bounded by d .

The corresponding new optimization problem is

Intuitively, the optimal solution for Q ′ i in the inner optimization decreases Q ′ i ( x ) when S i ( x ) is large and increases Q ′ i ( x ) when S i ( x ) is small.

The computation of the maximin solution can be done efficiently in ˜ O ( | Σ | ) time and the specific algorithm is shown in Appendix C.

300

301

302

303

304

305

306

307

308

309

310

311

312

313

314

315

316

317

318

319

320

321

(a) Text summarization

No Watermark

δ

-reweight

γ

-reweight

Soft(

δ

=0.0)

Soft(

δ

=1.0)

Soft(

δ

=2.0)

2

4

6

8

10

Perplexity

(b) Machine translation

No Watermark

δ

-reweight

γ

-reweight

Soft(

δ

=0.0)

Soft(

δ

=1.0)

Soft(

δ

=2.0)

19

20

21

22

23

BLEU

It is important to note that the maximin variant of the LLR score is more robust than the standard LLR score, as it yields higher scores when the text has undergone some degree of editing. However, it is not specifically designed to defend against any attacks.

A hyperparameter d ∈ [0 , 1] that represent the perturbation strength is introduced in the score. Intuitively, if the text to be detected has undergone more editing and deviates further from the distribution P M,w , d should be larger. In practice, we recommend using grid search to select the best value of d . Assuming there are A candidate values for d , corresponding to A different scores s ( a ) i ( 1 ≤ a ≤ A ), we can modify Theorem 8 as follows.

Theorem 9. Under the same conditions as Theorem 8, but with multiple scores s ( a ) i , we have

Thus, when using grid search, the final threshold should be adjusted as ˆ S = -log( α ) + log( A ) . This ensures that the upper bound of the type I error is still α .

6 Experiments

We evaluate the performance of our Unbiased Watermarks on two important applications of seq2seq models: text summarization (TS) and machine translation (MT). For the TS task, we use the BARTlarge model [37] and the CNN-DM [25] corpus as our training dataset. The MT task involves translating English to Romanian, for which we employ the Multilingual BART (MBart) [37] model on the WMT'14 En-Ro corpus. For further details on the experiment setup, please refer to Appendix E.

Our primary focus is to compare the performance of our proposed unbiased watermarking methods including the δ -reweight and γ -reweight, with the soft-red-list method presented by Kirchenbauer et al. [32]. The strength of the watermark in the soft-red-list approach is controlled by a parameter δ .

The quality of output post-watermarking is presented in Table 1. We observed that the output quality remains unaffected by our unbiased watermark methods, both for the δ -reweight and γ -reweight,

322

323

324

325

326

327

328

329

330

331

332

333

334

335

336

337

338

339

340

341

342

343

344

345

346

347

348

349

350

351

352

353

irrespective of the task and metric. Conversely, the soft-red-list method, when δ = 0 , does not introduce any watermark and hence does not affect output quality. However, for δ > 0 , it significantly deteriorate the quality of output.

Figure 3 provides a more intuitive depiction of the score distributions. It is evident that our unbiased watermark methods not only ensure that the mean performance remains unaffected but also that the performance distribution is stable. Conversely, the soft-red-list method shows a notable performance decrease.

In terms of watermark detection, we compute score associated with each token. The mean and variance of score per token for TS and MT are presented in Table 2. As a heuristic, if the sum of the scores for all tokens in a sentence reaches 10 , a p-value of less than 0 . 0005 is ensured. If the sum score hits 20 , the p-value must be less than 3e -8 .

Additionally, we provide an example of watermarking applied to a completion task in Table 3. It visually demonstrates the score distribution across tokens: positive scores are represented in green, and negative ones in red. The intensity of the color corresponds to the magnitude of the score, with darker shades representing larger absolute values.

7 Related work

The idea of watermarking text has been widely explored by many researchers [11, 31, 44, 45, 4, 28, 49, 43], even before the advent of large language models. Several techniques involve editing existing text to add a watermark, such as changing synonyms [54, 57, 9, 59, 66] or visually indistinguishable words [46], altering sentence structures [56, 55, 38], and employing neural networks [22, 23, 67].

Recent advancements in generative models have opened new possibilities for directly generating watermarked results. Two relevant works in this domain are by Kirchenbauer et al. [32] and Aaronson [1]. Due to space constraints, we moved the in-depth analysis and other related work to Section B.

8 Conclusion

Overall, this paper provides a novel framework of watermarking for language models, demonstrating that it is possible to use watermark to protect intellectual property and monitor potential misuse without compromising the quality of the generated text. This research serves as a valuable foundation for future work in the field of watermarking for large language models.

354

355

356

357

358

359

360

361

362

363

364

365

366

367

368

369

370

371

372

373

374

375

376

377

378

379

380

381

382

383

384

385

386

387

388

389

390

391

392

References

Scott Aaronson. My ai safety lecture for ut effective altruism. November 2022. URL https:// scottaaronson.blog/?p=6823 .

Sahar Abdelnabi and Mario Fritz. Adversarial watermarking transformer: Towards tracing text provenance with data hiding. In 2021 IEEE Symposium on Security and Privacy (SP) , pages 121-140. IEEE, 2021.

Yossi Adi, Carsten Baum, Moustapha Cisse, Benny Pinkas, and Joseph Keshet. Turning your weakness into a strength: Watermarking deep neural networks by backdooring. In 27th USENIX Security Symposium , pages 1615-1631, 2018.

Mikhail J Atallah, Victor Raskin, Michael Crogan, Christian Hempelmann, Florian Kerschbaum, Dina Mohamed, and Sanket Naik. Natural language watermarking: Design, analysis, and a proof-of-concept implementation. In Information Hiding: 4th International Workshop, IH 2001 Pittsburgh, PA, USA, April 25-27, 2001 Proceedings 4 , pages 185-200. Springer, 2001.

Loïc Barrault, Ondˇ rej Bojar, Marta R. Costa-jussà, Christian Federmann, Mark Fishel, Yvette Graham, Barry Haddow, Matthias Huck, Philipp Koehn, Shervin Malmasi, Christof Monz, Mathias Müller, Santanu Pal, Matt Post, and Marcos Zampieri. Findings of the 2019 conference on machine translation (WMT19). In Proceedings of the Fourth Conference on Machine Translation (Volume 2: Shared Task Papers, Day 1) , pages 1-61, Florence, Italy, August 2019. Association for Computational Linguistics. doi: 10.18653/v1/ W19-5301.

Franziska Boenisch. A systematic review on model watermarking for neural networks. Frontiers in big Data , 4:729663, 2021.

Ondˇ rej Bojar, Rajen Chatterjee, Christian Federmann, Yvette Graham, Barry Haddow, Shujian Huang, Matthias Huck, Philipp Koehn, Qun Liu, Varvara Logacheva, Christof Monz, Matteo Negri, Matt Post, Raphael Rubino, Lucia Specia, and Marco Turchi. Findings of the 2017 conference on machine translation (WMT17). In Proceedings of the Second Conference on Machine Translation , pages 169-214, Copenhagen, Denmark, September 2017. Association for Computational Linguistics. doi: 10.18653/v1/W17-4717.

Nicholas Boucher, Ilia Shumailov, Ross Anderson, and Nicolas Papernot. Bad characters: Imperceptible nlp attacks. In 2022 IEEE Symposium on Security and Privacy (SP) , pages 1987-2004. IEEE, 2022.

Yuei-Lin Chiang, Lu-Ping Chang, Wen-Tai Hsieh, and Wen-Chih Chen. Natural language watermarking using semantic substitution for chinese text. In Digital Watermarking: Second International Workshop, IWDW 2003, Seoul, Korea, October 20-22, 2003. Revised Papers 2 , pages 129-140. Springer, 2004.

Hyung Won Chung, Le Hou, Shayne Longpre, Barret Zoph, Yi Tay, William Fedus, Eric Li, Xuezhi Wang, Mostafa Dehghani, Siddhartha Brahma, et al. Scaling instruction-finetuned language models. arXiv preprint arXiv:2210.11416 , 2022.

Ingemar Cox, Matthew Miller, Jeffrey Bloom, Jessica Fridrich, and Ton Kalker. Digital watermarking and steganography . Morgan kaufmann, 2007.

Evan Crothers, Nathalie Japkowicz, and Herna Viktor. Machine generated text: A comprehensive survey of threat models and detection methods. arXiv preprint arXiv:2210.07321 , 2022.

Falcon Z Dai and Zheng Cai. Towards near-imperceptible steganographic text. arXiv preprint arXiv:1907.06679 , 2019.

Luc Devroye. Non-Uniform Random Variate Generation . Springer New York, 1986. 393

Tina Fang, Martin Jaggi, and Katerina Argyraki. Generating steganographic text with lstms. arXiv preprint 394 arXiv:1705.10742 , 2017. 395

Evgeniy Gabrilovich and Alex Gontmakher. The homograph attack. Communications of the ACM , 45(2): 396 128, 2002. 397

398

399

Margherita Gambini, Tiziano Fagni, Fabrizio Falchi, and Maurizio Tesconi. On pushing deepfake tweet detection capabilities to the limits. In 14th ACM Web Science Conference 2022 , pages 154-163, 2022.

Riley Goodside. There are adversarial attacks for that proposal as well - in particular, generating 400 with emojis after words and then removing them before submitting defeats it.,. January 2023. URL 401 https://twitter.com/goodside/status/1610682909647671306 . 402

Google. Palm-2-llm. https://blog.google/technology/ai/google-palm-2-ai-large-language-model/ , 2023. 403

404

405

406

407

408

409

410

411

412

413

414

415

416

417

418

419

420

421

422

423

424

425

426

427

428

429

430

431

432

433

434

435

436

437

438

439

440

441

442

443

444

445

446

447

448

449

450

451

452

453

Chenxi Gu, Chengsong Huang, Xiaoqing Zheng, Kai-Wei Chang, and Cho-Jui Hsieh. Watermarking pre-trained language models with backdooring. arXiv preprint arXiv:2210.07543 , 2022.

Tianyu Gu, Brendan Dolan-Gavitt, and Siddharth Garg. Badnets: Identifying vulnerabilities in the machine learning model supply chain. arXiv preprint arXiv:1708.06733 , 2017.

Xuanli He, Qiongkai Xu, Lingjuan Lyu, Fangzhao Wu, and Chenguang Wang. Protecting intellectual property of language generation apis with lexical watermark. In Proceedings of the AAAI Conference on Artificial Intelligence , volume 36, pages 10758-10766, 2022.

Xuanli He, Qiongkai Xu, Yi Zeng, Lingjuan Lyu, Fangzhao Wu, Jiwei Li, and Ruoxi Jia. Cater: Intellectual property protection on text generation apis via conditional watermarks. arXiv preprint arXiv:2209.08773 , 2022.

James N Helfrich and Rick Neff. Dual canonicalization: An answer to the homograph attack. In 2012 eCrime Researchers Summit , pages 1-10. IEEE, 2012.

Karl Moritz Hermann, Tomás Kociský, Edward Grefenstette, Lasse Espeholt, Will Kay, Mustafa Suleyman, and Phil Blunsom. Teaching machines to read and comprehend. In Corinna Cortes, Neil D. Lawrence, Daniel D. Lee, Masashi Sugiyama, and Roman Garnett, editors, Advances in Neural Information Processing Systems 28: Annual Conference on Neural Information Processing Systems 2015, December 7-12, 2015, Montreal, Quebec, Canada , pages 1693-1701, 2015.

Daphne Ippolito, Daniel Duckworth, Chris Callison-Burch, and Douglas Eck. Automatic detection of generated text is easiest when humans are fooled. arXiv preprint arXiv:1911.00650 , 2019.

Srinivasan Iyer, Xi Victoria Lin, Ramakanth Pasunuru, Todor Mihaylov, Dániel Simig, Ping Yu, Kurt Shuster, Tianlu Wang, Qing Liu, Punit Singh Koura, et al. Opt-iml: Scaling language model instruction meta learning through the lens of generalization. arXiv preprint arXiv:2212.12017 , 2022.

Zunera Jalil and Anwar M Mirza. A review of digital watermarking techniques for text documents. In 2009 International Conference on Information and Multimedia Technology , pages 230-234. IEEE, 2009.

Ganesh Jawahar, Muhammad Abdul-Mageed, and Laks VS Lakshmanan. Automatic detection of machine generated text: A critical survey. arXiv preprint arXiv:2011.01314 , 2020.

Hengrui Jia, Christopher A Choquette-Choo, Varun Chandrasekaran, and Nicolas Papernot. Entangled watermarks as a defense against model extraction. In USENIX Security Symposium , pages 1937-1954, 2021.

Nurul Shamimi Kamaruddin, Amirrudin Kamsin, Lip Yee Por, and Hameedur Rahman. A review of text watermarking: theory, methods, and applications. IEEE Access , 6:8011-8028, 2018.

John Kirchenbauer, Jonas Geiping, Yuxin Wen, Jonathan Katz, Ian Miers, and Tom Goldstein. A watermark for large language models. arXiv preprint arXiv:2301.10226 , 2023.

Kalpesh Krishna, Yixiao Song, Marzena Karpinska, John Wieting, and Mohit Iyyer. Paraphrasing evades detectors of ai-generated text, but retrieval is an effective defense. arXiv preprint arXiv:2303.13408 , 2023.

Zheng Li, Chengyu Hu, Yang Zhang, and Shanqing Guo. How to prove your model belongs to you: A blind-watermark based framework to protect intellectual property of dnn. In Proceedings of the 35th Annual Computer Security Applications Conference , pages 126-137, 2019.

Chin-Yew Lin. Rouge: A package for automatic evaluation of summaries. In Text summarization branches out , pages 74-81, 2004.

Yang Liu and Mirella Lapata. Text summarization with pretrained encoders. In Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing and the 9th International Joint Conference on Natural Language Processing (EMNLP-IJCNLP) , pages 3730-3740, Hong Kong, China, November 2019. Association for Computational Linguistics. doi: 10.18653/v1/D19-1387.

Yinhan Liu, Jiatao Gu, Naman Goyal, Xian Li, Sergey Edunov, Marjan Ghazvininejad, Mike Lewis, and Luke Zettlemoyer. Multilingual denoising pre-training for neural machine translation. Transactions of the Association for Computational Linguistics , 8:726-742, 2020.

Hasan Mesut Meral, Bülent Sankur, A Sumru Özsoy, Tunga Güngör, and Emre Sevinç. Natural language watermarking via morphosyntactic alterations. Computer Speech & Language , 23(1):107-125, 2009.

OpenAI. Chatgpt. https://openai.com/blog/chatgpt , 2023a.

OpenAI. Gpt-4 technical report. arXiv , 2023b. 454

Luca Pajola and Mauro Conti. Fall of giants: How popular text-based mlaas fall against a simple evasion 455 attack. In 2021 IEEE European Symposium on Security and Privacy (EuroS&P) , pages 198-211. IEEE, 456 2021. 457

458

459

460

Kishore Papineni, Salim Roukos, Todd Ward, and Wei-Jing Zhu. Bleu: a method for automatic evaluation of machine translation. In Proceedings of the 40th annual meeting of the Association for Computational Linguistics , pages 311-318, 2002.

461

462

463

464

465

466

467

Fabien AP Petitcolas, Ross J Anderson, and Markus G Kuhn. Information hiding-a survey. Proceedings of the IEEE , 87(7):1062-1078, 1999.

Christine I Podilchuk and Edward J Delp. Digital watermarking: algorithms and applications. IEEE signal processing Magazine , 18(4):33-46, 2001.

Vidyasagar M Potdar, Song Han, and Elizabeth Chang. A survey of digital image watermarking techniques. In INDIN'05. 2005 3rd IEEE International Conference on Industrial Informatics, 2005. , pages 709-716. IEEE, 2005.

Stefano Giovanni Rizzo, Flavio Bertini, and Danilo Montesi. Fine-grain watermarking for intellectual 468 property protection. EURASIP Journal on Information Security , 2019:1-20, 2019. 469

470

471

Vinu Sankar Sadasivan, Aounon Kumar, Sriram Balasubramanian, Wenxiao Wang, and Soheil Feizi. Can ai-generated text be reliably detected? arXiv preprint arXiv:2303.11156 , 2023.

472

473

474

475

476

477

478

479

480

481

482

483

484

485

486

487

488

489

490

491

492

493

494

495

496

497

498

499

500

501

502

MHassan Shirali-Shahreza and Mohammad Shirali-Shahreza. A new synonym text steganography. In 2008 international conference on intelligent information hiding and multimedia signal processing , pages 1524-1526. IEEE, 2008.

Katzenbeisser Stefan, A Petitcolas Fabien, et al. Information hiding techniques for steganography and digital watermarking, 2000.

Yuchen Sun, Tianpeng Liu, Panhe Hu, Qing Liao, Shouling Ji, Nenghai Yu, Deke Guo, and Li Liu. Deep intellectual property: A survey. arXiv preprint arXiv:2304.14613 , 2023.

Reuben Tan, Bryan A Plummer, and Kate Saenko. Detecting cross-modal inconsistency to defend against neural fake news. arXiv preprint arXiv:2009.07698 , 2020.

Ruixiang Tang, Yu-Neng Chuang, and Xia Hu. The science of detecting llm-generated texts. arXiv preprint arXiv:2303.07205 , 2023.

Yi Tay, Dara Bahri, Che Zheng, Clifford Brunk, Donald Metzler, and Andrew Tomkins. Reverse engineering configurations of neural text generation models. arXiv preprint arXiv:2004.06201 , 2020.

Mercan Topkara, Cuneyt M Taskiran, and Edward J Delp III. Natural language watermarking. In Security, Steganography, and Watermarking of Multimedia Contents VII , volume 5681, pages 441-452. SPIE, 2005.

Mercan Topkara, Giuseppe Riccardi, Dilek Hakkani-Tür, and Mikhail J Atallah. Natural language watermarking: Challenges in building a practical system. In Security, Steganography, and Watermarking of Multimedia Contents VIII , volume 6072, pages 106-117. SPIE, 2006.

Mercan Topkara, Umut Topkara, and Mikhail J Atallah. Words are not enough: sentence level natural language watermarking. In Proceedings of the 4th ACM international workshop on Contents protection and security , pages 37-46, 2006.

Umut Topkara, Mercan Topkara, and Mikhail J Atallah. The hiding virtues of ambiguity: quantifiably resilient watermarking of natural language text through synonym substitutions. In Proceedings of the 8th workshop on Multimedia and security , pages 164-174, 2006.

Honai Ueoka, Yugo Murawaki, and Sadao Kurohashi. Frustratingly easy edit-based linguistic steganography with a masked language model. arXiv preprint arXiv:2104.09833 , 2021.

Ashish Venugopal, Jakob Uszkoreit, David Talbot, Franz Josef Och, and Juri Ganitkevitch. Watermarking the outputs of structured prediction with an application in statistical machine translation. In Proceedings of the 2011 Conference on Empirical Methods in Natural Language Processing , pages 1363-1372, 2011.

Hong Wang, Xuan Luo, Weizhi Wang, and Xifeng Yan. Bot or human? detecting chatgpt imposters with a single question. arXiv preprint arXiv:2305.06424 , 2023.

Alex Wilson and Andrew D Ker. Avoiding detection on twitter: embedding strategies for linguistic 503 steganography. Society of Photo-optical Instrumentation Engineers, 2016. 504

Alex Wilson, Phil Blunsom, and Andrew D Ker. Linguistic steganography on twitter: hierarchical language 505 modeling with manual interaction. In Media Watermarking, Security, and Forensics 2014 , volume 9028, 506 pages 9-25. SPIE, 2014. 507

508

509

510

Alex Wilson, Phil Blunsom, and Andrew Ker. Detection of steganographic techniques on twitter. In Proceedings of the 2015 Conference on Empirical Methods in Natural Language Processing , pages 2564-2569, 2015.

511

512

513

514

515

516

517

518

519

520

521

522

523

524

525

526

Thomas Wolf, Lysandre Debut, Victor Sanh, Julien Chaumond, Clement Delangue, Anthony Moi, Pierric Cistac, Tim Rault, Rémi Louf, Morgan Funtowicz, et al. Huggingface's transformers: State-of-the-art natural language processing. arXiv preprint arXiv:1910.03771 , 2019.

Max Wolff and Stuart Wolff. Attacking neural text detectors. arXiv preprint arXiv:2002.11768 , 2020.

Xi Yang, Jie Zhang, Kejiang Chen, Weiming Zhang, Zehua Ma, Feng Wang, and Nenghai Yu. Tracing text provenance via context-aware lexical substitution. In Proceedings of the AAAI Conference on Artificial Intelligence , volume 36, pages 11613-11621, 2022.

KiYoon Yoo, Wonhyuk Ahn, Jiho Jang, and Nojun Kwak. Robust natural language watermarking through invariant features. arXiv preprint arXiv:2305.01904 , 2023.

Rowan Zellers, Ari Holtzman, Hannah Rashkin, Yonatan Bisk, Ali Farhadi, Franziska Roesner, and Yejin Choi. Defending against neural fake news. Advances in neural information processing systems , 32, 2019.

Jialong Zhang, Zhongshu Gu, Jiyong Jang, Hui Wu, Marc Ph Stoecklin, Heqing Huang, and Ian Molloy. Protecting intellectual property of deep neural networks with watermarking. In Proceedings of the 2018 on Asia Conference on Computer and Communications Security , pages 159-172, 2018.

Xuandong Zhao, Yu-Xiang Wang, and Lei Li. Protecting language generation models via invisible watermarking. arXiv preprint arXiv:2302.03162 , 2023.

Zachary M Ziegler, Yuntian Deng, and Alexander M Rush. Neural linguistic steganography. arXiv preprint 527 arXiv:1909.01496 , 2019. 528

Peer reviews:
{"IS_META_REVIEW": false, "comments": "SUMMARY:\nThis study explores watermarking large language models without reducing output quality. It introduces \"unbiased watermarking\" which avoids trade-offs in prior work. Two novel techniques - $\\delta$-reweight and $\\gamma$-reweight - are proposed along with an improved likelihood ratio test for detection. Risks of large language models and how unbiased watermarking enables responsible AI are discussed.\n\nSTRENGTHS:\n1. Introduces unbiased watermarking that maintains output quality. Prior work showed a trade-off, but an unbiased watermark avoids this. \n \n2. Proposes novel $\\delta$-reweight and $\\gamma$-reweight techniques that preserve output quality in experiments.  \n\n3. Develops an improved detection method with a proven upper error bound, improving detection reliability. \n\n4. Concrete demonstration of the efficacy of watermarking techniques in maintaining the utility of LLMs for downstream tasks.\n\nWEAKNESSES:\n1. The paper lacks a thorough examination of the efficacy of the watermark detection method. It would strengthen the findings to provide more details on factors like detection accuracy, robustness to interference, and computational efficiency of the detection approach.  \n\n2. The experiments only test the watermarking techniques on BART language models, without evaluating other popular LLMs like GPT models.\n\n3. Conducting additional experiments using LLMs for other natural language tasks, beyond text summarization and machine translation studied in the paper, would provide a wider test case set and bolster the claims regarding the output quality preservation of the watermarking techniques.  \n\n4. The paper does not discuss the resilience of the proposed watermarking methods against potential adversarial attacks or interference attempts.\n\nQUESTIONS:\nWhat kinds of decoding methods are suitable for this watermarking method?\n\nLIMITATIONS:\nSee the weakness section.", "TITLE": "Official Review", "RECOMMENDATION": 3, "REVIEWER_CONFIDENCE": 4, "SOUNDNESS_CORRECTNESS": 2, "DATE": "2023-07-04"}

{"IS_META_REVIEW": false, "comments": "SUMMARY:\nThe paper proposes a modification of the watermark of Kirchenbauer et al. that ensures each next token prediction is marginally indistinguishable from a regular sample from the language model (whereas Kirchenbauer et al. bias some tokens over others). The main idea is to use inverse transform sampling to sample the text token (the paper also proposes a \"soft\" version of inverse transform sampling, i.e., \\gamma-reweight), where the inputs to the sampler are a function of the context.\n\nSTRENGTHS:\nThe paper proposes a neat, simple modification of the watermark of Kirchenbauer et al. that addresses a salient problem (i.e., the watermark of Kirchenbauer et al. does not preserve the original text distribution). The paper validates the proposed watermark with both theory and experiments.\n\nWEAKNESSES:\nThe empirical validation of the watermarked proposed in the paper is somewhat lacking. For example, how robust is the watermark to paraphrasing compared to the watermark of Kirchenbauer et al.? Given Kirchenbauer et al. evaluate robustness to various kinds of paraphrasing attacks, it seems reasonable to expect the paper to do the same. It's also not clear what the purpose of Table 1 and Figure 3 is if the main claim is that the watermark *provably* does not bias the text distribution (i.e., shouldn't all the metrics stay the same?). Also, Section 4.1 seems needlessly abstract, since ultimately both watermarks are variations of inverse transform sampling (it feels unintuitive to think \\delta-reweighting as a \"reweighting\" of a distribution, since it is essentially deterministic).\n\nQUESTIONS:\nLine 88: \"Note that the one-shot-undetectable property implies the downstream invariant property.\" It is not immediately clear why this is the case. In general, both Definitions 1 and 2 would benefit from some more context and/or clearer presentation.\n\nLIMITATIONS:\nThe paper does not discuss limitations (if any) of the proposed methods in meaningful detail. The Conclusion section would benefit from more detailed/concrete discussion and takeaways.", "TITLE": "Official Review", "RECOMMENDATION": 5, "REVIEWER_CONFIDENCE": 4, "SOUNDNESS_CORRECTNESS": 2, "DATE": "2023-07-07"}

{"IS_META_REVIEW": false, "comments": "SUMMARY:\nThis paper discuss about the important problem of how to watermark the outputs from language models while keeping the model not impacted by watermarking. A perfect watermark scheme should be undetectable without prior information and should have no harm on the utilities of LLMs. This paper proposes some desired properties of watermark schemes such as **n-shot-undetectable** and **downstream-invariant**. The papers also gives the proof that there exists perfect watermark schemes. And guided by the proposed concepts, two reweighing watermark scheme are proposed, as well as corresponding methods for verification. The experiment results show the proposed methods have minor impact on the generated text quality.\n\nSTRENGTHS:\n1. It's super important to have a theoretical framework to guide researchers to design better watermark schemes, and this paper is one of the pioneers in this direction. \n2. Considering those automatic metrics, the results shown in the paper, the proposed two methods look good.\n\nWEAKNESSES:\nMajor concerns: \n1: The quality of the watermarked texts are only evaluated by automatic metrics. \n2: The results only comes from BART ( along with examples from OPT ).  \n\nMinor concerns:\n1: The word 'unbiased' is a little misleading.\n2: It's a little hard to understand some paragraphs in this paper.\n\nQUESTIONS:\n1: As I stated before, if some evaluation results from LLMs such as GPTScore. The results would be more convincing. \n2:  It's about the 2nd weakness. The generation power of OPT is poor compared to many other LLMs.  Actually, all the provided examples in the paper is not good to me, either for those from a watermarked model or those from a model without watermark. I'm wondering the results from other LLMs such as LLaMA and T5.  Will the quality be severely degraded? \n3: The so-called strength of watermark doesn't mean the degradation of generation quality of language models. For examples, texts from a  watermarked model may have a different style from the model without watermark. Both the two styles of texts can be good to us.\n\nLIMITATIONS:\nPlease read last section.", "TITLE": "Official Review", "RECOMMENDATION": 5, "REVIEWER_CONFIDENCE": 3, "SOUNDNESS_CORRECTNESS": 3, "DATE": "2023-07-07"}

{"IS_META_REVIEW": false, "comments": "SUMMARY:\nThis paper introduces a novel framework for embedding watermarks into Large Language Models (LLMs) without compromising their output quality. The proposed watermark is designed to be undetectable by LLM users. \n\nA general framework is put forth for incorporating this unbiased watermark into LLMs. This is achieved using two innovative and practical watermarking techniques: $\\delta$-rewrite and $\\gamma$-reweight.\n\nExperiments conducted on summarization and machine translation tasks demonstrate that these watermarking techniques do not degrade the LLM's output quality, thereby substantiating the effectiveness and practicality of the proposed framework.\n\nSTRENGTHS:\nThis paper tackles a critical issue in the Large Language Models (LLMs) field, which is the misuse of LLMs. The authors propose an unbiased watermark and a novel framework for its implementation. The experimental results clearly prove that this watermarking technique works effectively.\n\nThe paper's layout is good and easy to follow. It uses clear formulas and theorems that help explain both the problem and the proposed solution. Overall, this is a solid piece of work that contributes significantly to the field.\n\nWEAKNESSES:\nThis paper does not provide a comparison with any existing watermark baselines. Such comparisons would be beneficial to demonstrate the relative performance of non-unbiased watermark techniques.\n\nOnly two types of downstream tasks have been evaluated in the study, which limits the generalizability of the findings. It would enhance the robustness of the results if a broader range of tasks, perhaps using a comprehensive benchmark, were tested.\n\nAdditionally, the evaluation of model output quality relies solely on automatic metrics. However, these metrics alone may not be sufficient to provide a comprehensive assessment of output quality. Including more diverse and possibly human-centric evaluation measures or LLM auto evaluator could strengthen the evaluation process.\n\nQUESTIONS:\nCan you provide information on the computational complexity of the proposed watermarking framework?\n\nLIMITATIONS:\nN/A", "TITLE": "Official Review", "RECOMMENDATION": 6, "REVIEWER_CONFIDENCE": 2, "SOUNDNESS_CORRECTNESS": 2, "DATE": "2023-07-07"}