New Evaluation Metrics Capture Quality Degradation due to LLM Watermarking
Authors: Karanpartap Singh
James Zou
Venue: Accepted by TMLR ()

Abstract: With the increasing use of large-language models (LLMs) like ChatGPT, watermarking has emerged as a promising approach for tracing machine-generated content. However, research on LLM watermarking often relies on simple perplexity or diversity-based measures to assess the quality of watermarked text, which can mask important limitations in watermarking. Here we introduce two new easy-to-use methods for evaluating watermarking algorithms for LLMs: 1) evaluation by LLM-judger with specific guidelines; and 2) binary classification on text embeddings to distinguish between watermarked and unwatermarked text. We apply these methods to characterize the effectiveness of  current watermarking techniques. Our experiments, conducted across various datasets, reveal that current watermarking methods are moderately detectable by even simple classifiers, challenging the notion of watermarking subtlety. We also found, through the LLM judger, that watermarking impacts text quality, especially in degrading the coherence and depth of the response. Our findings underscore the trade-off between watermark robustness and text quality and highlight the importance of having more informative metrics to assess watermarking quality.

Published in Transactions on Machine Learning Research (04/2024)
New Evaluation Metrics Capture Quality Degradation due to LLM Watermarking
Karanpartap Singh
kanonps@sttaforddeel
Department of Electrical Engineering Stanford University
James Zou
jamesz(sstanforde edu
Deportment Biomedical Data Science Stanford University
Reviewed on OpenReview: openreveee.......... net/ forum? id= PuhFOhyDgg
Abstract
With the increasing use of largo-language models (LLMs) like ChatGPT, watermarking has emerged as a promising approach for tracing machine geeerrted content. However, research on LLM watermarking often relies on simple perplexity or diversity-based mcasures to assess the quality of watermarked text, which can mask important limitations in watermarking, Here introduce two casy-to-use methods for evaluating watermarking algorithms for LLMs: 1) evaluation by L.M.-judeer with specific guidelines; and 2) binary iassification on text embeddings to distinguish between watermarked and unwatermarked text. We apply these methods to characterize the effectiveness of current walermarking techniques. Our experiments, conducted across various datasets, reveal that current watermarking methods are moderately detectable by even simple classifiers, challenging the notion of watermarking subtlety. We also found, through the LLM judger, that watermarking impacts text quality, especially in degrading the coherence and depth of the response. Our findings underscore the trade-off between watermark robustness and text quality and highlight the importance of having more informative metrics to assess watermarking quality.
1
Introduction
The advancement. of Large Language Models (LLMs) like GPT-4 and Llama-2 has heralded a new era natural language processing, offering unprecedented capabilities in generating human-like text (OpenAl, 2023 Touvron al. 2023 Chowdhery et al.) 2022). However, this advancement also brings forth unique challenge: ensuring the integrity and traceability of machine-generated content (Clark et al. 2021 Mora- Cantallops et al. 2021). These concerns have led to the development of many watermarking techniques for
LLMs, aimed at embedding identifiable markers into generated text without compromising its quality or
readability (Kirchenbaner et al. 2023a b Takezawa et al. 2023 Zhao et al. 2023a Christ et al.
2023 Yoo
et al. 2023 Zhao al. 2023bb.
Watermarking in the context of LLMs is a relatively new and rapidly evolving field. The primary objective is to embed a non-obtrusive, detectable marker within the text generated by these models, enabling the identification of the source model and potentially deterring misuse such as plagiarism or misinformation. Recent advances have introduced sophisticated techniques aimed at embedding watermarks scamlessly into the language model's output through black-box approaches, ensuring minimal distortion or impact to gen- eration quality without access to the original model's weights. Low distortion is desirable because shows that watermarking does not introduce significant. side effocts. However, the effectiveness and subtlety of these methods remain under scrutiny (Tang et al. (202)). Key concerns include the detectability of these watermarks by third parties, their potential to degrade text quality, and the challenge of maintaining the

---

Published in Transactions on Machine Learning Research (04/2024)
GPT-3.5-Turbo
Unwatermarked Preferred
Unwatermarked Text
Watermarked Text
DOOOOO DOOD
ada-002
O
0.893 Watermarked
Binary Classifier
Figure 1: Can watermarked out puts from large language models be distinguished with a black-box approach? We answer this question through two new methods for evaluating LLM watermarks, showing that indepen- dent classifiers and judgers with no prior knowledge of watermarking algorithms prefer or effectively classify watermarked outputs.
watermark's integrity across different contexts and content types. While robustness, the detectability of watermark by the intended, knowledgable party using the proper algorithum, is a positive feature, detectabil- ity by an uninformed third party with no knowledge of the watermarking secret key or algorithm indicates a lack subtletly and potential for security risks.
Prior work has used metrics such as perplexity (Kirchenbauer et al. 2023a), n-gram log diversity (Kirchen- bauer et al. 2023b), or BLEU scores for machine-translation tests (Takezawa et al. 2023) to evaluate the quality impact of watermarking. However, these metrics fail to capture factors such the semantic co- herence or contextual relevance of the generated text. Additionally, watermarking algorithms may deliver similar results on these metrics to unwatermarked text, while still being discernible by other methods.
In this work, we propose two new benchmarks to assess watermarking algorithms for large language mod- cls. We focus our work on determining whether an independent classifier can distinguish a generation as being watermarked without prior knowledge about the watermarking algorithm or associated secret. keys. Detectability is a natural way quantify distortion; the more distorted the generation compared to un- watermarked lext, the easier would be for classifier to delect. Through experimentation across various algorithms and datasets, demonstrate that current watermarking methods indeed be detected by such classifiers. This finding challenges the prevailing notion of watermark subtlety and calls for a reevaluation of current techniques. By exploring the robustness and quality impact of different watermarking algorithms, we aim to advance the field towards developing more effective and less int rusivee watermarking solutions for LLMs.
2 Methods
2.1
Evaluation Mechanisms
We present two evaluation mechanisms to assess the robustness and quality of the outputs generated by the watermarked models:
2

---

Published in Transactions on Machine Learning Research (04/2024)
1. Automated Judgement with GPT / GPT-Judger: We employed GPT-3.5-Turbo with tailored prompt to act as an impartial judge, inspired by [heng et al. (202)), and rank generated outputs (watermarked and unwatermarked) on a 1-5 Likert scale for the following factors, selecting by prompting GPT-4 for an appropriate set of criteria for assessing language model generations: relevance to the prompt, depth of detail, clarity of writing, coherence and logical flow, originality and insight, use of specific examples, and accuracy of information. We chose GPT-3.5-Turile for our primary analyses because of its balance of capabilities and hett/accesibbillty However, to assess the impact of the LLM used, also conducted comparison study between the detailed prompt with Llama-2-7B-chat, SPP--.5-TTrrbe and GPT-4, as well as a simpler prompt without the categorical ranking system with GPT-33.-TTurbs
The prompt instructed the judger to provide specific examples and reasoning for its scoring, as well as a final verdict for which output it preferred overall. Lastly, to account for any positional biases inherent to GPT (Wang et al. 2023a we randomized the order of the outputs presented to the judger. The full prompts and representative responses are included in Appendix
B
2. Binary Classifier: Based on text embeddings obtained using OpenAl's text--embeddinn--dda--00 model from the two outputs (watermarked and unwatermarked), we trained a simple multi-layer perceptron (MLP)-based binary classifier, consisting of 961 neurons in layers, to classify given text. as either unwatermarked or watermarked. We also performed a hyperparameter search for each experiment and dataset to ensure the best performance. More details regarding the network and its training are presented in Appendix C We also tested simple logistic regression on the same embeddings as a classifier, and used k-fold cross-validation random shuflling and 5 folds whenever appropriate.
2.2 Datasets
We tested three datasets in this study. For all datasets, a section of text up to 50 words long was spliced from each sample, after which the billion parameter variant of the Llama-2 model was tasked with completing the output [Touvro al. 2023), both with and without a watermarking layer applied.
LongForm, Validation Set: we used the Wikipedia subset of the LongForm dataset's validation set, consisting of 251 human-written documents various topies (Köksal et al. 2023
2. C4-RealNewsLike,, Validation Set: A subset of the Common Crawl web crawl corpus, the RealNewsLike datasct contains text extracted from online news articles RRaffel et al. 2019 We used 500 samples from this dataset.
3. Scientific Papers, Test Set: A collection of long, structured documents from the arXiv and PubMed open access article repositories (Cohan et al. 2018 We used the abstracts from 252 samples.
2.3 Watermarking Techniques
We examined four distinct watermarking tochniques, the Soft- Soft-Watermark (Kirchenbauer et al. 2023a Ro- bust Distortion-Free Watermark (Kuditipudi et al. 2023), NS- Watermark (Takezawa et al. 2023). and Unigram Watermark (Zhao et al., 2023a) in this study, though our evaluation metrics could be applied to any watermark using a black-box approach, regardless of its complexity other attributes. In selecting these techniques, we prioritized methods that were readily reproducible with published codebascs. We study the soft-watermark for our primary analysis because of its computational elficiency, popularity, and its sim- ilarity to many subsequent methods, such as the NS and Unigram watermarks. Additionally, we also study a very recent distortion-free watermarking technique to illustrate how our metrics can reveal limitations watermarking methods that explicitly aim maintain the original quality in the generated text. We report our observed true and false positive rates for each watermarking method in Appendix A
3

---

Published in Transactions on Machine Learning Research (04/2024)
1. Watt-aammmnrk: Soft-watermarking, involves the pre-sclection of pscudo-random "green" " to- kens before word generation, promoting their use during sampling (Kirchenbauer et al. 2023a This watermark can be detected through a statistical test with knowledge of the secret key and token hashing function used to select the green list. This technique was optimized for improved reliability in Kirchenbauer al., and we use the updated SelfHash scheme in all of our experiments Kirchenbauer et al.) 2023bj. The watermarking parameters remain fixed at (7.6) (0.25, 4.0) unless otherwise stated. These parameters were chosen using recommendations from Kirchenbauer et al. (2023a), along with testing Llama-2-7B to ensure that watermarks with sufficiently high 7-8cores (x=4) were implanted.
2. Robust, Distortion-Free Watermarking: This watermarking technique involves using inverse transform sampling or exponential minimum sampling to embed a watermark into the output of a language model KKddiiipudi et al.) 2023). This is achieved by mapping a sequence of random numbers from a watermark key to the probabilities assigned by the language model to each possible next token in the text. The watermarked text thus encodes information about its source that can be detected by aligning the sequence of tokens with the known sequence of random numbers from the watermark key. This method is meant to be distortion-free by leveraging the inherent randomness in the language model's text. generation process, ensuring that the introduction of a watermark does not change the distribution of the generated text. We use the most robust default (EXP-edit) variant of the watermark provided in the open-source implementation, with the default paramoters (n = 256 for length of the watermark sequence, and key 42 for the secret key). We note that, on the same hardware (1x NVIDIA Tesla GPU), this watermark took between 15-20x the computation time for each generation as compared to the original model or soft-watermart
3. Necessary and Sufficient Watermark: The NS-Watermark can be considered an extension of woft-waaer,,,,ing, and implements an efficient beam-search algorithm to control the proportion of green words in generated texts, and make the resulting watermark strength / z-score as low as possible (necessary) to guarantee a (sufficient) detectable walermark. Given the lower strength of this watermark, it aims to not significantly alter the output distribution of the base LLM. In our testing, we use the default settings in the open-source implementation of y = 0.0001 and a = 1.
variant of the K-gram soft-watermark with K-1, meaning the green and red lists for each token are computed using only the previous token in the sequence and resulting in a consistent green list for each new token that the model generates. This choice is made to emphasize robustness of the watermark to attacks and edits. Theoretical guarantees of the generation quality are also provided. We use the default parameters fraction = 0.5. strength- = 2, a watermarking key of 0, and detection threshold of 4.0.
Unigram Watermark: The Unigram Watermark
3 Experiments
3.1
Detectability of Watermarked Text
Upon evaluation by the GPT-judger, across all samples, the unwatermarked outputs were preferred approx- imately 67.4% of the time, with the watermarked outputs being preferred 25.5% of the time. The remaining samples were declared lie by the judger. This trend held for each individual dataset well, with the un- watermarked text completions being declared better responses for between 64.5% and 74.5% of the samples (Figure 2a). These results indicate the independent judger's ability to distinguish between unwatermarked and soft-watermarked outputs, pointing to quality degradation as a result of the watermark, or other effects of watermarking on the LLM's outputs. They are also in agreement with the winrate of stronger LLMs like GPT-4 versus smaller ones, as evaluated by GPT-based judgers in Zheng et al. (2023). ). To extract more insight from the results of the GPT-judger, we also examined the scores given by the judger to each output.
3.2 Judger Reasoning
Across the judging criteria provided in the prompt, the GPT-judger, on average, gave a higher score to the unwatermarked text. completions (Figure 2b). However, the range of these scores was relatively large.
4

---

Published in Transactions on Machine Learning Research (04/2024)
a
b
Judger Preference by Dataset
Average Judger Scores
1000
Unwatermarked Waternarked
Unwatermarked
Watermarked
judger Preference 18%)
Tie
60
4
40
20
0
Datasets
SclentificPapers
ResiNewsLike
LongForm
0
Coherence
Examples
Depth
Originality
Clarity
Relevance
Accuracy
Dataset
C
d
Average Difference Between Judger Scores
Judger Scores as Classifiers
2.25
Difference Preferred
80
UW
2.00
Average Difference when W Preferred
W
1.75
Tie
60
1.50
Preference
50
0.75
30
0.50
20
0.25
10
0.00
Coherence
Examples
Depth
Originality
Clarity
Relevance
Accuracy
Coherence
Examples
Depth
Originality
Clarity
Relevance
Accuracy
Difference (Score, Score,)
Category
Figure 2: a) GPT-judger preference for the soft-watermark (given as the percentage of samples preferred for each class) for each dataset, separated into samples where unwatermarked outputs were preferred, water- marked samples were preferred, neither categorics provided to the judger for all unwatermarked and watermarked samples (N 1003). Error bars indicate the standard deviation in the scores. e) Average score differences for each evaluation category when either unwatermarked or watermarked samples were preferred. d) Judger preference when using the scores in each category as a classifier. Categories are ordered by the highest preference for unwatermarked samples as compared to watermarked samples or ties.
preferred (tie). b) Average scores for each of the 7 evaluation
Seeking to determine the reasons why certain unwatermarked oulputs were preferred their watermarked counterparts, we looked both the average difference in the scores of the two when either one was preferred, well used the scores in cach category completions was determined to be equal by the judger, indicaling that the watermark does not aflect this attribute of the resulting text (to the best evaluation capabilities of the judger) (Figure 2c). However, a large difference was seen in the coherence scores given to the texts, with both the average differences in the scores and judger preference for this category being significantly higher for the unwatermarked samples (Figure 2c-d). Similar results can be seen for the "use of examples" and "depth" categories, indicating that soft- watermarking tends negatively affect these attributes of the generated text.
classifier. In over 50% of samples, the accuracy of both text
3.3 Judger Comparison
To assess the impact of judger model strength on performance, as well as determine the impact of the category-based ranking system in the prompt, we evaluated four different setups: Llama-2-7B-chat with the ranking-based prompt, GPT-3.5-Turbo with simpler prompt without categorical ranking, GPT-3.5-
5

---

Published in Transactions on Machine Learning Research (04/2024)
a
b
Judger Preference by Model
Similarity Matrix
100
Unwatermarice Watermarked
LLama-2-78, Ranking
0.535
0.596
0.485
Tie
Judger Preference (%)
GPT-3.5, Simple
0.785
40
GPT-3.5, Ranking
0.596
0.765
1
0.727
GPT-4, Ranking
0.485
0.727
LLama-2-78. Ranking
GPT-3.5, Simple
GPT-3.5, Ranking
GPT-4, Ranking
LLama-2
GPT-3.5.
GPT-3.5,
GPT-4, R
Dataset
Figure 3: a) Judger preference when evaluating LLama-2-7B, GPT-3.5-Turbo, and GPT-4, with categorical ranking-based and simple prompts. All results are for subset of N=200 samples from the Rewliis dataset. b) Similarities, given as the percentage of classifications that were the same, between all judgers. The models on the x-axis are abbreviated for brevity but follow the same sequence as those on the y-axis.
Turbo with the ranking prompt, and lastly, GPT-4 with the ranking prompt. Both prompts are included in Appendix B
The results, shown in Figure 3, indicate that even a small model, Llama-2-7B, shows a preference for the unwatermarked model versus the watermarked model. It is worth noting that Llama-2 was significantly worse adhering to the result format required by the prompt, and required repeated sampling obtain parseable rosults. Upgrading to GPT-3.5-Turbo the simpler prompt, which only required the judger to output binary verdict for its preferred completion with reasoning, delivered nearly identical results to the ranking- banking-based prompt. However, the addition of the categorical ranking allowed us to extract further insights from the judger and determine which aspects of the generated text suffered the most from watermarking, GPT- showed an even higher preference 75% for the unwatermarked completions, howevcr at a substantial increase cost. These results indicate that larger models are more discerning the differences bet ween unwatermarked and watermarked outputs, and may extend to newer and more capable models such GPT-4-Turbo.
All of the GPT-based judgers agreed with each other over 71% of the time, with the highest agreement of 76.5% being between the GPT-3.5-Ture Turboo results for the simple and categorical ranking prompts. The Llama- 2-based judger agreed with the GPT-based judgers for roughly half of the samples. To further evaluate the GPT judger, we also conducted human evaluation study where an evaluator assessed 50 randomly selected samples. Each sample consisted of the prompt and a pair of watermarked and unwatermarked generations, with blinded human evaluator. The human evaluator preferred the unwatermarked output for 60% of the samples, agreeing with the GPT-3.5-Turlo judger 70% of the time, indicating significant overlap in preferences between the human and CPPT-based judgers.
3.4 Binary Classifier Performance
To further test whether watermarking discernibly altered the generated text, we employed two classification methods:
4-layer neural network (NN) and simple logistic regression. These classificrs were trained distinguish between watermarked and unwatermarked texts using eext-embbediig-aade embeddings obtained from both texts.

---

Published in Transactions on Machine Learning Research (04/2024)
The neural network classifier showed promising results, yielding average accuracy of 71% when trained all of the datasets, along with an AUC of 0.75, false-unwatermarken rate of 30.8%, and false-watermarkey rate of 30.4%. This level of accuracy was consistent, even when the model was trained on one dataset and tested on the other two, highlighting the network's ability to generalize and recognize watermarking patterns without explicit. knowledge of the watermarking techniques or access to the secret. key (Table !
Logistic regression, a simpler classification method, was also employed an ablation study determine if watermark detection was feasible with more basic techniques. This classifier achieved an accuracy just above random guessing, at approximately 56%, across various datasets using k-fold cross- voss-validation with 5 folds. Notably, its highest accuracy, approximately 60%, was observed when trained on the RNL (Real- NewsLike) dataset and tested on the others, likely due to the larger sample size of 500 in the RNL dataset. Despite logistic regression's 's relatively lower performance compared to the MLP-based classifier, these results nevertheless suggest the presence of a watermarking signal the texts (Table 1
Table 1: Binary classifier (neural- network based) accuracy, AUC, false positive (FP) rate, false negative (FN) rate, and regression accuracy on each dataset for the wott-waaerm.rk. When evaluating all of the datasets logether, k-fold cross-validattin trained the indicated dataset, and tested for generalizability the other two datascts.
used with 5-folds. For the three individual datasels, each algorit
All Datascts
ScientificPapers
LealNewsLike
LongForm
Accuracy
0.711
0.628
0.649
0.611
AUC
0.75
0.665
0.732
0.724
FP
0.304
0.491
0.507
0.670
FN
0.308
0.262
0.200
0.121
Regression
0.562
0.573
0.597
0.515
3.5
Detectability vs. Watermarking Parameters
a
b
Accuracy Watermark Strength
Judger Score Difference Watermark Strength
100
Judge Choice Unwatermarked (%)
Relevance
Classifier Accuracy
Regression Accuracy (%)
2.5
Clerity
Coherence
Originaiity
Parcentage (%)
2.0
Examples
W
Accuracy
UW
1.0
0.5
6-2
6m4
6-6
6-2
5m4
6-8
Avg Z-Score: 6.691 Avg Z-Score: 12.755
Avg Z-Score: 16.203 Avg Z-Score: 6.691
Z-Score: 12.755
Avg Z-Score: 16.203
Watermark Strength (Delta)
Watermark Strength (Delta)
Figure 4: a) Judger preference, classifier accuracy, and regression accuracy for various sofl-wadermark strengths (8 = (2, 4, 8)), evaluated across all of the datasets with k-fold cross-validation (5 folds). Larger 8 corresponds to stronger watermarking, b) Average judger score differences for cach category when varying the watermark strength. (All) error-bars represent the standard error in the measurement.
Watermarks for large language models present an inherent tradc-off between robustness, computational cost, and text quality (Kirchenbauer et al. 2023a b). We therefore evaluated the impact of the watermark strength the accuracy of our techniques. For both of the datasets tested, the judger's preference

---

Published in Transactions on Machine Learning Research (04/2024)
a
b
Judger Preference by Dataset
Classifier Performance by Dataset
100
Unwatermarked
Classi fier Regression
Watermarked
80
(%)
Accuracy 196
60
40
20
20
0-
All Datasets
ScientificPapers
RealNewsLike
Longform
Dacasets
ScientilicPapers
RealNewsLike
LangForm
Dataset
Dataset
C
Judger Scores as Classifiers Distortion-Free Free Watermark
70
60
W
Tie
Preference (80
50
Coherence
Examples
Depth
Originaliiyy
Clarity
Relevance
Accuracy
Category
Figure 5: a) GPT-judger preference for the distortion-free watermark (given as the percentage of samples pre- ferred for each class) for each dataset, separated into samples where unwatermarked outputs were preferred, watermarked samples were preferred, or neither was preferred (lie). b) Binary classifier (newrkl-network based and regression) performance each dataset for the distortion-free watermark. k-fold cross-validation was used with 5-folds when evaluating all of the datascts. For the three individual datasets, cach algorithm was trained on the indicated dataset, and tested on the other two. c) Judger preference when using the scores each category as a classifier.
for the unwatermarked output and classifier's accuracy in predicting whether an output was watermarked increased as the watermark was made stronger (Figure 5a). At the lowest tested delta parameter of = 2, the watermark had sufficient strength to be flagged as watermarked, but was not discernible by the judger or classifier. Additionally, as the watermarking strength was increased, the difference between the judger's scores for the unwatermarked and watermarked outputs also increased in all categories, with the largest difference being seen for the depth of the outputs (Figure 5b).
3.6 Other Watermarking Algorithms
Lastly, evaluated both techniques on three other watermarking algorithms, the Robust Distortion-Free Watermark proposed by Kuditipudi et al. (Kuditipudi al.) 2023). the Unigram watermark from Zhao al. (Zao et al.) 2023a), and the NS-Watermark by Takezawa et al. (Takozawa et al. 2023 to test the generalization of our techniques. As shown in Table 2 our results generalized across all of the techniques, with at least. one method achieving over 75% accuracy for each watermark.
Taking closer look the robust distortion-free watermark, the judger showed very similar preference of 67.5% for the unwatermarked samples and 25.5 25.3% for watermarked samples, with the remaining 7.2% of samples being declared ties (Figure 3.5a). On the ScientificPapen dataset, the judger preferred 84% of the

---

Published in Transactions on Machine Learning Research (04/2024)
Table 2: Judger preference and classifier accuracies for each of the four watermarking algorithms we tested (soft, distortion-free, unigram, and ns-watermarks), computed over 200 samples from the RealNowsLike dataset.
DISTORTION-FREE UNIGRAM
NS
GPT-Judger Preferred UW
0.72
0.73
0.60
Classifier
0.87
0.67
0.57
Logistic Regression Classifier
0.65
0.78
0.76
unwatermarked samples, indicating an even larger degradation in quality for this class of academic text generation. Once again, the judger's decisions were most influenced by the lack in coherence, depth, and usage of examples in the watermarked text (Figure 3.5b).
Meanwhile, the MT.P-based classifier achieved above 80% accuracy and 0.926 AUC in discerning watermarked samples when evaluated on all of the datasets, with linear regression also displaying detection accuracy near 75% when trained on each individual datasel (Figure 3.5p). Collectively, these results indicate that the robust, distortion- watermark also perceptibly affects generation quality.
4 Discussion
Related Works In the evolving landscape of large language model applications, watermarking has emerged as crucial technique for tracing model outputs. Most current LLM watermarking approaches involve subtly biasing the model's logits using pseudorandom distributions. These techniques vary, ranging from simple binary partitioning with 'green' and 'red' lists to more sophisticated methods. In parallel, techniques like
the LLM-judger have emerged as valuable tools for comparing and benchmarking LLMs (Zheng et al. 2023
or generating annotations for instruction tuning (Wang et al. 2023b Peng et al. 2023 Zhou et al. 2023
In this study, we introduced two new techniques for evaluating watermarks for large language models. Fundamentally, watermarking for LLMs should remain invisible to both automated systems and human evaluators. Our study reveals that, contrary to this ideal, current watermarking techniques, including soft- watermarking and the EXP watermark, introduce detectable patterns anomalies into the generated text. The ability of independent classifiers to detect watermarked content in LLMs without prior knowledge of the specific watermarking algorithın or secret keys is notable finding. The effectiveness of simple classifiers like logistic regression and multi-layer perceptrons, achieving over 70% and up to 86.5% accuracy in identifying watermarked content, further underscores this point. This finding not only questions the non-detectability of these watermarks but also suggests that in practice, even methods designed to be distortion-free still suffer from degradations. These results contrast. with prior work, which showed comparable results ween watermarked and unwatermarked text on metrics such as perplexity (Zhao et al. 2023a) and BLEU scores Takezawa et al. 2023). The ideal watermark should balance robustness (detectability by the intended party and resilience to tackk)) and subtlety (not discernible by outside methods that don't 't know the watermarking key or algorithm).
The GPT-judger's scoring further adds a dimension to this understanding by highlighting the specific areas of text. quality that are impacted by current watermarking methods. Watermarking, as we observed, tends to degrade text attributes like coherence and depth, whereas internal accuracy remains generally consistent. This degradation is crucial as it can compromise the utility and acceptability of watermarked texts in settings where high-quality outputs are paramount. Our results also shed light the trade-off between watermark strength and text quality. As watermark strength increases, so does its detectability, indicating more pronounced quality degradation. With a particularly robust watermark, we see detection accuracies over 90%, all without any priors regarding the watermarking algorithm and with relatively small sample sizes used for training.
The ability to detect watermarks in texts generated using various watermarking methods suggests that subtlety might be a crucial characteristic to consider when developing these methods. Looking ahead, our
9

---

Published in Transactions on Machine Learning Research (04/2024)
work opens new avenues for research. Future studies could employ more sophisticated classifiers, delve deeper into the nuances of how watermarking alters text generation, and ultimately use these insights to develop new watermarking techniques that implant sufficienlyy robust watermarks without altering the generated text perceptibly from the original model.
In conclusion, our findings provide an overview of the current state of watermarking in LLMs. The challenges in achieving biietectability and maintaining text quality are more pronounced than previously understood. As LLMs become increasingly prevalent, developing watermarking techniques that are both robust and subtle is crucial.
Code Availability: The source code for all experiments is available at hhttp:://giiee /ttt/bb..ttuu...../ thub. cww/su-kaaeeeeeeooee watermark, eval
References
Aakanksha Chowdhery, Sharan Narang, Jacob Devlin, Maarten Bosma, Gaurav Mishra, Adam Roberts, Paul Barham, Hyung Won Chung, Charles Sutton, Sebastian Gehrmann, Parker Schuh, Kensen Shi, Sasha Tsvyashchenko, Joshua Maynez, Abhishek Rao, Parker Barnes, Yi Tay, Noam Shazeer, Vinodkumar Prabhakaran, Emily Reif, Nan Du, Ben Hutchinson, Reiner Pope, James Bradbury, Jacob Austin, Michael Isard, Guy Gur-Ari, Pengeheng Yin, Toju Duke, Anselm Levskaya, Sanjay Ghemawat, Sumipa Dev, Henryk Michalewski, Xavier Garcia, Vedant Misra, Kevin Robinson, Liam Fedus, Denny Zhou, Daphne Ippolito, David Luan, Hyeontaek Lim, Barret Zoph, Alexander Spiridonov, Ryan Sepassi, David Dohan, Shivani Agrawal, Mark Omernick, Andrew M. Dai, Thamumalayan Sankaranarayana Pillai, Maric Pellat, Aitor Lewkowycz, Erica Moreira, Rewon Child, Oleksandr Polozov, Katherine Lee, Zongwei Zhou, Xuezhi Wang, Brennan Saeta, Mark Diaz, Orhan Firat, Michele Catasta, Jason Wei, Kathy Meier-Hellstern. Douglas Eck, Jeff Dean, Slav Petrov, and Noah Fiedel. PaLM: Scaling Language Modeling with Pathways. 4 2022.
Miranda Christ, Sam Gumn, and Or Zamir. Undetectable Watermarks for Language Models. 2023.
Elizabeth Clark, Tal August, Sofia Serrano, Nikita Haduong, Suchin Gururangan, and Noah A. Smith. All That's 'Human' Is Not Gold: Evaluating Human Evaluation of Generated Text. 6 2021.
Arman Cohan, Franck Dernoncourt, Doo Soon Kim, Trung Bui, Seokhwan Kim, Walter Chang, and Nazli Goharian. A Discourso-Awree Attention Model for Abstractive Summarization of Long Documents. 4 2018.
John Kirchenbauer, Jonas Geiping, Yuxin Wen, Jonathan Katz, lan Miers, and Tom Goldstein. A Watermark for Large Language Models. 2023a.
John Kirchenbauer, Jonas Geiping, Yuxin Wen, Manli Shu, Khalid Saifullah, Kezhi Kong, Kasun Fernando, Aniruddha Saha, Micah Goldblum, and Tom Goldstein. On the Reliability of Watermarks for Targe Language Models. 2023b.
Abdullatif Köksal, Timo Schick, Anna Korhonen, and Hinrich Schützo. LongForm: Optimizing Instruction Tuning for Long Text Generation with Corpus Extraction. 2023.
Rohith Kuditipudi, John Thickstun, Tatsunori Hashimoto, and Percy Liang. Robust Distortion Water- marks for Language Models. 2023.
Marçal CantaCaas,, Salvador Sánchez-Alonso, Elena García- Barriocanal and Miguel-Angel Sicilia. Traceability for Trustworthy AI: A Review of Models and Tools. Big Data and Cognitive Computing. 5(2):20, 2021. ISSN 2504-2289. doi: 10.3390/bdcc5520020.
OpenAl. GPT-4 System Card, 3 2023.
Baolin Peng, Chunyuan Li, Pengcheng He, Michel Galley, and Jianfeng Gao. Instruction Tuning with GPT-4. 4 2023.
10

---

Published in Transactions on Machine Learning Research (04/2024)
Colin Raffel, Noam Shazeer, Adam Roberts, Katherine Lee, Sharan Narang, Michael Matena, Yanqi Zhou, Wei Li, and Peter J. Liu. Exploring the Limits of Transfer Learning with a Unified Text-to-Text Trans- former. 10 2019.
Yuki Takezawa, Ryoma Sato, Han Bao, Kenta Niwa, and Makoto Yamada. Necessary and Sufficient Water- mark for Large Language Models. 10 2023.
Ruixiang Tang, Yu-Neng Chuang, and Xia Hu. The Science of Detecting LLM-Generated Texts. 2 2023.
Hugo Touvron, Louis Martin, Kevin Stone, Peter Albert, Amjad Almahairi, Yasmine Babaei, Nikolay Bash- lykov, Soumya Batra, Prajiwal Bhargava, Shruti Bhosale, Dan Bikel, Lukas Blecher, Cristian Canton Fer- rer, Moya Chen, Guillem Cucurull, David Esiobu, Jude Fernandes, Jeremy Fu, Wenyin Fu, Brian Fuller, Cynthia Gao, Vedamuj Goswami, Naman Goyal, Anthony Hartshorn, Saghar Hosseini, Rui Hou, Hakan Inan, Marcin Kardas, Viktor Kerkez, Madian Khabsa, Isabel Kloumann, Artem Korenev, Punit Singh Koura, Marie-Anne Lachaux, Thibaut Lavril, Jenya Lee, Diana Liskovich, Yinghai Lu, Yuning Mao, Xavier Martinet, Todor Mihaylov, Pushkar Mishra, Igor Molybog, Yixin Nie, Andrew Poulton, Jeremy Reizenstein, Rashi Rungta, Kalyan Saladi, Alan Schelten, Ruan Silva, Eric Michael Smith, Ranjan Subra- manian, Xiaoqing Ellen Tan, Binh Tang, Ross Taylor, Adina Williams, Jian Xiang Kuan, Puxin Xu, Zheng Yan, Nlyan Zarov, Yuchen Zhang, Angela Fan, Melanie Kambadur, Sharan Narang, Aurelien Rodriguez, Robert Stojuic, Sergey Edunov, and Thomas Scialom. Llama 2: Open Foundation and Tunee Chat Models. 7 2023.
Peiyi Wang, Lei Li, Liang Chen, Zefan Cai, Dawei Zhu, Binghuai Lin, Yunbo Cao, Qi Liu, Tianyu Liu, and Zhifang Sui. Large Language Models are not Fair Evaluators. 5 2023a.
Yidong Wang, Zhuohao Yu, Zhengran Zeng, Linyi Yang, Cunxiang Wang, Hao Chen, Chaoya Jiang, Rui Xie, Jindong Wang, Xing Xie, Wei Ye, Shikun Zhang, and Yue Zhang. PandaLM: An Automatic Evaluation Benchmark for LLM Instruction Tuning Optimization. 6 2023b.
KiYoon Yoo, Wonhyuk Ahn, Jiho Jang, and Nojun Kwak. Robust Multi-bit Natural Language Watermarking through Invariant Features. 5 2023.
Xuandong Zhao, Prabhanjan Ananth, Lei Li, and Yu-Xiang Wang. Provable Robust Watermarking for Al-Generated Text. 6 2023a.
Xuandong Zhao, Yu-Xiang Wang, and Lei Li. Protecting Language Generation Models via Invisible Water- marking. 2 2023b.
Lianmin Zheng, Wei-Lin Chiang, Ying Sheng, Siyuan Zhuang, Zhanghao Wu, Yonghao Zhuang, Zi Lin, Zhuohan Li, Dacheng Li, Eric. P Xing, Hao Zhang, Joseph E. Gonzalez, and lon Stoica. Judging LLM- as-a-Judge with MT-Bench and Chatbot Arena. 6 2023.
Chunting Zhou, Pengfei Liu, Puxin Xu, Srini Tyer, Jiao Sun, Yuning Mao, Xuezhe Ma, Avia Efrat,, Ping Yu, Lili Yu, Susan Zhang, Gargi Ghosh, Mike Lewis, Luke Zettlemoyer, and Omer Levy. LIMA: Less Is More for Alignment. 5 2023.
11

---

Published in Transactions on Machine Learning Research (04/2024)
A
Appendix A: Watermarking Algorithms
Watermark Strength:
Table 3: True and false positive rates for each of the four watermarking algorithms tested (soft, distortion- free, unigram, and ns-watermarks), computed over 200 samples from the RealNewsLike dataset.
SOFT DISTORTION-FREE UNIGRAM
NS
False Positive Rate
0.015
0.005
0.01
True Positive Rate
0.96
0.67
0.535
1.0
12

---

Published in Transactions on Machine Learning Research (04/2024)
Example Response Set:
Table 4: Example response set for the soft and distortion-free watermarks. Incoherent sections of the watermarked outputs that likely contribute to the judger's decision are highlighted. For longer such sections, only the first sentence is highlighted for visibility. DF-Watermark refers to the distortion-free watermark.
PROMPT
UNWATERMARKED SOFT-WATERMARE DE-WATERMARK
A little over acres of land in a prime area of Western North Carolina. Upon arrival to the home site, you'll pass gorgeous mountain views, babbling creek and pasture land. This property has a storage shed already in place and the land is ready for your use which
makes it easy to start building your
could be a home- stead, farm, or a place to park your the
includes double rural zoning. With a little creativity and 8ome elbow grease this property can be turned into your very own piece of paradisc. Sunset views are nothing short breathtal- ing. Location just minutes from the nearby cities of Mar- shall, Bryson City and Hayesville. This property is priced to sell and won't last long. Call today and schodule a viewing! When first meet Scott Morrison, he is sitting in his office. He is wearing a sky blue shirt with a hint of a smile, looking worn and exhausted. He has been awake for days...
dream home. The
property is located quiet and peaceful area with a short distance to the Blue Ridge Parkway and all the outdoor activ- ities has to offer. The nearby town of Spruce Pine offers shopping, dining and entertainment options. Don't miss out on this amazing opportunity to own a little piece of heaven.
RV. Close Blue Ridge parkway, Boone and Watauga river making this a prime location for outdoor enthusiasts or just great place to unwind. Property being sold with disclosures warranties. Seller would like a 30 day inspection period, and a cash deal would be the most likely outcome, however would discuss with the right offer. Agent is related to the seller. This is great chance to live the country life close to everything.
Directions: From the intersection of Hwy 221 and Hwy 70 in Spruce Pine, take Hwy 70 East for approximately miles. Property will be the right just. past the Blue Ridge Parkway Overlook. Look for the sign.
Scott Morrison is the 30th Prime Minister Australia, serving since August. 2018. Since his appoint- ment, he has been at the forefront of national issues, from climate change to the economy... and deal- ing with the fallout of the Kavenaugh affair.
13

---

Published in Transactions on Machine Learning Research (04/2024)
B Appendix B: Judger
Judger Prompt and Example Output:
Table 5: Judger prompt and completion from GPT-3.5-Turbo for the example response set given in Table g In this example, the randomization resulted in LLM A being the watermarked completion, and LLM B being the unwatermarked completion.
PROMPT
JUDGER RESPONSE
Please act as an impartial judge and evaluate the quality of the text completions provided by two large language models to the prompt displayed below. As- sess each response according to the criteria outlined, using a 1-5 Likert scale. After scoring each criterion. provide summary for each response, including examples that inffuenced your scoring. Additionally, ensure that the order in which the responses are presented does not affect your decision.
In my assessment, the superior response is from LLM B.
Relevance to the prompt:
LLM A:
LLM B:
LLM B's response directly addresses the prompt by emphasizing the opportunity to build dream home and providing directions to the property. In contrast, LLM A's response focuses more on general information about the location and potential uses of the land, which is somewhat relevant but not as specific.
Criteria:
1. Relevance to the prompt
Depth of detail
3. Clarity of writing
2. Depth of detail: LLM A:
4. Coherence and logical flow
LLM B: 3
5. Originality and insight
LLM A provides more details about the property, such mentioning the presence of mountain views, a babbling creek, pasture land, and storage shed. On the other hand, LLM B's response lacks specific details about the land itself, instead focusing more on the nearby town and activities.
6. Use of specific examples
7. Accuracy of information
Start with brief statement about which response you think is superior. Then, for each response and criterion, provide score, followed by a brief justification for that score. At the vory end your response, declare your verdict by choosing one the choices below, strictly following the given format:
"[[A]]: [list of scores]" if assistant A is better,
7. Accuracy of information:
[[B]: list of scores]" if assistant B is better,
-LLM A:
"[[C]] for tie.
-LLM B:
Both responses provide information that does require fact-checking. They introduce factual inconsistencies.
[Prompt]
LLM A's Answer]
Based on the evaluation above, my verdict is:
LLM B's Answer]
[B]]: 3, 3, 5, 5, 4,
14

---

Published in Transactions on Machine Learning Research (04/2024)
Table 6: Simple judger prompt and completion from GPT-3.5-Turbo for the example response set given in Table In this example, the randomization resulted in LLM A being the unwatermarked completion, and LLM B being the watermarked completion.
SIMPLE PROMPT
JUDGER RESPONSE
Please act as an impartial judge and evaluate the qual- ily of the text completions provided by two large lan- guage models to the prompt displayed below. You should choose the LLM that completes the prompt better. Begin your evaluation by comparing the two responses and provide short explanation. Avoid any position biases and ensure that the order in which the responses were presented does not influence your de- cision. Do not allow the length of the responses lo influence your evaluation. Be objective possible. After providing your explanation, output your final verdict by strictly following this format: "[[]]" if as- sistant A is better, "[[]]] if assistant B is better, and "[[C]]" for tie. [Prompt]
The response provided by LLM A provides more specific information about the property and its potential uses. It mentions that the land could be used as homestead, farm, or place to park an RV. also highlights the proximity to the Blue Ridge Parkway, Boone, and Watauga River, appealing to outdoor enthusiasts. Additionally, it mentions that the property is being sold as with no disclosures warranties, and that cash deal the most likely outcome.
On the other hand, LLM B's response is focused more on the location and nearby amenities. Tt mentions that the property is located a quiet and peaceful area, with nearby access to the Blue Ridge Parkway and various outdoor activities. It also briefly mentions the nearby town of Spruce Pine and the options for shopping, dining, and entertainment. The response concludes with directions to the property from a specific intersection.
LLM A's Answer]
LLM B's Answer]
Overall, both responses provide relevant information, but LLM A's response provides more details about the property and its potential uses. Therefore, based on the evaluation, [[A]] is better.
15

---

Published in Transactions on Machine Learning Research (04/2024)
C
Appendix C: Binary Classifier
The classifier is multi-layer perceptron (MLP) with the following architecture:
Input size: 1536
Output size: 1
Hidden layers: 512, 256, 128, 64
Activation: ReLU between layers, sigmoid at the output
Optimization performed using the Adam optimizer with: 81 0.5, B2 = 0.999, learning rale and weight decay determined through hyperparameter grid search (parameters shown in Table C A dynamic 1rrrim.._sschhuueerr
learning rate scheduler was used (torch. oethhh.pmi..... .r scheduler ReduceLROnPlateau) with a factor of 0.5 and a patience of 50 epochs. Training was conducted for 150 epochs.
Hyperparameter Adam Weight Decay
Value
Grid search over {2e 2e 2c 21
Learning Rate
Grid search over (2e-5 2e
2e 33-3
Batch Size
Grid search over (50, 75, 100}
Dataset Randomization
Grid search over {On, Off)
Table 7: Binary classifier hyperparamelers
16


Reviews:
[{"review_id": "5LMEmVEuqV", "reviewer": "Reviewer_Srd4", "summary": "", "questions": "", "limitations": "", "rating": null, "confidence": null, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "Strength: \n\n1) This paper focuses on a very important problem. Watermarking is a very important technique to enable tracing the IP of AI models. Exploring something regarding watermarking is always important at the current era.\n\n2) This paper demonstrates findings clearly, readers can easily understand the proposed two methods. \n\nWeaknesses:\n\nThe main issue of this paper is the ignore of the significance of existing watermarking methods. Lowering the quality is easy to understand. However, easy-to-detect might not be a drawback? The logic chain here is broken, and literature is used to force readers to believe this claim. In my opinion, a new section should be added to convince readers to believe that the two issues are significant in the field of watermarking. By doing so, it can also increase the significance of this paper in the field.", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "Yes", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "paper_topic_and_main_contributions": "", "excitement": null, "reproducibility": null, "ethical_concerns": "", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "summary_of_contributions": "This paper proposes two easy-to-use methods to evaluate existing watermarking methods. It argues that existing watermarks can be easily detected and lower the quality of the original text. The total content is clear, and the methods are introduced well. However, some key issues are identified in this paper, which might need to be fixed before the final acceptance.", "claims_and_evidence_yn": "Yes", "claims_explanation": "", "audience_yn": "No", "audience_explanation": "", "requested_changes": "Weaknesses:\n\nThe main issue of this paper is the ignore of the significance of existing watermarking methods. Lowering the quality is easy to understand. However, easy-to-detect might not be a drawback? The logic chain here is broken, and literature is used to force readers to believe this claim. In my opinion, a new section should be added to convince readers to believe that the two issues are significant in the field of watermarking. By doing so, it can also increase the significance of this paper in the field.", "broader_impact_concerns": "N/A", "additional_comments": "", "extra_scores": {}, "extra_text": {}}, {"review_id": "UxmTKVjdQv", "reviewer": "Reviewer_41vW", "summary": "", "questions": "", "limitations": "", "rating": null, "confidence": null, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "## Strength\n* This paper is well-written and easy to read.\n* This paper proposed two evaluation methods, and the experimental results are solid. Regarding the undetectability of watermarking methods, this paper demonstrated that even the DF-Watermark, which has the distortion-free property, is distinguishable from unwatermarked texts. \n\n## Weakness\n* This paper claimed that the existing evaluation metrics for watermarking methods, e.g., perplexity, can not capture the semantic coherence or contextual relevance of generated texts and then proposed the evaluation methods using GPT. However, the reviewer feels that this advantage of the proposed method over the existing evaluation metric is unclear. For instance, how do the results of evaluation by the proposed method, Automated Judgement with GPT, differ from the results by perplexity and other existing evaluation metrics? A similar result that the insertion of watermarks degrades the text quality has already been shown in a prior study using perplexity [1]. It would be better for the paper to clearly describe when the proposed evaluation method should be used instead of existing evaluation metrics. Furthermore, it would be better to discuss what different experimental results can be obtained using the proposed evaluation method.\n\n* This paper focused on the evaluation metrics for watermarking methods and proposed the new evaluation metric, Automated Judgement with GPT. However, it is a very fundamental topic in NLP to measure the correctness of generated texts, and there exist a lot of existing methods, including perplexity, BLUE score, and LLM-based methods (e.g., [2]). Are there any issues that are specific to evaluating generated texts with watermarks? The reviewer did not understand the motivation to propose a new evaluation metric to assess text quality degradation caused by watermarking.\n\n## Reference\n[1] Kirchenbauer et al., A Watermark for Large Language Models, ICML 2023\n\n[2] Zheng et al., Judging LLM-as-a-Judge with MT-Bench and Chatbot Arena, NeurIPS 2023", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "Yes", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "paper_topic_and_main_contributions": "", "excitement": null, "reproducibility": null, "ethical_concerns": "", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "summary_of_contributions": "This paper proposed two evaluation methods for watermarking methods. First, this paper proposed Automated Judgement with GPT to evaluate how text quality degrades by watermarking methods and then evaluated the existing watermarking methods by using the proposed methods. This paper also proposed a method to evaluate the undetectability of watermarking methods by using supervised learning. Then, this paper demonstrated that the generated texts with existing watermarks are distinguishable from unwatermarked texts with high accuracy.", "claims_and_evidence_yn": "Yes", "claims_explanation": "", "audience_yn": "Yes", "audience_explanation": "", "requested_changes": "* Could you please make the code public? Making the code public is very important for reproducibility, especially for the evaluation metric.\n* Sec. 2.3 explained the hyperparameter setting for each method, but does not report the false positive rates of watermarking methods. These values are very important to measure the strength of watermarks. Could you please report the false positive rate in Sec. 2.3?\n* Please see other comments listed in the weakness section.", "broader_impact_concerns": "N/A", "additional_comments": "", "extra_scores": {}, "extra_text": {}}, {"review_id": "1W4HEuh9c8", "reviewer": "Reviewer_CBc5", "summary": "", "questions": "", "limitations": "", "rating": null, "confidence": null, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "## Strengths\n\n- Novelty: This work seems to investigate the possibility of LLM-as-a-judge for the quality of watermarked texts as far as I surveyed.\n- Proposing factors to characterize text quality: In this paper, seven factors (coherence, clarity, accuracy, ...) to characterize text quality are proposed. These factors are reasonable, and surprisingly, we see significant score differences (between the cases when watermarked and unwatermarked texts are preferred) only for some factors, such as coherence and originality (Figure 2c). This may partially describe the characteristics of some watermarking methods.\n- Simplicity of proposed methods: Both the GPT-judger and binary classification are very simple and not costly to use in practice.\n\n## Weaknesses\n\n- Limited numbers of baseline watermarks: The authors evaluated only two watermark methods, Soft-Watermark and Distortion-Free Watermark. Though they discuss why they chose these two methods to evaluate (and perhaps mainly because of the availability of public implementation), some other works (cited in this paper) publicize their implementation. For instance, [Takezawa et al. (2023)](https://github.com/yukiTakezawa/necessary_and_sufficient_watermark) (not included in their paper, though) and [Zhao et al. (2023a)](https://github.com/XuandongZhao/Unigram-Watermark) provide their codebases. Evaluation using these other methods is important to assess the robustness of the proposed evaluation methods (i.e., how universally we observe the tendency in Fig. 2) across different watermarking methods.\n- Insufficient detection accuracy: With both GPT-judgers and binary classifiers, the detection performances of watermarked texts remain up to 70% accuracy (unless we increase the watermark intensity $\\\\delta$ of the Soft-Watermark). This observation is interesting because 70% accuracy cannot be regarded as practically reliable enough, and hence the existing watermarking methods partially succeed in being undetectable. For this reason, I don't think we can confidently claim that \"current watermarking methods are detectable by even simple classifiers\" (in the abstract) or \"the presence of a detectable watermarking signal in the texts\" (in the end of Section 3.4).", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "Yes", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "paper_topic_and_main_contributions": "", "excitement": null, "reproducibility": null, "ethical_concerns": "", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "summary_of_contributions": "To evaluate watermarked texts generated by LLMs, the authors provide two methods. One is automated judgment with GPT: By using carefully designed prompts, we let GPT evaluate text quality from several factors with the Likert scale, such as coherence, use of examples, depth of detail, originality, clarity, relevance to the prompt, and accuracy of information. The other is detection by binary classifier with multi-layer neural networks or simple logistic regression built on top of text-embeding-ada-002. In the experiments, they found that both the GPT-judgers and binary classifiers can detect watermarked texts with around 70% accuracy on average. This detectability is enhanced with more intense watermarks. In the evaluation by the GPT-judgers, they specifically found that watermarked and unwatermarked texts obtained large score gaps for coherence, use of examples, depth of details, and originality, while there were no significant score gaps for clarity, relevance, and accuracy.", "claims_and_evidence_yn": "Yes", "claims_explanation": "", "audience_yn": "Yes", "audience_explanation": "", "requested_changes": "## Major comments\n\nAs I discussed above, the potential suggestion is to add a few more baseline watermarks (in order to see the robustness of the proposed evaluation methods).\n\nAdditionally, I came up with several spontaneous ideas to extend the content of this paper. You can have a look and incorporate them into the paper.\n\n- Ensemble classifier for watermark detection: Though the GPT-judger and neural network classifiers are not powerful enough, as I criticized above, it makes sense to regard them as weak learners and apply ensemble methods or boosting. It must be interesting to see how powerful the detection accuracy will be.\n- Ensemble score with the GPT-judger: Currently, the GPT-judger only adopts a single LLM as a judger and lets it output the Likert scales. We can probably use multiple LLMs and average their Likert scales. Such an ensemble method may provide a more stable evaluation.\n\n## Minor comments\n\n- In Section 2.1 (the fourth line of \"1. Automated Judgement with GPT\"), should \"GPT-4\" be \"GPT-3.5-Turbo\"?\n- For typesetting double quotation marks, use `` ... '', that is, two backquotes and two single quotes.\n- In Section 3.1, the term \"GPT-judger\" appears for the first time. I suppose this indicates the automated judgment with GPT introduced in Section 2.1. It is better to introduce the terminology formally.\n- In Section 3.4, it is better to summarize the AUC scores and so on in a table. Currently, those watermark detection performances scatter here and there.\n- In Section 3.5 (the fourth last line), it seems strange to say \"the watermark was still robust enough to be detected\" because the detection accuracy is only around 50% (chance rate) with $\\\\delta=2$.", "broader_impact_concerns": "LLM watermarking is undoubtedly a vital technique to prevent false information and plagiarism from machine-generating texts. The authors discussed this perspective slightly in the introduction.", "additional_comments": "", "extra_scores": {}, "extra_text": {}}]

Author rebuttal:

Meta-review: The paper introduces two evaluation methods for assessing watermarked texts: Automated Judgment with GPT and binary classifier detection. Using GPT prompts, text quality factors like coherence, originality, and accuracy are assessed. Binary classifiers, leveraging text embeddings, detect watermarks effectively. Results show both methods achieve around 70% accuracy. This detectability is enhanced with more powerful watermarks.

The LLM watermarking is an emerging topic, with numerous methods being developed. The proposed approach is both intriguing and practical. All reviewers have recommended acceptance, and I also vote for acceptance.
Decision: Accept (poster)