W ater B ench: Towards Holistic Evaluation of Watermarks for Large Language Models (2024)

Abstract: To mitigate the potential misuse of large language models (LLMs), recent research has developed watermarking algorithms, which restrict the generation process to leave an invisible trace for watermark detection. Due to the two-stage nature of the task, most studies evaluate the generation and detection separately, thereby presenting a challenge in unbiased, thorough, and applicable evaluations. In this paper, we introduce WaterBench, the first comprehensive benchmark for LLM watermarks, in which we design three crucial factors: (1) For benchmarking procedure, to ensure an apples-to-apples comparison, we first adjust each watermarking method’s hyper-parameter to reach the same watermarking strength, then jointly evaluate their generation and detection performance. (2) For task selection, we diversify the input and output length to form a five-category taxonomy, covering 9 tasks. (3) For evaluation metric, we adopt the GPT4-Judge for automatically evaluating the decline of instruction-following abilities after watermarking. We evaluate 4 open-source watermarks on 2 LLMs under 2 watermarking strengths and observe the common struggles for current methods on maintaining the generation quality. The code and data are available at https://github.com/THU-KEG/WaterBench.

## introduction llm has achieved significant success in generating human-like texts @xcite @xcite . however, the potential misuse of llm has also raised concerns @xcite . for example, chatgpt can be used to generate fake news @xcite , which may manipulate the public opinion. to mitigate this kind of risk, it is necessary to develop a watermarking algorithm to detect whether a text is generated by llm @xcite . as shown in with watermark there is a very successful list actors who started in broadway... what are the names of some famous actors that started their careers on broadway? figure 1 : the generated texts without and with watermark @xcite on a test example from alpacafarm @xcite , an instructionfollowing benchmark. llm equipped with watermark will be more inclined to generate tokens in the green list , which can then be detected by a higher zscore measurement (@xmath0. we utilize tp, tn, and gm to jointly evaluate the watermarking performance. figure 1 , the watermarked texts are generated with a biased distribution of tokens, which distinguishes it from unwatermarked texts. we believe the goal of watermarking is to achieve high detection accuracy while maintaining the generation quality. so we utilize the commonly used tp (true positive), tn (true negative), and gm (generation metric) to evaluate watermarks @xcite . due to the two-stage nature of this task, most studies @xcite evaluate the generation and detection separately and they do not conduct a unified hyper-parameter search for each watermarking method, which may lead to unfair comparisons. since, there is usually a trade-off between the detection performance and the generation quality. besides, previous evaluations are often conducted via text completion on a single dataset, such as c4 realnewslike dataset @xcite , which cannot comprehensively measure the generation quality of llms. research for watermark control hyper para. jointly test (tp,tn,gm) number of tasks instruction following metric for generated text lm watermark @xcite ✓ × 1 × perplexity v2 watermark @xcite ✓ × 2 × perplexity robust watermark @xcite ✓ × 1 × perplexity gpt watermark @xcite × × 2 × perplexity semantic watermark @xcite ✓ × 2 × ref. three bricks @xcite furthermore, most evaluations only calculate the perplexity @xcite , which is not aligned with human preference and thus not practical in the era of llms @xcite . to address these issues, we propose waterbench, the first comprehensive benchmark for llm watermarks, which has three crucial factors: (1) benchmarking procedure: we first introduce the concept of watermarking strength @xcite , i.e. the detection robustness to disturbance, to quantify the llm watermarks' trade-off controlled by hyper-parameters. we present a reasonable hyperparameter search procedure: given a dataset and an llm, we adjust the hyper-parameters of each watermarking method to unify the watermarking strength and then freeze the parameters to jointly evaluate the detection and generation performance. based on the waterbench dataset, we conduct an experiment of 4 reproducible watermarks on 2 llms (llama2-chat @xcite and in-ternlm @xcite ), leading to some interesting findings: (1) we adjust two different watermarking strengths, 0.7 and 0.95, and observe that the detection and generation performance are significantly different. in other words, if we compare two watermark strategies without aligning their wa-termarking strengths, it is easy to let one "surpass" another in some aspects. (2) the tasks with short output length are generally more difficult to detect, with lower tp. the v2 watermark @xcite is the best watermarking method in terms of gm. ( 3 ) on the open-ended task, if we use gpt4-judge to evaluate, the watermarked llm will decrease over 96% from original llm, which shows the sensitivity of the metric and indicates the common struggles of watermarks on maintaining the generation quality. in human evaluation, the gpt4 obtains over 0.6 cohen's kappa coefficient with 3 annotators, achieving substantial agreement. to summarize, our contributions are three-fold: (1) we propose a new benchmarking procedure that first search hyper-parameters for watermarks then jointly evaluate detection and generation performance to eliminate the unfair comparison between different watermarking strengths. (2) we construct a multi-task benchmark to facilitate future research. ## related work to detect llm-generated text, previous works @xcite @xcite mainly explored the classifiers that distinguishes human and llm-generated texts based on features. however, as llms are becoming more and more alike human, some classifiers may mistakenly recognize human as llms @xcite . in addition to black-box classifiers, recent approaches have also introduced white-box detection methods that inject watermarks into llm generated texts @xcite @xcite . the inference-time watermarks @xcite randomly split the vocabulary and ad- soft watermark ( =5, =0.8) hard watermark ( =5, =0.8) output no watermark 1. hugh jackman 2. audra mcdonald 3.idina menzel... ## waterbench to investigate how inference-time watermarks perform on detection and generation, as shown in figure 2 , we propose a benchmarking procedure that ensures fair comparisons (section 3.2). then, we present the waterbench dataset with a diverse length distribution (section 3.3). finally, we introduce the gpt4-judge evaluation (section 3.4). ## conclusion in this paper, we propose waterbench, a new benchmark for evaluating large language model watermarks. we first introduce a benchmarking procedure that searches hyperparameters to ensure consistent watermarking strength across different methods, allowing for a fair comparison. second, we construct a multi-task benchmark spanning nine typical nlp tasks with varying input/output lengths. finally, we incorporate the gpt4-judge metric to automatically evaluate the results. experiments show that it sensitively reflects declines in instruction-following quality after watermarking. we hope that our work will inspire and facilitate the future research on llm watermarks.