Downstream Trade-offs of a Family of Text Watermarks (2024)

Abstract: Watermarking involves implanting an imperceptible signal into generated text that can later be detected via statistical tests. A prominent family of watermarking strategies for LLMs embeds this signal by upsampling a (pseudorandomly-chosen) subset of tokens at every generation step. However, such signals alter the model’s output distribution and can have unintended effects on its downstream performance. In this work, we evaluate the performance of LLMs watermarked using three different strategies over a diverse suite of tasks including those cast as k-class classification (CLS), multiple choice question answering (MCQ), short-form generation (e.g., open-ended question answering) and long-form generation (e.g., translation) tasks. We find that watermarks (under realistic hyperparameters) can cause significant drops in LLMs’ effective utility across all tasks. We observe drops of 10 to 20% in CLS tasks in the average case, which shoot up to 100% in the worst case. We notice degradations of about 7% in MCQ tasks, 10-15% in short-form generation, and 5-15% in long-form generation tasks. Our findings highlight the trade-offs that users should be cognizant of when using watermarked models.

## introduction large language models (llms), and derived chatbots of the likes of chatgpt, can generate humanlike responses to a variety of requests like writing emails, translating or summarizing content @xcite . as these systems gain popularity, there are looming concerns about their misuse for spreading targeted misinformation, influencing public opinion @xcite or conducting social engineering attacks @xcite . such concerns have spurred research towards distinguishing human-written and llm-generated content. naive approaches such as training posthoc classifiers for this purpose have been shown to be ineffective, as they typically have large falsepositive rates that can lead to false accusations of plagiarism @xcite . these classifiers can further degrade in accuracy when llm developers like openai continually finetune and update their public models. additionally, the output distributions of future llms may grow even more similar to that of human-written text causing the efficacy of such approaches to wane. a promising alternative is to intentionally embed a watermark signal @xcite @xcite into llm-generated text that is imperceptible to unsuspecting readers but can be algorithmically detected using statistical tests. a popular watermarking scheme, often referred to as kgw, works by boosting the probabilities of a psuedorandomly chosen subset of the model's vocabulary at every generation step @xcite . this scheme has been extensively studied and extended @xcite @xcite . the original approach and its derivatives, collectively called the kgw family in the literature, comprise the most popular watermarking strategies for llms today. previous works studying the trade-offs of watermarking llms mostly restrict their analysis to intrinsic evaluations of watermarked models generation quality such as perplexity or gpt4 judgements @xcite , eschewing evaluation on downstream task benchmarks. but since it is likely that all strong llms made available to the public (eg. through internet apis) will be watermarked in the near future (as promised by several leading llm developers (press, 2023)), it is important to understand how watermarks impact llms' performance on downstream tasks. in our work, we evaluate the downstream impact of 3 popular watermarks from the kgw family including the original kgw approach @xcite , ewd @xcite and sir @xcite over a diverse selection of tasks. since kgw-based watermarks perturb output probability distributions at the token level, we categorize the tasks as follows for our analysis: ## background the kgw watermark @xcite is a deterministic algorithm parameterized by 3 hyperparameters γ, δ and k, and a keyed psuedorandom function f • : @xmath0, r} m . ## evaluation setup datasets. to assess watermarks' effects on tasks that are framed as classification tasks (cls), we work with sst-2, boolq and cb from the glue @xcite and super-glue @xcite benchmarks. these correspond to sentiment analysis, yes/no question answering and textual entailment tasks respectively. we select the commonsense nli dataset called hellaswag @xcite and the questionanswering dataset piqa @xcite as mcq tasks, the reading comprehension datasets drop @xcite and squadv2 @xcite as sgen tasks, and the wmt14-en-fr @xcite and wmt20-en-de (barrault et al., 2020) translation tasks as lgen tasks. we evaluate models' performance on these tasks using the metrics typically associated with them. for instance, cls and mcq tasks are evaluated using accuracy while sgen tasks are evaluated using f1 scores and lgen translation tasks are evaluated using bleu scores @xcite . ## results we present our main results over all models, tasks and watermarks under the moderate signal strength and provide our full set of results in ## analysis why does sir perform worse? in section 4, we evaluate the performance drops on various different tasks due to applying moderate strength kgw, ewd and sir watermarks over 3 models, each of about 7 billion parameters. as explained in section 3, we define the 'moderate' intensity as the watermark strength required to obtain an empirical tpr @ @xmath1.01 of 0.75 using 50-token long freeform generations. despite this uniformity in calibration, we find that the sir watermark consistently leads to worse downstream tradeoffs than kgw and ewd over each cls (figure 1 ), mcq (figure 4 ), sgen (figure 7 ) and lgen (figure 8 ) task we study. one aspect that makes sir different from kgw and ewd is that its γ hyperparameter cannot be explicitly set by the user, but instead varies dynamically around 0.5. as effect of model strength. in downstream effects of watermarking. most prior work on watermarking has evaluated their resulting models using perplexity of the generated text. @xcite evaluates the performance of watermarked models on a single question -answering task. some follow-up work @xcite conducts small-scale evaluations but does not attempt to uncover the causes for observed performance drops. one contemporaneous work @xcite performs a similar study to ours over a broad range of tasks but in contrast to our work, chooses watermark hyperparameters relatively arbitrarily (see section 3) which we believe limits the practical applicability of their findings. to the best of our knowledge, our study is the first to conduct a principled analysis on the downstream effects of watermarking schemes over a broader spectrum of tasks, shedding light on underlying reasons for the observed trade-offs. ## conclusion we evaluate the extent to which watermarks from the kgw family hurt downstream performance by examining three watermark and three llms over a diverse suite of nlp tasks. we motivate a categorization of tasks into 4 buckets and analyze causes for the observed trade-offs in each category. we find the performance trade-offs for each category vary in a manner that simple perplexity measurements cannot capture or predict (an assumption implicit in prior work). watermarks, under realistic hyperparameters, can cause significant drops in llms' effective utility across all tasks. we observe drops of 10 to 20% in cls tasks in the average case, which shoot up to 100% in the worst case. we notice degradations of about 7% in mcq tasks, 10-15% in short-form generation, and 5-15% in long-form generation tasks. we also find some evidence that the downstream trade-offs posed by the kgw family of watermarks may diminish with increasing model strength. we believe that our work will (i) allow developers and practitioners to make informed choices about watermarked llms and their adaptations, (ii) spur research into novel watermarking strategies that present better trade-offs, and (iii) inspire techniques for maintaining model performance under existing watermarking schemes.