W ater P ool: A Language Model Watermark Mitigating Trade-Offs among Imperceptibility, Efficacy and Robustness (2025)

Abstract: Watermarking is a prominent technique to trace the usage of specific large language models (LLMs) by injecting patterns into model-generated content. An ideal watermark should be imperceptible, easily detectable, and robust to text alterations, yet existing methods typically face trade-offs among these properties. This paper utilizes a key-centered scheme to unify existing methods by decomposing a watermark into two components: a key module and a mark module. We show that the trade-off issue is the reflection of the conflict between the scale of the key sampling space during generation and the complexity of key restoration during detection within the key module. To this end, we introduce WaterPool, a simple yet effective key module that preserves a complete key sampling space for imperceptibility while utilizing semantics-based search to improve the key restoration process. WaterPool can integrate seamlessly with existing watermarking techniques, significantly enhancing their performance, achieving near-optimal imperceptibility, and markedly improving their detection efficacy and robustness (+12.73% for KGW, +20.27% for EXP, +7.27% for ITS).

## introduction the world has recently witnessed the great power of large language models (llms). however, the widespread use of these systems has raised significant concerns about their potential misuse. for example, llms could be used to generate massive amounts of fake news or automated comments to manipulate social media, posing threats to academic integrity and intellectual property rights @xcite . to address these issues, watermarking has been proposed to track the usage of specific models @xcite . it embeds a hidden exp-pool kgw-pool unigram kgw its-pool gamma delta its exp efficacy & robustness imperceptibility upperbound trade-off curve w a t e r p o o l w at er po ol wa ter po ol figure 1: previous methods face trade-offs among imperceptibility, efficacy and robustness. waterpool mitigates this problem to a significant extent. it can be integrated with other watermarking methods, improving them on all three aspects. this figure is based on the experiments on open-ended text generation presented in in this paper, we try to explore this issue. we begin by unifying existing watermarking methods with a key-centered scheme. it decomposes a watermarking technique into two independent modules, a key module and a mark module, as shown in figure 2. during generation, the key module samples a private key. it is then utilized by the mark module as a random seed to distort the next token distribution, from which watermarked texts are sampled. during detection, the key module attempts to restore the private key. then the mark module aligns the text with the restored key to compute statistics, which imply the likelihood of watermark presence. the decomposition separates requirements of imperceptibility, efficacy and robustness into the two modules. subsequently, we show that the key module significantly contributes to the trade-off problem. specifically, the trade-off problem actually stems from the conflict between the scale of key sampling space during generation and the complexity of key restoration during detection. to overcome this trade-off, we introduce water-pool, a simple but effective key module. waterpool maintains the complete key sampling space, crucial for imperceptibility, while leveraging a semanticsbased search to significantly enhance the precision and effectiveness of the key restoration process, thereby ensuring high robustness against attacks. we integrate waterpool into three of the most renowned watermarking techniques, exp @xcite ), kgw @xcite and its @xcite ). waterpool effectively mitigates the traditionally "inevitable" trade-offs, achieving superior performance as shown in figure 1 . our experiments include two scale of large language models @xcite across tasks of open-ended generation and long-form question answering. experimental results demonstrate the supreme capabilities of our proposed waterpool. on one hand, it elevates the imperceptibility of kgw, exp and its to near-optimal levels. on the other hand, it significantly enhances the efficacy and robustness of previous watermarking techniques, yielding substantial improvements across different experimental settings (+12.73% for kgw, +20.27% for exp, +7.27% for its) 2 . imperceptibility. an ideal watermark should maintain the output distribution of llm as unchanged as possible @xcite @xcite . formally, we define the imperceptibility following "n -shot undetectable" from @xcite : for all x n , y @xmath0, here, superscripts n indicates different rounds of generation. for llms, which model language in an auto-regressive manner, the equation above can be expressed as: we want to highlight the importance of the product over multiple generations above. it indicates that it is infeasible to distinguish between the original and the watermarked texts without prior knowledge about the modified distribution, even when multiple queries are allowed @xcite . efficacy. an ideal watermark technique should be able to distinguish watermarked texts from the others. empirically, it is required to achieve high true positive rate with low false positive rate. most watermarking techniques achieve this by ensuring a substantial difference between the per-token statistic under alternative hypothesis (h 1 : the candidate token is sampled from the modified distribution pm (•|x n , y @xmath1: the candidate token is sampled from other distributions). this can be formulated as, ## experiments datasets. following @xcite , we include two common used datasets, the c4 dataset and "explain like i'm five" (eli5) @xcite for open-ended text generation and long-form question answering, respectively. we randomly select about 3000 texts from both datasets as prompts for two llms, opt-1.3b and opt-6.7b, following @xcite . metrics. we generate 20 watermarked outputs for each prompt, while considering outputs of the original llm as non-watermarked. subsequently, 120,000 samples are used to evaluate each method. for both efficacy and robustness, we report true positive rate at 1% false positive rate, denoted as tpr@@xmath2. we also include roc-auc in appendix e. to evaluate the robustness of watermarking techniques, we include three kinds of attacks, namely lexical-attack, dipper-attack and translation-attack. lexical-attack randomly add/delete/replace 10% tokens of texts. dipper is a paraphrasing model @xcite . translation-attack represents roundtrip-translation, which is a paraphrasing pipeline. for the evaluation of imperceptibility, we split the criteria into two aspects: (1) the distribution bias within each output (2) the independence among different outputs. the former can be evaluated with perplexity while the latter can be roughly evaluated with n-gram distinction @xcite . specifically, we consider the distinction across all outputs (globdistinct-n ) and within outputs in response to one single prompt (group-distinct-n ). baselines and implementation details. we include several typical methods as baselines. in addition to exp, its and kgw, we also include gamma @xcite , delta @xcite and unigram @xcite . all baselines are reproduced based on source codes provided by original paper. we use a 128 dimension sentence embedding model @xcite as enc(•) in waterpool. for implementation of mark modules in different waterpool (i.e. kgw-pool, its-pool, exp-pool), we use identical hyperparameter settings as the corresponding baselines. other details are presented in appendix d. ## related work watermarking is a specific form of steganography. steganography requires that, without the knowledge about private keys, distributions of original texts and texts with steganography must be indistinguishable @xcite @xcite @xcite , which leads to imperceptibility objective of watermarking in section 2. llm watermarking has recently gained many attention @xcite @xcite @xcite @xcite , which can be seamlessly integrated into all llms without further training. kgw @xcite is the pioneering work, using context windows as private keys to increase probabilities of a specific partition of vocabulary. building on kgw, follow-ups made many improvements, such as proposing different hash functions @xcite @xcite , heuristic partition strategies @xcite , embedding multi-bit messages @xcite , and robust hypothesis testing techniques @xcite . despite these advancements, the trade-off among imperceptibility, efficacy, and robustness has been widely recognized and remains unresolved. a concurrent work @xcite also tries to break the trade-off by resampling until observing significant watermarking signals. although this approach maintains imperceptibility in a single turn, it significantly alters the n -shot output distribution. ## conclusion in this paper, we focus on the trade-off challenges among imperceptibility, efficacy and robustness in llm watermarking. through a key-centered scheme, we have identified that the trade-offs arise from the conflict between the scale of the key sampling space during generation and the complexity of key restoration during detection. this insight motivates the design of waterpool, a key module utilizing semantic search to alleviate this conflict. waterpool integrates seamlessly with most existing watermarking methods, significantly enhancing their performance across all three dimensions. we hope this work offers valuable insights to future research for better solutions to this challenge.