Watermark under Fire: A Robustness Evaluation of LLM Watermarking (2025)

Abstract: Various watermarking methods (“watermarkers”) have been proposed to identify LLM-generated texts; yet, due to the lack of unified evaluation platforms, many critical questions remain under-explored: i) What are the strengths/limitations of various watermarkers, especially their attack robustness? ii) How do various design choices impact their robustness? iii) How to optimally operate watermarkers in adversarial environments? To fill this gap, we systematize existing LLM watermarkers and watermark removal attacks, mapping out their design spaces. We then develop WaterPark, a unified platform that integrates 10 state-of-the-art watermarkers and 12 representative attacks. More importantly, by leveraging WaterPark, we conduct a comprehensive assessment of existing watermarkers, unveiling the impact of various design choices on their attack robustness. We further explore the best practices to operate watermarkers in adversarial environments. We believe our study sheds light on current LLM watermarking techniques while WaterPark serves as a valuable testbed to facilitate future research.

## introduction the recent advances in large language models (llms), including gpt @xcite and llama @xcite , have significantly enhanced our capabilities of general-purpose text generation and complex problem-solving, but also raised concerns about misuse through disinformation @xcite , phishing @xcite , and academic dishonesty @xcite . there is thus a pressing need for the capability of identifying llmgenerated content. a simple approach is to train classifiers to distinguish between llm-and human-generated texts @xcite . however, as llms improve, 1 all the source code and data are publicly available: http s://github.com/jackpurcell/waterpark adversary llm watermark @xmath0 rq1 -what are the strengths and limitations of various watermarkers, especially their robustness against manipulations? rq2 -how do different design choices impact the attack robustness of watermarkers? rq3 -what are the best practices for operating watermarkers in adversarial environments? despite recent efforts to benchmark llm watermarkers, existing research is limited in addressing these questions. waterbench @xcite primarily focuses on watermarking effectiveness; markmywords @xcite mainly evaluates the robustness of a specific watermarker @xcite ; markllm @xcite focuses on providing a platform to compare watermark detectability, basic robustness, and text quality. @xcite provides a comprehensive overview of watermarking techniques. moreover, these studies lack an in-depth analysis of how a watermarker's design choices impact its robustness. consequently, the aforementioned questions remain largely unexplored. ## llm watermarking a large language model (llm) is typically an autoregressive model that generates the next token x t based on previous tokens @xmath1, . . . , x t-1 (including its prompt), modeled as sampling from a conditional distribution p(x t |@xmath2. conceptually, a watermark is a pattern embedded in a given signal (e.g., text) for identifying the signal's source. in the context of llms, watermarks can be used to prove that a given text is llm-generated (or even generated by specific llms). an llm watermarking method ("watermarker") comprises three components: the llm, watermarking procedure (generator), and detection procedure (detector), as shown in figure 1 . typically, the generator produces watermarked texts iteratively. at each iteration, with access to a secret key k, the previous tokens @xmath3, and the llm's next-token distribution p(x t |@xmath4, the generator generates a perturbed distribution p(x t |@xmath5, from which the next token x t is sampled. meanwhile, with access to a secret key k, the detector determines whether m generates a given text. in symmetric schemes, @xmath6, whereas in asymmetric ones @xcite , k ≠ k ′ . this study mainly focus on the symmetric case. the current watermarkers can be categorized based on the information carried by the watermarks (e.g., one-bit versus multi-bit) and their key design choices, including context dependency, generation strategy, and detection method. the key design factors of each category are deferred to §b . ## evaluation we leverage waterpark to empirically assess representative llm watermarkers, focusing on their attack robustness and other related criteria. ## discussion next, we examine the current practices of operating watermarkers in adversarial environments and explore potential improvements. for each watermarker, we mainly use its specific detector to detect watermarked texts. here, we explore a generic, neural network-based detector as an alternative. to this end, we employ a pre-trained roberta model and fine-tune it as a binary classifier using watermarked and non-watermarked texts. we use opt-1.3b as the underlying llm and c4 as the reference dataset. the watermarked text is generated by the llm and the watermarker jointly, whereas the non-watermarked text is produced by chatgpt using the same prompt to mimic human response. ## conclusion in this paper, we systematize the existing llm watermarkers and watermark removal attacks, mapping out the design space for various watermarking and attacking techniques. we then design and implement waterpark, the first open-source platform devoted to assessing the attack robustness of llm watermarkers in a unified and comprehensive manner. leveraging waterpark, we conduct a systematic evaluation of the robustness of existing watermarkers, addressing unresolved questions, revealing design trade-offs, and identifying opportunities for further improvement. our findings shed light on the current llm watermarking techniques, while waterpark serves as a valuable benchmark aiding future research.