Bypassing LLM Watermarks with Color-Aware Substitutions (2024)

Abstract: Watermarking approaches are proposed to identify if text being circulated is human- or large language model- (LLM) generated. The state-of-the-art watermarking strategy of Kirchenbauer et al. (2023a) biases the LLM to generate specific (“green”) tokens. However, determining the robustness of this watermarking method under finite (low) edit budgets is an open problem. Additionally, existing attack methods failto evade detection for longer text segments. We overcome these limitations, and propose Self Color Testing-based Substitution (SCTS), thefirst “color-aware” attack. SCTS obtains color information by strategically prompting the watermarked LLM and comparing output tokensfrequencies. It uses this information to determine token colors, and substitutes green tokens with non-green ones. In our experiments, SCTS successfully evades watermark detection using fewer number of edits than related work. Additionally, we show both theoretically and empirically that SCTS can remove the watermark for arbitrarily long watermarked text.

## introduction large language models (llms) are pervasively used in applications related to education @xcite and programming @xcite . as with most generative ai technologies, however, they have dual use-for generating misinformation @xcite and facilitating academic dishonesty @xcite . detecting llm-generated content is of immense interest; paraphrasing @xcite , "detection techniques have witnessed advancements propelled by innovations in zero-shot methods, fine-tuning" etc. resulting in various methods being proposed. however, most detection techniques are susceptible to a large number of false positives @xcite . erroneous detection is further exacerbated when llms' output more closely mirrors human content @xcite . watermarking circumvents this issue by embedding information into the generation process by manipulating the decoding step. for example, the watermark of @xcite biases the logits to boost the probability of specific tokens (denoted "green tokens") in contrast to the remaining "red" tokens. the set of green tokens is determined by a hash function, and the security of this watermark is based on the hardness of finding hash collisions. despite the emergence of more watermarking schemes @xcite , research shows that methods that (slightly) modify logit distributions, such as @xcite outperform others without significant output quality degradation @xcite : humans can not identify if the text is watermarked, and detection requires processing approximately one hundred tokens. thus, for the remainder of this work, we focus on the @xcite watermark and its variant by @xcite as exemplars. additional related watermarking methods are discussed in § 2.2. we study if watermarking approaches based on logit perturbation are robust to post-processing distortions. this is an important question across various applications. for example, it will reliably determine if academic integrity was violated in educational settings: if the watermark is not robust, a student can easily edit the generated watermark, and the resulting data will fail verification. while prior works also ask this or similar questions @xcite , they do so under unreasonable settings, where edits (or post-processing distortions) to watermarked text are unconstrained (i.e., they edit large fractions of the generated text with potential degradation to output quality). they use dedicated paraphrasing models @xcite , or design customized prompts for post-hoc output paraphrasing @xcite . alternatively, they design prompts to generate hard-to-detect text (which is often contextually unrelated), sometimes with low entropy @xcite . we evaluate robustness under a more realistic setup: post-hoc editing involves making a "reasonable" number of edits to maximally retain information generated by the llm. to achieve our goal, we make two observations. first, previous attack methods are color-agnostic: existing approaches that replace text via paraphrasing still preserve some fraction of the original text, resulting in dilution rather than erasure of watermarks @xcite (i.e., erasure needs more edits). even if they manage to evade detection for certain samples by removing "most" fragments, our experiments demonstrate that they would fail when constrained by a smaller edit budget ( § 6). second, prior attacks require another high-quality "unwatermarked" llm for paraphrasing; this llm often generates verbose or inarticulate text, and this method is not very practical as the paraphrasing llm can be used for generation in the first place. building atop these observations, we propose the self color testing-based substitution (scts) attack. this introduces new text fragments that contain fewer green tokens than human-written text, with high probability. we prompt the llm to perform targeted (periodic) generations given an input context (refer prompt in § 4) to get color information from the frequency of words in the generated text. this is used to perform color-aware substitution, and in turn, neutralizes the higher number of green list tokens from the preserved fragments. consequently, scts can evade watermark detection for arbitrarily long text segments within a reasonable edit distance budget, without using an unwatermarked llm. our evaluation compares scts and existing representative attack methods over a series of edit distance budgets. we conclude that across various settings, our approach is superior in reducing auroc (a proxy for detection success) to less than 0.5 on two llms and two watermark-ing schemes. additionally, and most importantly, our approach is theoretically grounded, with a comprehensive analysis. in summary, our main contributions are: ## the building blocks we will first show some properties of the watermarking efficacy as a function of output (generation) length, and use this to explain why existing color-agnostic attack methods fail for sufficiently long watermarked text. then, we focus on substitution-based attacks and formalize the coloragnostic baseline. but before we begin, we outline the assumptions and definitions we make throughout this section. assumption 1: we consider the left hash for ease of exposition. as stated earlier, the left hash is one where the green list for the current token is obtained by hashing c tokens counting backward from the current token (not including it). assumption 2: each (generated) token from the watermarked llm is green with constant probability p, i.i.d, s.t. γ < @xmath0. in the generated output, the number of c-grams is tc. definition 1 (c + 1-gram). this is the set of tokens used for color testing. it includes the token whose color is to be checked and the c tokens before it. the color of a c + 1-gram refers to the color results when this c + 1-gram is sent to the detector. definition 2 (effective length t e ). this is the number of c + 1-grams used for color testing i.e., t @xmath1. this is the threshold used in detection. the detector outputs "watermarked" ⇐⇒ @xmath2. definition 4 (critical length t c ). this is the value of t e such that e[z] = z th . this can be thought of as the effective length needed for successful detection, and may or may not exist. definition 5 (average green probability q). : this is the average probability for a c + 1-gram being green for any given arbitrary sample, which can be watermarked or non-watermarked, attacked or non-attacked. by definition, e[∥t ∥ g ] = qt e . ## experiment results through our evaluation, we wish to answer the following questions: we observe that: ## discussion and open questions a "green list" approach baseline? a straightforward methodology for detecting "green list" words involves the analysis of publicly available humanwritten texts, as demonstrated by @xcite . however, this attack method is not applicable to the target watermarking @xcite ; see appendix a. can scts be faster? one way is to store the color information already found to reduce repetitive color testing, with the risk of the accumulation of incorrect results and the cost of space. such caching is more practical for small c. can one llm query get more color information? currently, our color testing can only test one pair in each llm query, and it can not distinguish if the two words/tokens are both red or green. more candidates for random generation can help with the cost of more undesired factors getting involved, like the increased complexity of the prompt, the reduced frequency count for each candidate, and the more complex cases in hypothesis testing. unknown c besides prompting the model to guess c first, one way to use scts in this case is to use a large estimated c. we leave this for future work. can the accuracy of sct be higher? @xcite shows the complicated behavior when the llm model is prompted to generate uniform random strings, which are far from uniformity and vary model by model. such behavior makes our color testing sometimes inaccurate. one possible way is to have multiple variants, like exchanging the position of the new candidate and the old to do a second prompt, to improve accuracy with the cost of more computation. ## conclusions our study presents scts, an algorithm to evade watermark detection without using external llms. we demonstrate that scts can effectively eliminate watermarks from long texts using a straightforward algorithm. this approach reveals that specific prompting techniques can uncover and exploit private watermarking information, enabling evasion. we aim to inspire further research on developing more robust and secure watermarking schemes.