Revisiting the Robustness of Watermarking to Paraphrasing Attacks (2024)

Abstract: Amidst rising concerns about the internet being proliferated with content generated from language models (LMs), watermarking is seen as a principled way to certify whether text was generated from a model. Many recent watermarking techniques slightly modify the output probabilities of LMs to embed a signal in the generated output that can later be detected. Since early proposals for text watermarking, questions about their robustness to paraphrasing have been prominently discussed. Lately, some techniques are deliberately designed and claimed to be robust to paraphrasing. Particularly, a recent approach trains a model to produce a watermarking signal that is invariant to semantically-similar inputs. However, such watermarking schemes do not adequately account for the ease with which they can be reverse-engineered. We show that with limited access to model generations, we can undo the effects of watermarking and drastically improve the effectiveness of paraphrasing attacks.

## introduction given the remarkable fluency and relevance with which language models (lms) respond to varied queries, it is challenging for humans to distinguish language model outputs from human-written text. past studies note that human performance in making such a distinction is close to that of random chance @xcite . in response, watermarking language models is seen to be a principled way to certify whether a piece of text was generated by a model. a prominent watermarking approach works by implanting a signal during decoding, wherein a certain set of tokens (aka a green list) is boosted @xcite . this signal, albeit imperceptible to an unsuspecting reader, can be verified by running a statistical test. for watermarking to be effective, the implanted signal should be easy to detect and hard to remove. unsurprisingly, there has been considerable discussion about the robustness of watermarking approaches against paraphrasing attacks @xcite . there exist different ways of choosing tokens in the green list and the extent to which they should be boosted. these approaches offer varying levels of robustness against paraphrasing. the original paper @xcite recommends pseudo-randomly selecting a different set of green tokens at every timestep based on a hash of the last k tokens. the authors note that higher values of k would render the watermarking scheme ineffective, as any changes to a token would disrupt the green lists for the next k timesteps, and therefore suggest using the last one or two tokens (@xmath0. a recent study @xcite argues that "a consistent green list is the most robust choice," as any modifications to the input text have no effect whatsoever on the (fixed) green list. relatedly, @xcite propose a "semantic-invariant robust" watermarking (sir) which is designed to produce similar green lists for semantically-similar contexts and is touted to be robust to paraphrasing. in this ongoing debate, our work highlights just how easy it is to decipher the green list for both the semantic-invariant watermarking scheme @xcite and watermarking with consistent green list @xcite . while a recent contemporaneous study @xcite corroborates that watermarking with a fixed green list can be easily reverse-engineered, we show that similar results also hold for semantic-invariant watermarking scheme from @xcite . for both these watermarking schemes, with just 200k tokens of watermarked output, we can predict green lists with over 0.8 f1 score. this knowledge of green lists can be exploited while paraphrasing to launch at-tacks that cause the detection rates to plummet below 10%, rendering the watermark ineffective. overall, our findings suggest that one should consider the possibility of reverse-engineering the watermarking scheme, when discussing its robustness to paraphrasing attacks. our work also raises potential concerns about the generalization of watermarking algorithms that use machine learning models to generate the watermarking signal. ## background a prominent approach to watermarking is to compute watermarking logits that are added to logits generated by a language model at each generation step. formally, for a language model m with vocabulary v , and a prefix comprising tokens w 1 , w 2 , . . . , w n , the scheme involves first computing the logits m(w 1 . . . , w n ) = (l 1 , . . . , l |v | ) of the language model that would ordinarily be used to predict the subsequent token. as per @xcite , the last k tokens, w n-k+1 to w n , are then fed to a psuedo-random function f to partition v into a green list g and a red list r such that |g| + |r| = |v |. finally, the logits corresponding to the tokens in the green list, g, are boosted by δ (δ > 0). the watermark can then be detected through a one-proportion z-test on the fraction of green tokens in the generated text. a recent study @xcite makes a case for using a fixed green list (where the partitioning function, f , does not depend on the context) to confer robustness to paraphrasing attacks. the underlying intuition is that any changes in the text will not disrupt the constant green list. to counter paraphrasing attacks, another recent proposal @xcite is to train a model, w, to output watermarking logits using the context: w(w 1 . . . , w n ) = (δ 1 , . . . , δ |v | ). this model, w, is designed such that similar contexts yield similar watermarking logits. this property is supposed to make models robust to paraphrasing. further, diverse contexts are supposed to yield different watermarking logits, thus making it hard to reverseengineer the green list-this is not true in practice, as we show later in our experiments ( §4.2). paraphrasing attacks. @xcite introduce a controllable paraphraser and launch paraphrasing attacks on various text detectors. their findings indicate that although paraphrasing reduces the effectiveness of most ai-generated text detectors, watermarking is the most resilient method. another study @xcite investigates the reliability of watermarks across different paraphrasing models and suggests that the reliability of watermarking should be discussed in terms of the length of the available input. the study concludes that watermarking is extremely reliable for longer texts. ## methods we study the robustness of watermarking approaches against paraphrasing attacks. unlike prior attacks @xcite , we first attempt to decipher the tokens in the green list and then incorporate that knowledge in existing paraphrasing attacks. ## results & discussion we first share details about our setup and then discuss the results of paraphrasing attacks. ## conclusion we analyze watermarking schemes believed to be specifically robust to paraphrasing and show that it is easy to reverse engineer these algorithms and launch severe paraphrasing attacks. the effectiveness of our attacks underscores the need to account for the ease of reverse-engineering watermarking schemes, when discussing its robustness to paraphrasing attacks. additionally, we highlight that existing metrics concerning the security of watermarking are overly optimistic.