SIRA: Exposing Vulnerabilities in Text Watermarking with Self-Information Rewrite Attacks

Authors: Yixin Cheng, Hongcheng Guo, Yangming Li, Leonid Sigal
Primary area: alignment, fairness, safety, privacy, and societal considerations
Keywords: LLM watermark, robustness, safety ai, paraphrasing attack

Abstract:
Text watermarking is designed to embed hidden, imperceptible, markers within
content generated by large language models (LLMs), with the goal of tracing and
verifying the content’s origin to prevent misuse. The robustness of watermarking
algorithms has become a key factor in evaluating their effectiveness, but remains
an open problem. In this work, we introduce a novel watermark removal attack,
the Self-Information Rewrite Attack (SIRA), which poses a new challenge to the
robustness of existing watermarking techniques. Since embedding watermarks
requires both concealment and semantic coherence, current methods prefered to
embed them in high-entropy tokens. However, this reveals an inherent vulnera-
bility, allowing us to exploit this feature to identify potential green tokens. Our
approach leverages the self-information of each token to filter potential pattern to-
kens that embed watermarks and performs the attack through masking and rewrit-
ing in a black-box setting. We demonstrate the effectiveness of our attack by
implementing it against seven recent watermarking algorithms. The experimental
results show that our lightweight algorithm achieves state-of-the-art attack success
rate while maintaining shorter execution times and lower computational resource
consumption compared to existing methods. This attack points to an important
vulnerability of existing watermarking techniques and paves way towards future
watermarking improvements.

Decision: reject
Reviewer scores: 6, 5, 5, 3

Reviews and discussion:
<thread>
creation date:
1730517168918

thread_type:
official_review

<thread_object>
creator: reviewer_atel

<content>
summary:
This is an interesting paper, it introduces the Self-Information Rewrite Attack (SIRA) as a novel watermark removal method targeting the robustness of text watermarking techniques used in content generated by large language models (LLMs). These watermarks allow tracing and verifying content origins to prevent misuse.

strengths:
The proposed method, namely SIRA, leverages the high self-information of certain tokens where watermarks are often embedded, applying a lightweight masking and rewriting technique to evade watermark detection. The experimental results demonstrate that SIRA effectively disrupts watermarking in seven algorithms with minimal computational resources, achieving a 90% success rate, suggesting critical vulnerabilities in current watermarking techniques.

weaknesses:
1. **Unclear motivation.** In lines 57-59, the authors point out the limitations of previous methods and later propose their method to overcome these challenges. One limitation is transparency, but why is transparency desirable for watermark removal attacks via paraphrasing? The second limitation is computational cost, but their method requires three models: one for masking, one to generate reference text, and a third one for paraphrasing given the masked reference text, which is not computationally efficient.

2. **The method lacks soundness.** How can it be ensured that the LLaMA 3 rewriting can favor red tokens to achieve watermark removal? Why can this method remove SIR watermarking while existing methods fail?

3. **Unclear experiment settings.** Which models are used as $ M_{\text{attack}} $ (line 290) and the base LLM (line 307), respectively? How does LLaMA3-8b ensure the lightweight and usability of the proposed method? The lightweight aspect of LLaMA3-8b is questionable, considering the watermarked model in the experiment is OPT 1.3B.

4. **The experimental results lack analysis and sometimes are even inconsistent.** Why does word deletion yield a high s-BERT score for KGW-1 (i.e., deletion preserves sentence-level embedding similarity) but low scores for other watermarks in Figure 4b? Why are the results in Figure 4b inconsistent with those in Table 8?

5. **Minor issues:** 
   - Typo: `paraphase` in Figure 1 and its caption.
   - The threshold in line 296 should be $\epsilon$ instead of $\sigma$.

questions:
I have listed most of my questions associated with these weaknesses above.
</content>
</thread_object>

</thread>
<thread>
creation date:
1730528667878

thread_type:
official_review

<thread_object>
creator: reviewer_aqqw

<content>
summary:
The paper proposes a model-based paraphrasing attack. It identifies potential green words and provides a template for rephrasing.

strengths:
The attack is performed in a black-box setting.

The semantics of the paraphrased text is preserved.

weaknesses:
The paper has limited optimization for preserving semantics. The primary strategy for maintaining semantics is changing only potential "green" words while providing a masked template. However, the paper could be strengthened by presenting evidence on how the template contributes to semantic preservation.

Additionally, the method requires two paraphrasing steps: one to generate a reference text and another to create the attack text. The paper would benefit from explaining how it achieves shorter execution time.

In the ablation study, besides comparing the self-information mask with the random mask, it would be valuable to include a comparison between the self-information mask and no mask (paraphrasing twice).

questions:
Why does SIRA have a shorter execution time than GPT Paraphraser, considering that SIRA requires two paraphrasing steps? Table 2 indicates that the execution speed of GPT Paraphraser may vary depending on the network status and real-time OpenAI server load. Does this make for a fair comparison?
</content>
</thread_object>

</thread>
<thread>
creation date:
1730645301799

thread_type:
official_review

<thread_object>
creator: reviewer_91eq

<content>
summary:
This paper presents the Self-Information Rewrite Attack (SIRA), a watermark removal method that targets vulnerabilities in existing watermarking techniques applied to LLM-generated text. By using self-information to identify and modify high-entropy tokens, SIRA effectively removes watermarks while preserving text quality. The authors conduct extensive experiments across multiple watermarking schemes and demonstrate that SIRA achieves a high attack success rate with minimal computational resources.

strengths:
1.	The paper addresses a widely recognized problem in the field of text watermarking. By leveraging self-information, it enables effective watermark removal without compromising text quality.
2.	The experimental setup is comprehensive, covering various watermarking schemes and attack methods.

weaknesses:
1.	Figures 3 and 4 are incorrectly placed, with Figure 4 missing some information, and table formats in this paper are inconsistent.
2.	The proposed method does not appear highly novelty, as it builds upon existing paraphrase attacks by using self-information to locate keywords.
3.	The experiments lack comparisons between self-information and other metrics, such as entropy and token probability, which could help establish the advantage of self-information.
4.	The proposed approach shares characteristics with watermark-stealing attacks [1,2,3], especially in the selection of keywords for targeted editing. A comparison with watermark-stealing attacks in both theoretical analysis and experiments would provide additional insights.
[1] N. Jovanović, R. Staab, and M. Vechev, “Watermark Stealing in Large Language Models.” http://arxiv.org/abs/2402.19361
[2] Q. Wu and V. Chandrasekaran, “Bypassing LLM Watermarks with Color-Aware Substitutions.” http://arxiv.org/abs/2403.14719
[3] Z. Zhang et al., “Large Language Model Watermark Stealing With Mixed Integer Programming.” http://arxiv.org/abs/2405.19677

questions:
1.	What distinguishes self-information from entropy and probability, and what are the specific advantages of using self-information in this context?
2.	In Algorithm 1, is the model M in line 14 the attack model M_attack?
3.	In Figure 4(a), why does the word deletion method have a large impact on KGW-1’s PPL? Additionally, why is this impact much more significant than that of other methods in Figure 4(b)?
4.	In Table 1, the GPT Paraphraser shows a much higher attack success rate for Unigram watermarks than for DIPPER-based attacks, a phenomenon not observed with other watermarking methods. Additionally, SIR, a sentence embedding-based watermark scheme, should theoretically have robustness only second to Unigram, but this is not reflected in Table 1. Further discussion on these points is necessary.
</content>
</thread_object>

</thread>
<thread>
creation date:
1730862045028

thread_type:
official_review

<thread_object>
creator: reviewer_ej9y

<content>
summary:
This paper introduce a novel watermark removal attack SIRA.
Current watermarking methods often favoring high-entropy tokens to embed watermark patterns. High-entropy tokens usually have high self-information. SIRA utilized self-information to identify potential ”green list” token candidates,  which are masked and then completed by LLM.

strengths:
1 SIRA can be implemented using a more lightweight model compared to other model-based paraphrasing attacks.
2 This paper is well organized and discussions are relatively sufficient.

weaknesses:
1 The semantic preservation of the proposed method is inferior compared to GPT Paraphraser.
A clerical error:Line 24 “tempering”? I think it should be "tampering".

questions:
1 How to fairly evaluate the balance between the generated text quality and the attack effect? 
2 Will the attacker really care so much about the resource reduction as shown in Table 2?
</content>
</thread_object>

</thread>
<thread>
creation date:
1732006635132

thread_type:
official_comment

<thread_object>
creator: authors

<content>
title:
common reply

comment:
Dear AC and reviewers,

We sincerely thank all reviewers and the Area Chair for their valuable time and insightful feedback. We are pleased that Reviewers 91eq and ATEL have recognized **the effectiveness of our method**, and that **the low resource requiremen** of our approach has been acknowledged by EJ9Y and ATEL. Additionally, EJ9Y and 91eq found our **experiments and discussions to be comprehensive**, and Reviewer AqqW acknowledged the advantage of **our method being effective in a black-box setting**.

We address some common concerns raised by the reviewers below.

> **Q1**: Novelty and difference with other paraphrasing attacks

A1: Current paraphrasing simply instructs the model to perform paraphrasing in a relative brute force manner. The key insight of our method is that watermarking algorithm **require** embedding in high-entropy tokens [1,2,3,4] to maintain text quality, as detailed in Section F. We are the first to  **reveal that this necessary requirement can also serve as a potential vulnerability** and propose an effective way on **how to exploit it in a black-box setting**. Our method achieves the best empirical attack performance while requiring minimal resources which makes it a suitable method to evaluate watermark robustness . We believe our work provides insights for developing more robust watermarking algorithms in the future and serves as an easy to use robustness evaluation tool to benefit the community.

> **Q2**: Shorter execution time

A2: The total time consumption for SIRA consists of two parts: two generations by the base model and  the self-information mask. The self-information mask is nearly negligible, as it does not require any text generation (less than 0.1 seconds).  The other two generations take around 5 seconds per generation on a single A100 GPU. Thus the total execution time is around 10 seconds. We use huggingface library in our experiment.

The DIPPER method utilizes a specially fine-tuned T5-XXL model for text paraphrasing. This model needs at least 40 GB VRAM to run and one time generation requires around 15 seconds on two A100GPU. DIPPER relies on this specific fine-tuned model, preventing it from transferring to a smaller model.  We use DIPPER official open-source code and weights  in our experiment. 

For GPT,  We use gpt-4o-2024-05-13 in our experiment.  In response to Reviewer AqqW's request, we conducted retests at different times of the day and reported the average results.The execution time is 12.6±0.4 seconds 

All above mentioned we set max output token number is 256.  

> **Q3**: Can we achieve better semantic preservation?

A3: The answer is yes. As the fill in blank prompt we showed in section D, a more powerful paraphrasing will better understand the prompts and its better capability will lead to higher semantic preservation. Our work achieves state-of-art performance in black box paraphrasing attacks and less resource requirement. We clarify that the attack method involves a trade-off between resource consumption, attack effectiveness, and semantic preservation. For example, the white-box watermark attack method RandomWalk[5] theoretically removes any watermark completely. However, it does not guarantee preserving any semantics and requires multiple models; each watermarked text takes approximately 20 minutes to attack on 3 A100 GPU. Our motivation is to propose an easy-to-use, low-resource, and effective tool to further advance robust watermark research, thus we choose to present our work on the lightweight model Llama3-8b.

In response to all feedback received, we have updated our manuscript further, marking all changes in $\color{red}red$\. All updates  and discussion will be included in the revised manuscript.  We sincerely thank you for your suggestions to improve our manuscript.
</content>
</thread_object>

</thread>
<thread>
creation date:
1732510842630

thread_type:
official_comment

<thread_object>
creator: authors

<content>
title:
Are the concerns of the reviewer addressed?

comment:
Dear Reviewers,

Thank you for your comprehensive and constructive reviews. Your feedback is invaluable for improving our manuscript.

 Given that the discussion window is closing soon, please let us know if you have any remaining concerns, *we are really looking forward to hearing from you further*. If you are satisfied with our responses and revisions,, we would appreciate it if you increase your score to reflect our revised manuscript.

Thank you once again for your time and help. Wishing you a wonderful day!

Best Regards, 

The Authors
</content>
</thread_object>

</thread>
<thread>
creation date:
1732871140419

thread_type:
official_comment

<thread_object>
creator: authors

<content>
title:
We are looking forward to your further feedback

comment:
Dear Reviewers,

We sincerely appreciate your valuable feedback on our submission. As the extended discussion phase is about to close, we kindly wish to follow up, as we have not yet received any further comments or acknowledgment regarding our latest updates and responses.

Thank you for your time and consideration, and we look forward to your response.

Best regards,
Authors
</content>
</thread_object>

</thread>
<thread>
creation date:
1733311405273

thread_type:
official_comment

<thread_object>
creator: authors

<content>
title:
Summary of Reviews and Responses

comment:
**Summary of our paper**:Our paper introduces the Self-Information Rewrite Attack , a method that exploits vulnerabilitiy in current text watermarking algorithms that require embed pattern in high-entropy tokens . By identifying high-entropy tokens where watermarks are typically embedded, SIRA effectively removes these watermarks through a **black-box** approach. 

**Experimental Results**:Experimental results demonstrate that SIRA achieves **SOTA** attack performance with over 90% attack success rate in 6 recent watermark removal while maintaining efficiency in execution time and computational resources. Our method achieves a **30%-60% improvement** in attack success rates and **50% reduction in cost** compared to previous algorithms, 

**Contribution**: Our paper suggesting **critical vulnerabilities** in current watermarking techniques. Meanwhile our algorithm **change the brute-force paradigm of exisiting paraphrasing attacks**, reducing resource consumption and effectively lowering the requirements for watermark research. It has the potential to become an easy-to-use tool for testing watermark robustness.


-------
We appreciate that reviewers found our proposed method **effective(91eq,ATEL), efficient(EJ9Y,ATEL)**  and considered the paper well-written,experiments are **comprehensive(EJ9Y,91eq)** . Our paper **addresses a widely recognized problem** in the field of text watermarking(91eq).

**Key initial concerns included**:
> **Semantic preservation(EJ9Y,Aqqw)** 

We clarify that semantic preservation is related to the performance of the rewriting model. For efficiency, we opted for a small model in our initial paper. To resolve the concern by the reviewers, we included experiments with Llama3-70B and ChatGPT-4, demonstrating that stronger models significantly enhance semantic preservation.

> **Motivation to reduce resources requirement (EJ9Y)** 

We clarify that our goal is to reveal vulnerabilities in watermarking methods and lower research barriers, while demonstrating our method's scalability through experiments.

> **Novelty compared to current Paraphrasing Attack(91eq)** 

We clarify that current paraphrase attacks are untargeted and brute-force, while our method is more controllable, effective, and achieves SOTA performance according to our experiment.

> **Comparison with watermark stealing attack(91eq)** 

We added relevant sections to the appendix as per the reviewer's suggestions,we emphasize that our method is fundamentally different from watermark stealing, as evidenced by the fact that such methods cannot operate in black-box settings.
> **The advantage of self-information compared to probability and entropy**(91eq)

We added the requested ablation experiments and included theoretical explanations in the corresponding appendix section.
> **Ablation regarding iterative paraphrasing(AqqW)** 

We have added the requested ablation experiments as per the reviewer’s suggestions.The experimental results demonstrate the effectiveness of our method.
> **How can SIRA achieve shorter time and comparison with GPT Paraphraser(AqqW,ATEL)**


Following the reviewer’s suggestions, we conducted a more rigorous experiment and included a cost analysis for using third-party services.

> **Results are inconsistency(ATEL)**

We emphasize that the inconsistency mentioned by the reviewer refers to two different experiments, and thus, this concern is **not valid**.

-------
**During the rebuttal period**

**Reviewer EJ9Y** acknowledged our contributions, increasing the contribution score from 2 to 3 and raising the overall rating from 5 to 6.

**Reviewer AqqW** decided to keep the score at 5.

**Reviewer 91eq, ATEL** has not responded to our rebuttal.

-------

We have added the experiments requested by the reviewers and provided further clarification on our novelty and any potential misunderstandings. Unfortunately, during the rebuttal period, we did not receive additional feedback from Reviewer ATEL or Reviewer 91eq. However, we are confident that we have addressed all the concerns raised by reviewers. **We would like to sincerely thank all the reviewers and the AC for their valuable comments and feedback, which have helped us improve the quality of our paper**.
</content>
</thread_object>

</thread>


Metareview: