Paper: Unbiased Watermark for Large Language Models (ICLR-2024)

Reviewer comment (Evaluation):
How do you plan to address concerns (in practice) regarding the potential manipulation or removal of watermarks by users?

Author response:
We believe the battle between watermarking and watermark removal will be a long-living problem. This dynamic has seen rapid developments recently. Our current research marks significant progress in this area by theoretically ensuring that users cannot detect the presence of watermarks. This advancement complicates the manipulation and removal of watermarks, as it becomes challenging to assess whether the watermark remains intact. We look forward to future research that will build upon these advancements.

Reviewer comment (Experiments):
The evaluations do not contain any attacks (the authors mentioned them in Appendix A2). I highly recommend the authors evaluate those attacks. If the proposed watermark methods suffer from a simple attack like paraphrasing, it would significantly undermine the paper's contributions.

Author response:
> The evaluations do not contain any attacks (the authors mentioned them in Appendix A2)

Actually, we have included preliminary evaluations against random substitution attack in Appendix F.5. Additionally, we have now supplemented our research with an evaluation against a random deletion attack:

|                 |     0.0      |     0.1      |     0.2      |     0.3      |     0.4      |     0.5      |
|-----------------|-------------|-------------|-------------|-------------|-------------|-------------|
| Delta_Reweight  | 0.9997±0.0005 | 0.9674±0.0026 | 0.9107±0.0061 | 0.8319±0.0056 | 0.7634±0.0011 | 0.7032±0.0044 |
| Gamma_Reweight  | 0.9936±0.0016 | 0.9348±0.0045 | 0.8616±0.0032 | 0.7697±0.0023 | 0.7020±0.0054 | 0.6577±0.0082 |
| Soft(delta=1.0) | 0.8446±0.0069 | 0.7895±0.0044 | 0.7377±0.0079 | 0.6903±0.0074 | 0.6387±0.0026 | 0.5894±0.0075 |
| Soft(delta=2.0) | 0.9705±0.0030 | 0.9257±0.0030 | 0.8697±0.0043 | 0.8042±0.0067 | 0.7336±0.0084 | 0.6626±0.0104 |

> If the proposed watermark methods suffer from a simple attack like paraphrasing, it would significantly undermine the paper's contributions.

Unfortunately, it is true that our watermarking methods are vulnerable to simple attacks. Specifically, although the watermarks can still be detected after changes to the input, the efficiency of this detection drops notably, necessitating longer sequences for reliable watermark detection. This vulnerability is fundamentally due to the fact that watermark detection relies on the presence of identical n-grams during watermark generation and detection. Any form of attack, including paraphrasing or simple word substitutions, could easily disrupt these n-grams, thereby reducing the efficiency of watermarks detection. This vulnerability is not unique to our work; it is a common challenge in the field of watermarking, similar to recent findings (Kirchenbauer et al. 2023).

The primary contribution of our paper is the development of the concept of an unbiased watermark. While we recognize the importance of robustness in watermarking research, our current work is not aimed at enhancing the robustness of watermarks. Our approach maintains a level of robustness on par with existing methodologies, as discussed in Appendix F.5.

Reviewer comment (Experiments):
Larger models like LLAMA-2 should also be evaluated to further enhance the practicality.

Author response:
> Larger models like LLAMA-2 should also be evaluated to further enhance the practicality.

We have conducted evaluations with larger models, including LLAMA-2 in our appendix. The results of these evaluations are detailed in Appendix F.1, where we used LLAMA-2 for text summarization and poetry generation tasks. Furthermore, we have expanded our experiments to include natural language generation with Wikitext and instruction following with FLAN, providing additional practical evidence for downstream invariant property:

| Model                       | Wikitext (Llama 2), PPL | Flan (Llama 2), PPL |
|-----------------------------|------------------------|---------------------|
| $\delta$-reweight           | 1.692±0.009                       | 2.57±0.03           |
| $\gamma$-reweight           | 1.719±0.011                       | 2.557±0.028         |
| Soft($\delta$=0.0)          | 1.693±0.011                       | 2.558±0.025         |
| Soft($\delta$=1.0)          | 1.761±0.011                       | 2.697±0.028         |
| Soft($\delta$=2.0)          | 2.025±0.017                       | 3.17±0.04           |

Reviewer comment (Experiments):
Is this because the OPT-6.7B model is not fine-tuned for the NW cases?

Author response:
We didn't finetune the model, and we use the same model OPT-6.7B across NW, $\delta$-RW, $\gamma$-RW to ensure a fair comparison.

The lower readability output in the 'NW' section is due to the inherent variability of the OPT-6.7B model's performance. This model sometimes generates lower-quality outputs, which is evident in the NW case. This is a characteristic of the OPT-6.7B model itself, rather than an effect of our watermarking strategy or a lack of fine-tuning.

In light of your feedback, we have replicated table 3 with a state-of-the-art model, LLaMA2. We find that the readability of the outputs improve significantly, and the variability of output quality also decreases. The updated results with LLaMA2 are as follows:

| Prompt   | What is a watermark? What’s the purpose of it? | score | p\-value |
|----|----|----|----|
|NW | What are the different types of watermarks?<br><br>A watermark is a transparent image or pattern embedded in a digital image, document or video, that can be seen when the image is enlarged or magnified. The purpose of a watermark is to provide a means of identifying the owner of the image, document or video, or to indicate that the image has been altered or copied without| 0.0 | 11|
| $\delta$-RW | What are the benefits of using a watermark? Are there different types of watermarks? How do I create a watermark for my images?<br><br>What is a watermark?<br><br>A watermark is a visual representation of a brand, logo, or text that is embedded or superimposed over an image or document. The purpose of a watermark is | 15.24 | 2.6e-6 |
| $\gamma$-RW | What are the different types of watermarks? What are the features of a digital watermarking system? Who are the parties involved in a digital watermarking process? What are the applications of digital watermarking? What are the challenges in digital watermarking? What is the future of digital watermarking? Conclusion Questions and Answers References and Recommendations | 8.86 | 0.00155 |  

We hope that the additional information will address your concerns. Once again, thank you for recognizing our work and providing valuable feedback.

Reviewer comment (Experiments):
I am concerned about whether this comparison is fair enough.

Author response:
We didn't finetune the model, and we use the same model OPT-6.7B across NW, $\delta$-RW, $\gamma$-RW to ensure a fair comparison.

The lower readability output in the 'NW' section is due to the inherent variability of the OPT-6.7B model's performance. This model sometimes generates lower-quality outputs, which is evident in the NW case. This is a characteristic of the OPT-6.7B model itself, rather than an effect of our watermarking strategy or a lack of fine-tuning.

In light of your feedback, we have replicated table 3 with a state-of-the-art model, LLaMA2. We find that the readability of the outputs improve significantly, and the variability of output quality also decreases. The updated results with LLaMA2 are as follows:

| Prompt   | What is a watermark? What’s the purpose of it? | score | p\-value |
|----|----|----|----|
|NW | What are the different types of watermarks?<br><br>A watermark is a transparent image or pattern embedded in a digital image, document or video, that can be seen when the image is enlarged or magnified. The purpose of a watermark is to provide a means of identifying the owner of the image, document or video, or to indicate that the image has been altered or copied without| 0.0 | 11|
| $\delta$-RW | What are the benefits of using a watermark? Are there different types of watermarks? How do I create a watermark for my images?<br><br>What is a watermark?<br><br>A watermark is a visual representation of a brand, logo, or text that is embedded or superimposed over an image or document. The purpose of a watermark is | 15.24 | 2.6e-6 |
| $\gamma$-RW | What are the different types of watermarks? What are the features of a digital watermarking system? Who are the parties involved in a digital watermarking process? What are the applications of digital watermarking? What are the challenges in digital watermarking? What is the future of digital watermarking? Conclusion Questions and Answers References and Recommendations | 8.86 | 0.00155 |  

We hope that the additional information will address your concerns. Once again, thank you for recognizing our work and providing valuable feedback.

Reviewer comment (Experiments):
The tasks included in the experiment are only summarization and translation, and more tasks, such as general natural language generation, are required to demonstrate the effectiveness of the proposed method.

Author response:
We appreciate your suggestion to extend our experiments beyond translation and summarization tasks. We have expanded our datasets to include wikitext for natural language completion and the FLAN dataset for instruction-following tasks. Our results demonstrate the downstream-invariant properties across these new tasks.

| Model                       | Wikitext (Llama 2), PPL | Flan (Llama 2), PPL |
|-----------------------------|------------------------|---------------------|
| $\delta$-reweight           | 1.692±0.009                       | 2.57±0.03           |
| $\gamma$-reweight           | 1.719±0.011                       | 2.557±0.028         |
| Soft($\delta$=0.0)          | 1.693±0.011                       | 2.558±0.025         |
| Soft($\delta$=1.0)          | 1.761±0.011                       | 2.697±0.028         |
| Soft($\delta$=2.0)          | 2.025±0.017                       | 3.17±0.04           |

Reviewer comment (Experiments):
It seems that the robustness towards the exiting watermarking attacks are not verified in the paper. It seems that only in Appendix F.5, the authors evaluate the robustness under random perturbation. More attacks mentioned in Appendix A.2 are expected.

Author response:
Your point about the robustness of our watermarking method was well-taken. To address this, we've now incorporated tests against random token deletion. These additional experiments showcase the robustness of our methods. Exploring a broader range of sophisticated watermarking attack types, including emoji attack, will be a focus of our future work.

|                 |     0.0      |     0.1      |     0.2      |     0.3      |     0.4      |     0.5      |
|-----------------|-------------|-------------|-------------|-------------|-------------|-------------|
| Delta_Reweight  | 0.9997±0.0005 | 0.9674±0.0026 | 0.9107±0.0061 | 0.8319±0.0056 | 0.7634±0.0011 | 0.7032±0.0044 |
| Gamma_Reweight  | 0.9936±0.0016 | 0.9348±0.0045 | 0.8616±0.0032 | 0.7697±0.0023 | 0.7020±0.0054 | 0.6577±0.0082 |
| Soft(delta=1.0) | 0.8446±0.0069 | 0.7895±0.0044 | 0.7377±0.0079 | 0.6903±0.0074 | 0.6387±0.0026 | 0.5894±0.0075 |
| Soft(delta=2.0) | 0.9705±0.0030 | 0.9257±0.0030 | 0.8697±0.0043 | 0.8042±0.0067 | 0.7336±0.0084 | 0.6626±0.0104 |

Reviewer comment (Presentation):
Minor: Since the highlight of the proposed method is the unbiased property, in my opinion, the theorems 14 should be in the main body instead of the appendix.

Author response:
Although we briefly mention that "Both methods are unbiased1" on page 4, we agree that a formal presentation of Theorem 14 in the main body would better clarify the argument. Accordingly, we will revise our manuscript to include Theorem 14 in a more prominent section.

Reviewer comment (Experiments):
While the LLR scheme is novel and interesting, it is not clear how robust the maximin LLR scheme is to different scenarios. Meaningful ablation experiments are missing.

Author response:
>  it is not clear how robust the maximin LLR scheme is to different scenarios. Meaningful ablation experiments are missing.

We kindly direct your attention to Section F.2 in the appendix, where we provided ablation experiments specifically to explore the robustness of the maximin LLR approach in different scenarios. These experiments cover variations in temperature settings, top-k selections, different prompts, and different language models.

We believe that these experiments meaningfully address concerns regarding the robustness of our approach.

Reviewer comment (Experiments):
How does the watermarking scheme perform on context completion and instruction following tasks? Is it possible to present some empirical evidence on how many tokens are needed to achieve a certain level of detectability on open source LLMs?

Author response:
> How does the watermarking scheme perform on context completion and instruction following tasks?

We've broadened our datasets to include wikitext for context completion and flan for instruction following. The downstream-invariant property is also observed in these new tasks.

| Model                       | Wikitext (Llama 2), PPL | Flan (Llama 2), PPL |
|-----------------------------|------------------------|---------------------|
| $\delta$-reweight           | 1.692±0.009                       | 2.57±0.03           |
| $\gamma$-reweight           | 1.719±0.011                       | 2.557±0.028         |
| Soft($\delta$=0.0)          | 1.693±0.011                       | 2.558±0.025         |
| Soft($\delta$=1.0)          | 1.761±0.011                       | 2.697±0.028         |
| Soft($\delta$=2.0)          | 2.025±0.017                       | 3.17±0.04           |

We want to emphasize that our methods are supported by rigorous theoretical guarantee, and the downstream-invariant property is independent of specific task.

> Is it possible to present some empirical evidence on how many tokens are needed to achieve a certain level of detectability on open source LLMs?

Yes. This can be computed by the score per token. For instance, if each token increases the score on average by 0.8784 (as reported in Table 2), then 12 tokens would be enough to achieve a score of 10.5, which means watermark can be detected with p-value < 0.0005, while 23 tokens would bring it down to p-value < 3e-8.

Reviewer comment (Experiments):
What happens if you do not have access to the original LLM? (e.g., if you watermarked with ChatGPT but ran verification with Llama-1b, the shift in distributions would be quite large). Would the maximin LLR scheme outperform a z-score based statistical test?

Author response:
> What happens if you do not have access to the original LLM? (e.g., if you watermarked with ChatGPT but ran verification with Llama-1b, the shift in distributions would be quite large).

We have provided a pertinent ablation study in Section F.2, as indicated in Table 9, where we employ a different language model for watermark detection. Although we were unable to conduct experiments directly with ChatGPT, we believe that experiment closely reflects the scenario under consideration by the reviewer.

We find that the gamma reweight method is able to substantially maintain watermark strength even when switching models. In the case of the delta reweight scheme, while there is a large decrease in the watermark's detection capability, it still remains functional. As shown in Table 9, the score per token for delta reweight on a different model is about 0.041. This is still significant as, for instance, with 200 tokens, the cumulative score would be approximately 8.2, enabling watermark detection with a p-value of less than 0.0003.

Moreover, we have also developed watermark detection methods that are fully independent of the language model. These are detailed in Sections D and F.3 of our appendix. These likelihood-agnostic watermarking methods offer additional flexibility in scenarios where access to the language model is not available.

> Would the maximin LLR scheme outperform a z-score based statistical test?

To clarify, while both the maximin LLR scheme and z-score scheme are statistical tests and they share the same null hypothesis H0, they test under different alternative hypotheses H1 which corresponds to different watermark distributions. Our maximin LLR scheme aims to distinguish between an unbiased watermark distribution and the original distribution, while the z-score based statistical test is designed to differentiate between a biased watermark distribution and the original distribution. 

Therefore, a direct comparison between the two must be approached with caution. If a comparison is necessary, one could refer to the AUCs presented in Section F.5 of our paper.

Reviewer comment (Experiments):
Even for the same model, if you have edits (e.g., you don't have access to the prompt, or if someone has shuffled the sentences in the text), does the maximin LLR scheme work?

Author response:
> Even for the same model, if you have edits (e.g., you don't have access to the prompt, or if someone has shuffled the sentences in the text), does the maximin LLR scheme work?

We have conducted relevant ablation studies concerning these situations. In Section F.2 Table 8 of our appendix, we considered the scenario where one does not have access to the prompt. Furthermore, the impact of text perturbation was explored in Section F.5.