Paper: On the Learnability of Watermarks for Language Models (ICLR-2024)

Reviewer comment (Novelty):
The method presented is neither novel nor new. Knowledge distillation is a well-established concept, and this paper simply tests existing watermarking methods in conjunction with it. The paper appears more evaluative in nature.

Author response:
We thank you for the useful and insightful feedback and for taking the time to review our paper. We address questions and concerns below.

**Contribution**

Please see our general comment clarifying our contribution. https://openreview.net/forum?id=9k0krNzvlV&noteId=UyaqBwxSNh Our main contribution is our empirical investigation and findings on watermark learnability. It is true that knowledge distillation is a well-established concept, but our method is not our primary contribution. Our findings and evaluations are novel and new, as the learnability of decoding-based text watermarking was previously unexplored, and it was unknown whether existing decoding-based watermarks are learnable or not.

Reviewer comment (Novelty):
There's a body of prior works focused on watermark distillation. These works show that if the student model distills the watermarked teacher model, it offers protection against model extraction attacks. Although the context may differ, it is better for the author to acknowledge this in the related works section. Key references include:  
- He et al., "Protecting Intellectual Property of Language Generation APIs with Lexical Watermark," AAAI 2022.  
- Zhao et al., "Distillation-Resistant Watermarking for Model Protection in NLP," EMNLP 2022.  
- He et al., "CATER: Intellectual Property Protection on Text Generation APIs via Conditional Watermarks," NeurIPS 2022.  
- Zhao et al., "Protecting Language Generation Models via Invisible Watermarking," ICML 2023.

Author response:
**Relationship to model watermarking for protection against model extraction attacks**

We have updated the related work section to include a discussion of the relationship of these provided works with ours. 

These works concern model watermarking for protection against model extraction attacks. Model watermarking for model protection is crucially different from text watermarking, which we study. Model watermarking is meant to protect language model APIs from model stealing/extraction attacks, where an adversary trains a model on outputs from the victim model API. The model watermark enables detecting if a suspect model was extracted from the victim model. Detection is done by querying the suspect model multiple times on specially chosen inputs. The model watermark is specifically designed to be learnable, so that the adversary model learns the watermark during the model extraction attack. On the other hand, text watermarking enables detecting if a given text was model-generated. Detection does not need access to any language model, only the tokenizer, and detection is meant to be performed on any generated text, regardless of the input prompt. Existing decoding-based text watermarking methods are specifically designed to be subtle and imperceptible in order to have minimal impact on text quality, and it was previously unknown whether they are learnable or not.

In short, the object being detected is different (model itself versus generated text), and model watermarking for model protection is specifically designed to be learnable, whereas the learnability of existing text watermarks was previously unexplored and unknown. Because of these crucial differences, these prior works into model watermarking for protection against model extraction attacks do not diminish the value and significance of our findings.

Reviewer comment (Evaluation):
Although the p-value is utilized for watermark detection, I also recommend that the author present quantitative metrics such as AUC, FPR, etc., for a more comprehensive understanding of watermark detection.

Author response:
**AUROC and FPR**

As requested, we have included the AUROC metric for classifying watermarked and non-watermarked text for a more comprehensive understanding of watermark detection, as shown below for the main logits-based watermark distillation (WD-L) and sampling-based watermark distillation (WD-S) experiments. Dec indicates decoding-based watermarking. Additional full tables can be found in the paper.

| Model | Dec AUROC |WD-L AUROC | WD-S AUROC |
| --- | --- | --- | --- |
| KGW $\delta=2$ | 1.000 | 0.997 | 0.995 |
| KGW $\delta=1$ | 0.992 | 0.913 | 0.874 |
| Aar $k=2$ | 0.999 | 0.996 | 0.993 |
| Aar $k=3$ | 1.000 | 0.773 | 0.866 |
| Aar $k=4$ | 0.999 | 0.580 | 0.631 |
| KTH $s=1$ | 0.994 | 0.998 | 0.989 |
| KTH $s=2$ | 0.998 | 0.994 | 0.972 |
| KTH $s=4$ | 0.995 | 0.969 | 0.955 |
| KTH $s=256$ | 0.996 | 0.893 | 0.955 |

Because watermark detection returns p-values, the false positive rate is equal to the significance level that is chosen as the threshold for classifying a text as model-generated or not. So we do not report FPR because it is set by the detection user depending on the FPR they are willing to tolerate.

Reviewer comment (Experiments):
The experimental methodology could be strengthened by controlling for model architecture (use the Llama-2 as the student model for both logits-based and sample-based experiments for a more intuitive comparison of the results). This would better isolate the distillation techniques themselves.

Author response:
**Llama-2 sampling-based watermark distillation evaluation**

As requested, we have included an evaluation of sampling-based watermark distillation using Llama 2 7B as both the teacher and student model. We use the same evaluation procedure as before. Results are shown below, and also have been added to the paper. The trends are broadly similar to the original logits-based and sampling-based distillation experiments.

| Model | Dec p-value | WD-S p-value | Dec AUROC | WD-S AUROC | Dec LM-score | WD-S LM-score | Dec seq-rep-3 | WD-S seq-rep-3 | Dec MAUVE | WD-S MAUVE |
| --- | --- | --- | --- | --- | --- | --- | --- | --- | --- | --- |
| KGW $\delta=2$ | 4.4e-18 | 8.5e-07 | 1.000 | 0.996 | 16.4 | 19.7 | 0.04 | 0.04 | 0.89 | 0.85 |
| KGW $\delta=1$ | 1.0e-05 | 2.0e-02 | 0.992 | 0.869 | 12.5 | 15.4 | 0.03 | 0.03 | 0.91 | 0.89 |
| Aar $k=2$ | 1.3e-75 | 5.1e-17 | 0.999 | 0.984 | 6.5 | 7.6 | 0.34 | 0.34 | 0.28 | 0.29 |
| Aar $k=3$ | 4.8e-73 | 8.5e-03 | 1.000 | 0.873 | 9.5 | 10.5 | 0.14 | 0.17 | 0.74 | 0.66 |
| Aar $k=4$ | 4.0e-72 | 2.6e-01 | 0.999 | 0.651 | 10.7 | 11.9 | 0.09 | 0.10 | 0.86 | 0.83 |
| KTH $s=1$ | 1.0e-04 (596) | 1.0e-04 (560) | 0.994 | 0.998 | 10.5 | 15.3 | 0.03 | 0.03 | 0.88 | 0.86 |
| KTH $s=2$ | 1.0e-04 (600) | 1.0e-04 (526) | 0.998 | 0.994 | 10.7 | 13.5 | 0.03 | 0.04 | 0.91 | 0.87 |
| KTH $s=4$ | 1.0e-04 (593) | 1.0e-04 (485) | 0.995 | 0.984 | 10.5 | 12.6 | 0.03 | 0.04 | 0.91 | 0.87 |
| KTH $s=256$ | 1.0e-04 (600) | 1.0e-04 (455) | 0.996 | 0.968 | 10.61 | 11.3 | 0.03 | 0.04 | 0.92 | 0.90 |
|Base student | 5.0e-01 | | 0.500 | | 11.7 | | 0.03 | | 0.92 | |

However, we note that logits-based and sampling-based distillation have key differences, which means that they should not be compared solely based on efficacy to determine which method is better. Logits-based distillation requires white-box access to the teacher model and that the teacher and student models share the same tokenizer and vocabulary. It is also convenient if the teacher and student models share the same model architecture. On the other hand, sampling-based distillation only requires black-box access to samples from the teacher model, and the teacher and student models can have different tokenizers and vocabularies. Also, sampling-based distillation requires autoregressive sampling from the teacher model, which is slow, whereas logits-based distillation does not require sampling and is faster overall. So, logits-based and sampling-based distillation are each suitable and applicable for different settings, so neither is strictly better than the other across all scenarios.

Reviewer comment (Experiments):
The experimental methodology could be strengthened by increasing the dataset diversity (currently, only one dataset is used).

Author response:
**Increasing dataset diversity**

We have included additional evaluations of logits-based and sampling-based watermark distillation (including both Llama 2 7B and Pythia 1.4B as student model) on two additional datasets, Wikipedia articles and arXiv papers. We observe similar trends to the original main evaluation on the C4 RealNewsLike dataset. The full results are in Appendix F.

Reviewer comment (Experiments):
The experimental methodology could be strengthened as, somehow, seq-rep-3 is not evaluated on the sample-based distillation.

Author response:
**Seq-rep-3 on sampling-based distillation**

Apologies for the confusion, we originally omitted seq-rep-3 on sampling-based distillation in order to respect margin constraints, but we have now re-included it in Appendix B, as shown below.

| Model | DecT seq-rep-3 | DecS seq-rep-3 | WD-S seq-rep-3 |
| --- | --- | --- | --- |
| KGW $\delta=2$ | 0.04 | 0.04 | 0.02 |
| KGW $\delta=1$ | 0.03 | 0.03 | 0.02 |
| Aar $k=2$ | 0.34 | 0.53 | 0.23 |
| Aar $k=3$ | 0.14 | 0.27 | 0.10 |
| Aar $k=4$ | 0.09 | 0.12 | 0.05 |
| KTH $s=1$ | 0.03 | 0.03 | 0.02 |
| KTH $s=2$ | 0.03 | 0.03 | 0.02 |
| KTH $s=4$ | 0.03 | 0.03 | 0.02 |
| KTH $s=256$ | 0.03 | 0.03 | 0.02 |
|Base student | | 0.03 | |

Reviewer comment (Writing):
The technical writing is condensed in some areas, making it harder to follow for non-experts. Expanding explanations around key concepts like the distillation objectives could improve readability.

Author response:
**Expanding explanations around distillation objectives**

We have expanded the explanations around the watermark distillation objectives to improve readability.

Reviewer comment (Experiments):
The limitations around fine-tuning robustness are acknowledged but not thoroughly investigated. Exploring techniques to improve robustness could be valuable.

Author response:
**Robustness to fine-tuning**

We agree that improving robustness of weights-based watermarking to fine-tuning is an important challenge towards developing watermarking for open models. However, we believe that methods to improve robustness to fine-tuning are not as directly related to our main focus of investigating watermark learnability and would best be explored in future work.

Reviewer comment (Experiments):
Conducting more in-depth studies on spoofing attacks across diverse topics and datasets could further increase the impact of the work's findings. Expanding the spoofing experiments to include varied contexts beyond the current proof of concept could substantiate the risks posed. Rigorously demonstrating successful spoofing in different settings would validate concerns around watermark security and attribution. This expanded analysis could strengthen the evaluative assessment by demonstrating comprehensive consideration of the practical implications.

Author response:
**More in-depth spoofing experiments**

We agree that more in-depth studies and evaluations of watermark spoofing attacks across different settings would be valuable for more comprehensively evaluating the practical implications and risks of spoofing attacks. However, we believe that our findings are still valuable and lay a foundation for future work in this direction. We demonstrate that sampling-based distillation is able to learn to generate watermarked text from black-box watermarked samples, which is the setting for spoofing attacks. We also include experiments on sample efficiency (Appendix D) and different generation datasets (Appendix F). These findings suggest the possibility of spoofing attacks, and we show a simple proof-of-concept spoofing attack to substantiate their possibility. We hope that our work will inform future research and responsible usage of watermark detection.

Reviewer comment (Novelty):
As the authors show, open-model watermarking using distillation is not robust against fine-tuning. I am unclear about the contribution of the authors to open model watermarking. No prior work has used distillation to watermark LMs, hence there is no security threat. What is the use of a watermark that (i) lacks robustness and (ii) can be spoofed by design? I would love to hear the author's thoughts on this.

Author response:
**Contribution to open model watermarking**

Please see our general comment clarifying our contribution. https://openreview.net/forum?id=9k0krNzvlV&noteId=UyaqBwxSNh One important technical challenge towards open model watermarking is robustness to decoding procedure, since users of open models can run inference themselves and choose the decoding procedure, so it is easy to not use a decoding-based watermark algorithm. Our findings on watermark learnability and weights-based watermarking address this key challenge, taking a necessary first step towards developing open model watermarking. However, as you correctly point out, robustness to fine-tuning is another important technical challenge which we leave for future work. We believe that our contribution is still valuable because it investigates a necessary first step towards open model watermarking that was previously unexplored.

Regarding point (ii), we do not design watermarks to be spoofable. We investigate the learnability of existing decoding-based watermarks, rather than design new watermark algorithms, and high learnability suggests the possibility of being vulnerable to spoofing attacks. Spoofability suggests that watermarks should not be used to assign provenance or blame to a specific model, but it does not hinder the primary case of detecting model-generated text, e.g., preventing students from cheating on essays or preventing LLM-generated misinformation campaigns. Watermarking is still useful and valuable even if it is spoofable.

Reviewer comment (Experiments):
One would expect that watermarking can be spoofed if the attacker can access sufficiently many watermarked samples. If I understood correctly, the authors use **163 million tokens** ($640$k samples times 256 tokens each) of watermarked text with the _same_ message. How realistic is this assumption? Is the attack effective when a provider uses a different message after generating a set number of tokens?

Author response:
**Using multiple different keys/messages**

We run additional experiments in Appendix E to investigate the behavior of sampling-based watermark distillation if the samples are watermarked using multiple different keys/messages. We also vary the number of training samples as before. The table below shows watermark detection p-values.

		                     Number of training samples
| Model | 20k | 40k | 80k | 160k | 320k | 640k |
| --- | --- | --- | --- | --- | --- | --- |
| KGW $\delta = 2$ (one key) | 4.1e-02 | 1.4e-02 | 2.4e-03 | 5.2e-04 | 5.1e-05 | 3.6e-06 |
| KGW $\delta = 2$ (two keys) | 1.7e-01 | 1.4e-01 | 7.8e-02 | 5.7e-02 | 2.9e-02 | 1.4e-02 |
| Aar $k = 2$ (one key) | 7.4e-02 | 9.4e-03 | 3.5e-04 | 1.0e-06 | 5.1e-10 | 2.9e-14 |
| Aar $k = 2$ (two keys) | 4.2e-01 | 2.3e-01 | 1.2e-01 | 4.0e-02 | 1.0e-02 | 1.2e-03 |

The results show that using two different keys hinders watermark learning, as indicated by the larger p-values. However, learning is not completely prevented, as the p-values are still much lower than 0.5 and decrease as the number of training samples increases.

This suggests that using multiple watermark keys for generation may be a potential defense against spoofing attacks. However, if multiple watermark keys are used for generation, then all of those keys will need to be tested at detection, so multiple testing correction will need to be performed to obtain accurate, weaker p-values. So, this defense will slightly reduce the statistical power and increase the false negative rate of watermark detection.

Reviewer comment (Experiments):
Spoofing is possible when sufficiently many samples are revealed. I believe a more interesting approach is to measure the number of samples that enable spoofing or to derive a theoretical bound for the spoofer's sample efficiency. Insights into that question could guide the party that injects the watermark into deploying defenses, such as switching to a different key or message after this number of samples has been revealed. Unfortunately, the paper does not investigate this problem. Do the authors have any insights into that problem?

Author response:
**Sample efficiency**

To investigate sample efficiency, we have included additional experiments measuring watermark detection p-values as we vary the number of training samples for sampling-based watermark distillation. As in the main experiments, each sample is 256 tokens long, and we use Llama 2 7B as the teacher model and Pythia 1.4B as the student model. Note that the training batch size is held the same, so fewer training samples means fewer gradient steps. P-value results are shown below, and we have also included a plot of p-values against the number of training samples in the paper (Appendix D).

		               Number of training samples
| Model | 20k | 40k | 80k | 160k | 320k | 640k |
| --- | --- | --- | --- | --- | --- | --- |
| KGW $\delta = 2$| 4.1e-02 | 1.4e-02 | 2.4e-03 | 5.2e-04 | 5.1e-05 | 3.6e-06 |
| Aar $k = 2$ | 7.4e-02 | 9.4e-03 | 3.5e-04 | 1.0e-06 | 5.1e-10 | 2.9e-14 |
| KTH $s = 1$ | 7.4e-02 | 4.2e-02 | 1.0e-02 | 1.1e-03 | 6.0e-04 | 1.0e-04 |


As expected, the results show that as the number of training samples increases, the watermark is learned more strongly. However, even at small numbers of training samples, the watermark is still learned (the p-values are much lower than chance), just to a lesser degree. Overall, we note that watermark learnability and sample efficiency is a continuous spectrum. More samples are helpful but not necessarily always crucial, and sample efficiency varies across watermarking strategies and hyperparameters.

Reviewer comment (Novelty):
What are your contributions towards open model watermarking?

Author response:
**Contribution to open model watermarking**

Please see our general comment clarifying our contribution. https://openreview.net/forum?id=9k0krNzvlV&noteId=UyaqBwxSNh One important technical challenge towards open model watermarking is robustness to decoding procedure, since users of open models can run inference themselves and choose the decoding procedure, so it is easy to not use a decoding-based watermark algorithm. Our findings on watermark learnability and weights-based watermarking address this key challenge, taking a necessary first step towards developing open model watermarking. However, as you correctly point out, robustness to fine-tuning is another important technical challenge which we leave for future work. We believe that our contribution is still valuable because it investigates a necessary first step towards open model watermarking that was previously unexplored.

Regarding point (ii), we do not design watermarks to be spoofable. We investigate the learnability of existing decoding-based watermarks, rather than design new watermark algorithms, and high learnability suggests the possibility of being vulnerable to spoofing attacks. Spoofability suggests that watermarks should not be used to assign provenance or blame to a specific model, but it does not hinder the primary case of detecting model-generated text, e.g., preventing students from cheating on essays or preventing LLM-generated misinformation campaigns. Watermarking is still useful and valuable even if it is spoofable.

Reviewer comment (Evaluation):
How sample-efficient is the spoofing attack?

Author response:
**Sample efficiency**

To investigate sample efficiency, we have included additional experiments measuring watermark detection p-values as we vary the number of training samples for sampling-based watermark distillation. As in the main experiments, each sample is 256 tokens long, and we use Llama 2 7B as the teacher model and Pythia 1.4B as the student model. Note that the training batch size is held the same, so fewer training samples means fewer gradient steps. P-value results are shown below, and we have also included a plot of p-values against the number of training samples in the paper (Appendix D).

		               Number of training samples
| Model | 20k | 40k | 80k | 160k | 320k | 640k |
| --- | --- | --- | --- | --- | --- | --- |
| KGW $\delta = 2$| 4.1e-02 | 1.4e-02 | 2.4e-03 | 5.2e-04 | 5.1e-05 | 3.6e-06 |
| Aar $k = 2$ | 7.4e-02 | 9.4e-03 | 3.5e-04 | 1.0e-06 | 5.1e-10 | 2.9e-14 |
| KTH $s = 1$ | 7.4e-02 | 4.2e-02 | 1.0e-02 | 1.1e-03 | 6.0e-04 | 1.0e-04 |


As expected, the results show that as the number of training samples increases, the watermark is learned more strongly. However, even at small numbers of training samples, the watermark is still learned (the p-values are much lower than chance), just to a lesser degree. Overall, we note that watermark learnability and sample efficiency is a continuous spectrum. More samples are helpful but not necessarily always crucial, and sample efficiency varies across watermarking strategies and hyperparameters.

Reviewer comment (Evaluation):
Can the distilled and original watermarked distributions be distinguished? I assume the spoofed watermarked does not learn rare watermarked sequences, but instead focuses first on the most often occurring ones. If a defender can detect and ignore those during detection, can they evade your spoofing attack while effectively detecting watermarked text?

Author response:
**Distinguishing between distilled and original watermarked distributions**

Your assumption that the spoofed watermark tends to learn more common watermarked sequences first seems reasonable. If a defender tries to evade the spoofing attack by ignoring the most common watermarked sequences or n-grams, they could reduce the proportion of texts generated by the adversary which are detected as watermarked, but this would also reduce the proportion of texts genuinely generated by original model which are detected as watermarked. This is because the most common watermarked sequences/n-grams contribute more towards the watermark detection score. Rare watermarked sequences are less likely to appear, so it is more likely that a genuinely generated text contains none or few rare sequences, thus resulting in a false negative. So this attempt to evade the spoofing attack will reduce the watermark’s efficacy to detect model-generated texts, which is its primary purpose.

**Spoofing attacks vs defenses**

More generally, we believe that defenses against spoofing attacks vs. spoofing attacks to overcome these defenses is a security cat and mouse game, with no conclusive winner or answer at this time. Some new defense could later be defeated by a new attack, which is then defeated by a new defense, and so on. Ultimately, potential vulnerability to spoofing attacks is one reason among others suggesting that text watermarking should not be used to conclusively attribute provenance or blame to a specific model. Text watermarking is best suited for its original use case of detecting model-generated text, e.g., for preventing students from cheating on essays or preventing LLM-generated misinformation campaigns.

Reviewer comment (Theory):
The motivation of this paper is there is a limitation of decoding-based watermarking, namely, replacing it with a normal decoder. Is the assumption practical? In practice, for an LLM API, how do we conduct such an operation?

Author response:
For API-only LLMs, decoding-based watermarking is suitable, since the LLM provider controls generation, and users cannot replace the decoding-based watermarking algorithm with a normal decoder. However, for open LLMs, e.g., Llama 2, users run generation themselves, so they can easily choose not to use a decoding-based watermark algorithm, and many inference libraries do not watermark outputs by default or at all. So, decoding-based watermarking is unsuitable for robust watermarking for open LLMs. Our findings into weights-based watermarking address this key technical challenge towards watermarking for open LLMs, though additional challenges remain, e.g., robustness to further fine-tuning.

Reviewer comment (Experiments):
For wright-based LLM watermarking, more attacks (besides fine-tuning) shall be considered, such as pruning and further distillation.

Author response:
We demonstrate that simple fine-tuning removes weights-based watermarking capabilities, showing that developing weights-based watermarking that is robust to modifying model weights is an important unsolved challenge. Further attacks would not change our conclusion.

Reviewer comment (Presentation):
The paper exceeds 9 pages.

Author response:
Per the ICLR author guide https://iclr.cc/Conferences/2024/AuthorGuide, the optional ethics statement and reproducibility statement do not count towards the page limit. The main body of our paper does not exceed 9 pages.

Reviewer comment (Experiments):
For decoding-based watermarking, if different users are assigned different keys, will the distillation or spoofing attacks fail?

Author response:
We run additional experiments in Appendix E to investigate the behavior of sampling-based watermark distillation if the samples are watermarked using multiple different keys. We also run different trials varying the number of training samples. The table below shows watermark detection p-values.

                          Number of training samples
| Model | 20k | 40k | 80k | 160k | 320k | 640k |
| --- | --- | --- | --- | --- | --- | --- |
| KGW $\delta = 2$ (one key) | 4.1e-02 | 1.4e-02 | 2.4e-03 | 5.2e-04 | 5.1e-05 | 3.6e-06 |
| KGW $\delta = 2$ (two keys) | 1.7e-01 | 1.4e-01 | 7.8e-02 | 5.7e-02 | 2.9e-02 | 1.4e-02 |
| Aar $k = 2$ (one key) | 7.4e-02 | 9.4e-03 | 3.5e-04 | 1.0e-06 | 5.1e-10 | 2.9e-14 |
| Aar $k = 2$ (two keys) | 4.2e-01 | 2.3e-01 | 1.2e-01 | 4.0e-02 | 1.0e-02 | 1.2e-03 |

The results show that using two different keys hinders watermark learning, as indicated by the larger p-values. However, learning is not completely prevented, as the p-values are still much lower than 0.5 and decrease as the number of training samples increases.

Note that if multiple watermarking keys are used, then the watermark detection will lose statistical power. Any suspected text will need to be tested using each of the keys in use, so multiple testing correction must be performed, leading to weaker p-values and a higher false negative rate.

Reviewer comment (Evaluation):
What is the additional computational cost? Evaluate or compare it.

Author response:
Weights-based watermarking uses standard generation procedures, so there is no additional computational cost at inference time compared to normal LLMs. Compared to decoding-based watermarking, weights-based watermarking has less computational cost at generation time, since there is no need to run an additional decoding-based watermarking algorithm on top of the LLM. Weights-based watermarking does require model training, the computational cost of which varies depending on model size and dataset size, but this cost is one-time and not incurred at inference time.

Reviewer comment (Experiments):
Besides decoding-base LLM watermarking, there exist some backdoor-based LLM watermarking methods. Are the findings still suitable for them?

Author response:
Our findings concern the learnability of decoding-based text watermarking methods. Backdoor-based model watermarking is crucially different in both setting and method. Backdoor-based model watermarking is meant to enable detecting if a suspect model was stolen from a victim model, e.g., through leaking model weights. Detection is done by querying the suspect model multiple times on specially chosen inputs, i.e., the backdoored inputs. If the suspect model produces the chosen backdoored outputs on these backdoor inputs (which are chosen to be different from how a normal, non-backdoored model would behave), then the suspect model is detected as stolen. Note that the backdoored inputs are meant to be stealthy and hard to find, and the backdoored model still behaves normally on normal inputs. On the other hand, decoding-based text watermarking enables detecting if a given text was generated by an LLM using any input prompt. Detection is done without using any LLM, only the tokenizer is needed. Decoding-based text watermarking is designed to affect all model outputs regardless of input, so that ideally all generated texts can be detected as watermarked. Because of these crucial differences in both setting and method, our findings are not directly relevant to backdoor-based model watermarking, and vice versa.