Paper: I Know You Did Not Write That! A Sampling Based Watermarking Method for Identifying Machine Generated Text (ICLR-2024)

Reviewer comment (Experiments):
Dependency on Datasets and Prompts: The performance of the watermarking method seems to be influenced by the given prompts and datasets used in the experiments. For instance, watermarking the output to factual questions with limited flexibility in answers is noted as challenging.

Author response:
We agree that more datasets and models will be useful for more conclusive findings. However,  due to space limitations and the cost of experiments (in terms of time and hardware requirements), we could only use three different models and two different datasets to increase the reliability of our results. We also conducted experiments in which we vary the temperature under generation. This allowed us to evaluate our methods with various probability distribution entropies. We would like to note that our experiments with various temperature values also address the need of using different tasks. For instance, in the task of answering factual questions, we expect that the entropy of the probability distribution will be low while we expect the opposite (i.e., high entropy) when we utilize LLMS to generate stories without any restriction to be factual.

Reviewer comment (Experiments):
Focus on Text Completion Tasks: The evaluation of the watermarking method is mainly conducted in the context of text completion tasks. The applicability and effectiveness of the watermark across different downstream tasks, such as question-answering and summarization, are suggested as areas for future evaluation.

Author response:
We agree that more datasets and models will be useful for more conclusive findings. However,  due to space limitations and the cost of experiments (in terms of time and hardware requirements), we could only use three different models and two different datasets to increase the reliability of our results. We also conducted experiments in which we vary the temperature under generation. This allowed us to evaluate our methods with various probability distribution entropies. We would like to note that our experiments with various temperature values also address the need of using different tasks. For instance, in the task of answering factual questions, we expect that the entropy of the probability distribution will be low while we expect the opposite (i.e., high entropy) when we utilize LLMS to generate stories without any restriction to be factual.

Reviewer comment (Experiments):
Can the proposed method be extended to watermark multiple models?

Author response:
As also the reviewer expresses the model-agnostic nature of our approach as one of the strengths of our paper, our proposed approach is not a model specific watermarking method. It can be applied on any LLM.

Reviewer comment (Evaluation):
The performance of the proposed method is not significantly better than the baseline method and is influenced by various factors such as the models used, the datasets, and the given prompts. I doubt the generalizability of the proposed method based on the current experiments.

Author response:
We agree that more datasets and models will be useful for more conclusive findings. However,  due to space limitations and the cost of experiments (in terms of time and hardware requirements), we could only use three different models and two different datasets to increase the reliability of our results. We also conducted experiments in which we vary the temperature under generation. This allowed us to evaluate our methods with various probability distribution entropies. We would like to note that our experiments with various temperature values also address the need of using different tasks. For instance, in the task of answering factual questions, we expect that the entropy of the probability distribution will be low while we expect the opposite (i.e., high entropy) when we utilize LLMS to generate stories without any restriction to be factual.

Reviewer comment (Evaluation):
The use of a paraphrasing similarity model (P-SP) to evaluate generation quality is questionable. If the premise is that the generated text should be similar or paraphrasing of the human text, isn't this contrary to the premise of open-ended text generation? This seems like a misapplication of P-SP.

Author response:
“....Why is a paraphrasing similarity model (P-SP) being used to evaluate generation quality?”  
Regarding the usage of paraphrasing similarity model evaluation, we use the paraphrasing similarity model (P-SP) because it was also used in prior work on watermarking, e.g., Kirchenbauer et al. (2023b).

Reviewer comment (Evaluation):
The reviewer suspects the metrics underestimate the degradation caused by SWOR, as for low-entropy predictions, it forces the model to sample uniformly among unlikely candidates, which seems quite bad.

Author response:
Firstly, we realized that we were not clear enough to emphasize the differences between SWR and SWOR approaches. When in sampling without replacement (SWOR) approach, we ensure that we select y different candidate tokens. Therefore their corresponding secret numbers will be distinct. In contrast, sampling with replacement (SWR) approach allows a token to be selected multiple times, and there is no assurance of uniqueness among the y candidate tokens. This becomes relevant when dealing with distributions of low entropy, where the probability mass is heavily concentrated on one or a few tokens. In such cases, SWR may repeatedly sample the same token, reducing the number of distinct candidate tokens.

Reviewer comment (Experiments):
Question: Why do the reports of experimental results distinguish between a SWR detector and a SWOR detector? Aren't these the same algorithm?

Author response:
Firstly, we realized that we were not clear enough to emphasize the differences between SWR and SWOR approaches. When in sampling without replacement (SWOR) approach, we ensure that we select y different candidate tokens. Therefore their corresponding secret numbers will be distinct. In contrast, sampling with replacement (SWR) approach allows a token to be selected multiple times, and there is no assurance of uniqueness among the y candidate tokens. This becomes relevant when dealing with distributions of low entropy, where the probability mass is heavily concentrated on one or a few tokens. In such cases, SWR may repeatedly sample the same token, reducing the number of distinct candidate tokens.

Reviewer comment (Experiments):
This approach does not have a factor that can control the strength of the watermark. It always chooses the candidate token that can maximize the secret number. If we can control the strength of the watermark injection, then we can balance the tradeoff between the quality of generated watermarked text and the strength of the watermark.

Author response:
We also realized that we were not clear about how to control  the strength of the watermark. The sampling count has a direct impact on the strength of the watermark. In particular, as shown in Table 3 and discussed in section 5.2.4 as we increase the sampling count y, there are more distinct candidate tokens to maximize the secret number from. Maximizing from more distinct values results in a higher overall secret number value which results in a more detectable watermark.

Reviewer comment (Novelty):
What is the main difference between the proposed approach with Kirchenbauer et al. (2023a)? The proposed approach does not interfere with the probability distribution of LLM, but it chooses the tokens according to their secret numbers instead of the original probability distribution. Is this just another way to change the probability distribution implicitly because the proposed approach adds some “logits” to the token with the largest secret number?

Author response:
We thank the reviewer for the insightful reviews. We understand that we should have put more emphasis on the distinctions between our approach and Kirchenbauer et al.’s approach. We first would like to note that all watermarking methods for LLMs would need to somehow affect probability distributions. Therefore, the main difference between watermarking methods for LLMs will be how these probability distributions are changed and the impact of these changes on the resultant texts. Now we would like to highlight these differences.

1) In Kirchenbauer  et. al.’s  approach, after the probability distribution for the next token is computed, a randomly selected subset of tokens (depending on the previous token(s)) are promoted.  In contrast, in our approach we sample from the distribution y (the sampling count) times. From these y candidate tokens we pick one that maximizes the secret number (essentially uniformly sampling from the candidate tokens). We would like to note that the candidate tokens are all tokens that could have been sampled by the sampler in the first place. 

2) The reviewer mentioned that our candidate tokens can be considered as “green list” and other tokens as “red list”, similar to Kirchenbauer  et. al.’s  approach. We kindly disagree with  this conclusion because there is a slight difference: We pick the token that maximizes the secret number from the y candidate tokens which are selected based on only LLM’s probability distribution (i.e., without any promotion of randomly selected tokens). We don’t pick tokens that are not in this set. We try to guarantee the “quality” of the y candidate tokens by directly sampling from the distribution.

3) Another distinction between Kirchenbauer 2023a’s approach and ours is the way we handle low entropy distributions. If the probability mass is concentrated in a single or few tokens, in our approach (SWR), sampling from the distribution is done “with replacement” meaning the y candidate tokens could all be the same token. In such a case, there would be only a single distinct candidate token which is picked regardless of its secret number. This token is essentially “unwatermarked”. The effect of this is shown in Table 4, in which we artificially manipulate the entropy of the distributions using temperature. As shown, the watermark is less detectable when the underlying distributions have less entropy. In Kirchenbauer et. al.’s  approach, we randomly promote some tokens, after this we sample from this distribution. If tokens from the “red list” capture a significant probability mass, they can potentially be selected. In Table 4, the watermark is less detectable when the underlying distributions are of lower entropy. However, SWOR variant of our approach in which the detectability is independent from the underlying distributions entropy is not affected by the entropy of the probability distribution. This is because we ensure y candidate tokens to be distinct by sampling without replacement, thus we always select y different candidates.

Reviewer comment (Experiments):
Why does the proposed approach work better than Kirchenbauer et al. (2023a)? Instead of changing the interference from a probability distribution to the sampling process, are there other reasons the proposed approach is better?

Author response:
We thank the reviewer for the insightful reviews. We understand that we should have put more emphasis on the distinctions between our approach and Kirchenbauer et al.’s approach. We first would like to note that all watermarking methods for LLMs would need to somehow affect probability distributions. Therefore, the main difference between watermarking methods for LLMs will be how these probability distributions are changed and the impact of these changes on the resultant texts. Now we would like to highlight these differences.

1) In Kirchenbauer  et. al.’s  approach, after the probability distribution for the next token is computed, a randomly selected subset of tokens (depending on the previous token(s)) are promoted.  In contrast, in our approach we sample from the distribution y (the sampling count) times. From these y candidate tokens we pick one that maximizes the secret number (essentially uniformly sampling from the candidate tokens). We would like to note that the candidate tokens are all tokens that could have been sampled by the sampler in the first place. 

2) The reviewer mentioned that our candidate tokens can be considered as “green list” and other tokens as “red list”, similar to Kirchenbauer  et. al.’s  approach. We kindly disagree with  this conclusion because there is a slight difference: We pick the token that maximizes the secret number from the y candidate tokens which are selected based on only LLM’s probability distribution (i.e., without any promotion of randomly selected tokens). We don’t pick tokens that are not in this set. We try to guarantee the “quality” of the y candidate tokens by directly sampling from the distribution.

3) Another distinction between Kirchenbauer 2023a’s approach and ours is the way we handle low entropy distributions. If the probability mass is concentrated in a single or few tokens, in our approach (SWR), sampling from the distribution is done “with replacement” meaning the y candidate tokens could all be the same token. In such a case, there would be only a single distinct candidate token which is picked regardless of its secret number. This token is essentially “unwatermarked”. The effect of this is shown in Table 4, in which we artificially manipulate the entropy of the distributions using temperature. As shown, the watermark is less detectable when the underlying distributions have less entropy. In Kirchenbauer et. al.’s  approach, we randomly promote some tokens, after this we sample from this distribution. If tokens from the “red list” capture a significant probability mass, they can potentially be selected. In Table 4, the watermark is less detectable when the underlying distributions are of lower entropy. However, SWOR variant of our approach in which the detectability is independent from the underlying distributions entropy is not affected by the entropy of the probability distribution. This is because we ensure y candidate tokens to be distinct by sampling without replacement, thus we always select y different candidates.