Paper: An Unforgeable Publicly Verifiable Watermark for Large Language Models (ICLR-2024)

Reviewer comment (Experiments):
While the authors have identified the security flaws in public watermark methods, the discussion lacks specific references or examples of existing attack methods. To further bolster the superiority of their proposed approach, the authors might need to include comparative assessments against how public watermark methods fare against the two types of attacks discussed in Section 4.5, "Analysis of Privacy."

Author response:
## About the privacy of public watermark methods 

Thank you for your question. 

Firstly, please refer to first response in "Responses to all reviewers". We appreciate the concerns raised by reviewers seaW and CjAu regarding the definition of 'private watermark'. Indeed, we acknowledge the potential confusion in using this term to describe our method. Consequently, we have renamed our approach from 'A private watermark' to 'An Unforgeable Publicly Verifiable Watermark for Large Language Models'. It means that our proposed watermarking method could be publicly verified by a third-party detector, and even under this scenario, the watermark cannot be easily forged by an untrustworthy detector.  In other words, the previous concept "privacy" is interpreted more clearly as "unforgeablity" under third-party detection. 

Moreover, previous work that used shared key in both watermarking and detection process are referred to as key-based method, while our detection approach is named as network-based method. All related concepts in the updated version of our paper have been revised accordingly.

Under this concept, the question "about the privacy of public watermark method" will be interpreted as "about the unforgeability of key-based methods". In a third-party detection scenario, the shared key used in key-based methods are exposed to the public.  Therefore, the key could be used to directly forge the watermark with no need of any attacks. That is, the key-based methods have no unforgeability.

You might get confused about why we leverage spoofing attack and reverse engineering attack to test unforgeability of our watermarking method. That is because, by using our network-based detecting method, there is no watermark generation details (such as the watermark key) exposed to public. Therefore, if an attacker wants to acquire the watermark generation details and then use it to forge watermark, they could only try using sophisticated analysis such as spoofing attack and reverse engineering attack mentioned in our paper.

We greatly appreciate your query and have clarified these issues in the updated version of our PDF.

Reviewer comment (Writing):
The last paragraph in Section 4.4, "Watermark Detection Network," regarding the construction of a cyclic document, is somewhat vague. A more detailed explanation, ideally supplemented with concrete examples, would enhance the clarity of this segment.

Author response:
# About the cyclic document

Consider a sentence $t$ with tokens $A B C D E F$. When the window size is 3, the labeling of each token is as follows: $C: \mathcal{W}(ABC) = \text{watermarked(green)}$, $D:  \mathcal{W}(BCD) = \text{not watermarked(red)}$, $E:  \mathcal{W}(CDE) = \text{watermarked(green)}$, $F:  \mathcal{W}(DEF) = \text{watermarked(green)}$. However, without using a cyclic document, tokens A and B remain unlabeled, potentially creating a system vulnerability. For instance, if a user continuously alters the last token (here, F) and observes the detection confidence $\mathcal{D}(t = ABCDE?)$ of the network, a clear score distinction between watermarked and unwatermarked tokens emerges. If the user tests four characters and finds $\mathcal{D}(t = ABCDEA) \approx \mathcal{D}(t = ABCDEB) > \mathcal{D}(t = ABCDEC) \approx \mathcal{D}(t = ABCDED)$, it's easy to infer that with DE as a prefix, A and B are watermarked tokens, while C and D are not.

To address this vulnerability, we set the entire document as a cyclic document. We label the beginning token A as $\mathcal{W}(EFA)$ and B as $\mathcal{W}(FAB)$, ensuring each token has a corresponding label and thereby avoiding the aforementioned vulnerability.

Thank you for your question. We have updated our PDF with the above content in its updated version (appendix F).

Reviewer comment (Experiments):
It would be informative if the authors could elucidate the reasoning behind choosing LSTM networks for the Detection Network, especially when more advanced models are available.

Author response:
# The selection of LSTM network

The primary reason for selecting LSTM is that the input to the watermark detector is a variable-length sequence, and LSTM is a relatively classic RNN structure capable of processing such sequences. Although models like Transformer could be considered, we did not pursue further selection or optimization in this direction because the performance of LSTM was already satisfactory for our purposes. This lack of deliberate choice and optimization in network architecture also suggests that there is still room for improvement in our method.

Reviewer comment (Novelty):
The contribution of the private watermarking scheme is not clearly justified. If the designer wants to protect the secret key, he can use a public key encryption scheme to design the watermarking scheme. Particularly, they can use a secret key in watermark generation, and a public key for watermark detection. It's not clear why the designer has to use two neural networks for watermark generation/detection.

Author response:
## The contribution of the private watermarking

Please refer to first response in "Responses to all reviewers".

We acknowledge the potential confusion in using "private watermark" to describe our method. Consequently, we have renamed our approach from 'A private watermark' to 'An Unforgeable Publicly Verifiable Watermark for Large Language Models.' All related concepts in the updated version of our paper have been revised accordingly. Next, we will thoroughly explain the contribution of our "unforgeable publicly verifiable watermark".

You have mentioned that a public key encryption algorithm could easily achieve the public verifiability. We assume the "public key encryption" refers to integrating methods like asymmetric encryption or digital signatures, which employ public-private keys into watermark algorithms. While these methods sound promising, its implementation in the context of large model watermarks is not straightforward. And most importantly, it is hard to design a public key encryption algorithm that can make the watermark unforgeable.  

For example, in a recent work, Fairoze et al. (published after the ICLR submission deadline) use digital signature technology to achieve public verifiability. However, in the watermark detection process, they still require extracting the watermark features first and then use the public key to verify them. Therefore, under a third-party public detection scenario, the extracted watermark features are exposed to public,  which can be directly used to forge watermark. 

Our method, however, has unique advantages. Users can only get from our detection network whether a text contains a watermark, without knowing the specific generation details of the watermark. Thus, while the method you propose is an admirable vision, integrating this idea into the watermarking domain is highly challenging, and currently, no feasible solutions exist. 

We believe our approach remains irreplaceable in the unforgeable publicly verifiable watermark domain, contributing uniquely to this field.

We welcome further discussion on this topic.

Reviewer comment (Reproducibility):
There is no clear discussion about how watermark generation and the watermark detection model are trained. There is no explanation about the loss function used either. Alg.1 assumes watermark generation network W as input while it's not clear how W is trained in the first place.

Author response:
## Training detail of watermark generation and detection network

Thank you very much for your detailed question. Indeed, we did not elaborate on all the training details of the watermark generation and detection networks in the original text. However, we have thoroughly described the network architecture, input, and output in the original text. Moreover, the hyperparameters related to network training have already been introduced in  section 4.
Overall, the training of both networks does not involve any particularly unique tricks. We will provide a detailed introduction here, and this information has already been added to the updated version of the PDF.

Regarding the watermark generation network, firstly, for a token within a window size, an embedding network is used to generate its representation. This embedding network is a fully connected network. Then, all tokens within that window size are concatenated and passed through another fully connected network, which performs a binary classification. This involves first using a sigmoid function followed by binary cross-entropy loss. The formula can be represented as follows:

$$ L_g = -\frac{1}{N} \sum_{i=1}^{N} \left[ y_i \cdot \log(\sigma(W(x_{n-w+1:n}))) + (1 - y_i) \cdot \log(1 - \sigma(W(x_{n-w+1:n}))) \right] $$

For the watermark detection network, the process is quite similar. The representation of each token to be detected is generated using an embedding network. Then, all these representations are fed into an LSTM network. The output of this network undergoes a binary classification. The formula for this is:

$$ L_d= -\frac{1}{N} \sum_{i=1}^{N} \left[ y_i \cdot \log(\sigma(D(x)_i)) + (1 - y_i) \cdot \log(1 - \sigma(D(x)_i)) \right] $$

Reviewer comment (Writing):
The definition of private watermarking is unclear. Some statements are confusing. For example, on p1, the authors write: "However, current watermarking algorithms are all public, which means the detection of watermarks requires the key from the watermark generation process." But then, they state that any watermarking method can be made private by limiting who has access to this detection key (also p1). They also reference Kirchenbauer et al. (2023), who suggest placing the watermark detection process behind a web API to achieve private watermarking, though at the cost of substantial server resources and robustness against hacking. This suggests a trivial method to make watermarking algorithms private, meaning not all watermarking methods are public. This leads to confusion about the distinction between private and public watermarks. While the general idea of separate keys for generating and verifying a watermark is understood, the reviewer requests elaboration on the authors' definition of a private watermark and why their method is not public, given Kirchenbauer's definition that a watermark is public if anyone can detect it in text.

Author response:
Please refer to first response in "Responses to all reviewers".

The objective of the private watermark is to detect of the watermark without revealing watermark generation details.  This is valuable because if the key is exposed, others can forge the watermark.  

Although Kirchenbauer et al. (2023) claim that the key can be concealed within an API for subsequent services, this approach still relies on the generation key for detection, disqualifying it as a true private watermark algorithm.  Furthermore, it's important to note that hiding watermark generation details at the algorithmic level does not contradict to additional system design (API).

A relatable analogy is with asymmetric encryption algorithms like RSA, where encryption is done with a private key, but decryption uses a public key. In contrast, symmetric encryption algorithms might employ a key for encryption and offer an API service using the same key for decryption, mimicking asymmetric decryption. However, this is not genuine asymmetric encoding, as decryption still requires the encryption key.

Hosting the decryption or watermark detection process on servers introduces additional complications, such as costly API fees, the need to secure servers against hacking, and trust issues with the service provider. For instance, consider a scenario where an entity accused of generating inappropriate texts is brought to court. It would be unreasonable to rely on the same entity to verify the texts through watermark detection.

We welcome further discussion on this topic.

## Unclear Definition of Our Watermarking

Please refer to new first response in "Responses to all reviewers".

Thank you for raising your question. We have indeed recognize that the concept of a "private watermark" can be somewhat confusing. Consequently, we have renamed our methodology from "A private watermark" to "An Unforgeable Publicly Verifiable Watermark for Large Language Models." We have also revised all related concepts in the updated version of our paper.

Current watermarking algorithms for large models share the same key during the generation and detection phases. This approach functions well when watermark detection is restricted to the watermark owner (private detection). However, if the detection process is made public, the watermark's key becomes exposed, leading to the risk of watermark forgery.

Our watermarking algorithm, however, employs a neural network for detection, allowing public verification without revealing the watermark's generation method. This feature renders our watermark unforgeable and publicly verifiable.

The implementation of an unforgeable publicly verifiable watermark presents significant challenges and, to date, there has been no other work achieved this goal. For example, Fairoze et al. (published after the ICLR submission deadline), use digital signature technology to avoid exposing the private key, they still require extracting the watermark features first and the use the public key to verify them. The extracted watermark features expose the watermark rules, which can be directly used to forge watermark. 

We greatly appreciate your valuable feedback and patient response. We welcome further discussion on this topic.

Reviewer comment (Writing):
The term "cracking the watermarking rules" is vague. It is unclear whether the authors are referring to the integrity of their watermarks.

Author response:
The watermarking rules essentially provides rules on how watermarking is generated.  For instance, in the algorithm of Kirchenbauer et al. (2023), watermarking is implemented by dividing the vocabulary into green (watermarked) and red (unwatermarked) lists, then increasing the probability of tokens from the green list at each generation step. The rule here refers to the partition of the green and red list. For example, in Kirchenbauer et al. (2023), for a window size of 2, determining which tokens follow the prefix token 'there' and belong to the green list constitutes a rule.

'Cracking the watermarking rules' fundamentally means inferring these specific rules directly, rather than obtaining the watermark's key.  For example, in Sadasivan et al. (2023), a spoofing attack could deduce a specific watermark rule by analyzing word frequency within watermarked texts. If a watermarking algorithm's rules can be easily deduced by such methods, it is essentially not a private algorithm. 

A good private watermark algorithm could not hide the generation key but also robust to the attacks that try to crack the watermarking rules.

Reviewer comment (Writing):
The meaning of "implement watermarking in a privacy-preserving manner" is unclear.

Author response:
Thank you for your question. "Privacy-preserving manner" refers to the ability to detect watermarks without needing the key used in the watermark generation process. Indeed, we acknowledge that the original statement was somewhat confusing. We have clarified this point in the revised PDF provided.

Reviewer comment (Writing):
The meaning of "definite labels" (p9) is unclear.

Author response:
In our context, 'definite labels' refer to labels with high confidence levels, where the label here refers to whether a text contains a watermark. The corresponding z-scores for data with 'definite labels' significantly deviate from the threshold value used in the public z-score based detection.

Reviewer comment (Experiments):
The authors focus on the effectiveness and privacy of their watermark but do not study its robustness. An attacker who can verify a watermark in any text sequence should be much more capable of removing the watermark. The reviewer asks why robustness was not evaluated and questions the utility of a watermark that is not robust. Even if the watermark is not robust, the reviewer requests experiments showing its limitations.

Author response:
Please refer to third response in "Responses to all reviewers" and the table5 in the updated PDF version.

## About Robustness

Thank you for your valuable suggestion. We have mention more robustness issue in the limitation part of conclusion section.

Reviewer comment (Writing):
The meaning of "FFN" in EQ 2 is unclear, and clarification is requested.

Author response:
The term "FFN" here refers to a fully connected classification network. We apologize for any confusion caused and have provided a revised explanation in the updated version of the PDF.

Reviewer comment (Reproducibility):
The number of parameters in the LSTM is not provided, and this information is requested.

Author response:
Our LSTM only comprises five layers, with an input dimension of 64 for each unit, an intermediate output dimension of 128, and a final output dimension of 1 for the output logits. 

Therefore, the parameters of LSTM are negligible compared to those of large language models.

Reviewer comment (Writing):
In EQ 3, it is unclear if "P" is supposed to be the probability, since it was earlier defined as the logit score, and clarification is requested.

Author response:
P indeed represents probability, and P_n was previously used to denote a special probability symbol. However, considering potential confusion, we have now adopted L_n to represent logits. This change has been clarified in the updated version of our PDF.

Reviewer comment (Writing):
In EQ 8, the definition of "$f$" is missing and clarification is requested.

Author response:
In this context, 'f' represents an ideal generation network, corresponding to the 'g' function mentioned in EQ9, where 'g' refers to an ideal detection network. Our aim here is to demonstrate that training 'g' using 'f' is significantly simpler than the reverse process of training 'f' using 'g', which forms the basis of our algorithm's privacy. Thank you for your question; we have addressed this point in the revised version of our PDF.

Reviewer comment (Writing):
Clarification is requested regarding the motivation and definitions of "privacy." Specifically, why existing watermarking methods are not considered private.

Author response:
Please refer to first response in "Responses to all reviewers".

The objective of the private watermark is to detect of the watermark without revealing watermark generation details.  This is valuable because if the key is exposed, others can forge the watermark.  

Although Kirchenbauer et al. (2023) claim that the key can be concealed within an API for subsequent services, this approach still relies on the generation key for detection, disqualifying it as a true private watermark algorithm.  Furthermore, it's important to note that hiding watermark generation details at the algorithmic level does not contradict to additional system design (API).

A relatable analogy is with asymmetric encryption algorithms like RSA, where encryption is done with a private key, but decryption uses a public key. In contrast, symmetric encryption algorithms might employ a key for encryption and offer an API service using the same key for decryption, mimicking asymmetric decryption. However, this is not genuine asymmetric encoding, as decryption still requires the encryption key.

Hosting the decryption or watermark detection process on servers introduces additional complications, such as costly API fees, the need to secure servers against hacking, and trust issues with the service provider. For instance, consider a scenario where an entity accused of generating inappropriate texts is brought to court. It would be unreasonable to rely on the same entity to verify the texts through watermark detection.

We welcome further discussion on this topic.

## Unclear Definition of Our Watermarking

Please refer to new first response in "Responses to all reviewers".

Thank you for raising your question. We have indeed recognize that the concept of a "private watermark" can be somewhat confusing. Consequently, we have renamed our methodology from "A private watermark" to "An Unforgeable Publicly Verifiable Watermark for Large Language Models." We have also revised all related concepts in the updated version of our paper.

Current watermarking algorithms for large models share the same key during the generation and detection phases. This approach functions well when watermark detection is restricted to the watermark owner (private detection). However, if the detection process is made public, the watermark's key becomes exposed, leading to the risk of watermark forgery.

Our watermarking algorithm, however, employs a neural network for detection, allowing public verification without revealing the watermark's generation method. This feature renders our watermark unforgeable and publicly verifiable.

The implementation of an unforgeable publicly verifiable watermark presents significant challenges and, to date, there has been no other work achieved this goal. For example, Fairoze et al. (published after the ICLR submission deadline), use digital signature technology to avoid exposing the private key, they still require extracting the watermark features first and the use the public key to verify them. The extracted watermark features expose the watermark rules, which can be directly used to forge watermark. 

We greatly appreciate your valuable feedback and patient response. We welcome further discussion on this topic.

Reviewer comment (Evaluation):
Clarification is requested on how robust the authors’ watermark is.

Author response:
Please refer to third response in "Responses to all reviewers" and the table5 in the updated PDF version.

## About Robustness

Thank you for your valuable suggestion. We have mention more robustness issue in the limitation part of conclusion section.

Reviewer comment (Experiments):
Please correct me if I am wrong. It seems that there is no comparison with baseline methods in terms of both watermarking effectiveness and text generation quality.

Author response:
## About Baseline Comparison

Related responses: please refer to the second and forth responses in "Responses to all reviewers".

The key-based detection method in Table1 is the same to the KGW watermark algorithm by Kirchenbauer et al. (2023).

This implies that Table 1 compares the watermark detection accuracy of Kirchenbauer et al. (2023) (denoted as 'key-based') with our unforgeable watermark algorithm.  And due to the lack of alternative unforgeable watermark algorithms, we did not conduct direct comparisons with other unforgeable watermark methods.

Moreover, since our watermark generation method is almost identical to that of Kirchenbauer et al. (2023), the impact on text quality should be exact the same to Kirchenbauer et al. (2023).

Despite this, we have demonstrated in Figure 2(b) the variation of detection F1 score and PPL with changes in the $\delta$ parameter. In our method, the $\delta$ hyperparameter is set to 2, indicating that our approach achieves high detection accuracy without significantly affecting text quality.

We have also conducted analysis of text quality changes in machine translation task, affirming that our watermarking method does not significantly impact text quality. More details in the forth response in "Responses to all reviewers".

The updated PDF version now includes additional clarifications and descriptions.

Reviewer comment (Experiments):
It is not clear whether text edit methods like paraphrase can make the proposed method invalid.

Author response:
## About robustness to paraphrase attack

Please refer to third reply in "to all reviewers" and the table4 in the updated PDF version.

Reviewer comment (Experiments):
Are your methods robust to text edit methods like paraphrase?

Author response:
## About robustness to paraphrase attack

Please refer to third reply in "to all reviewers" and the table4 in the updated PDF version.