Venue: NAACL 2024

# SemStamp: A Semantic Watermark with Paraphrastic Robustness

for Text Generation

 Abe Bohan Hou\({}^{\clubsuit}\)  Jingyu Zhang\({}^{\clubsuit}\) Tianxing He\({}^{\heartsuit}\)

Yichen Wang\({}^{\diamondsuit}\) Yung-Sung Chuang\({}^{\spadesuit}\) Hongwei Wang\({}^{\ddagger}\) Lingfeng Shen\({}^{\spadesuit}\) Benjamin Van Durme\({}^{\clubsuit}\) Daniel Khashabi\({}^{\clubsuit}\) Yulia Tsvetkov\({}^{\heartsuit}\) Johns Hopkins University University University of Washington \({}^{\Diamond}\)Xi'an Jiaotong University

\({}^{\spadesuit}\)Massachusetts Institute of Technology Tencent AI Lab

{bhou4, jzhan237}@jhu.edu goosehe@cs.washington.edu

###### Abstract

Existing watermarked generation algorithms employ _token-level_ designs and therefore, are vulnerable to paraphrase attacks. To address this issue, we introduce watermarking on the _semantic representation_ of sentences. We propose SemStatamp, a robust sentence-level semantic watermarking algorithm that uses locality-sensitive hashing (LSH) to partition the semantic space of sentences. The algorithm encodes and LSH-hashes a candidate sentence generated by a language model, and conducts rejection sampling until the sampled sentence falls in watermarked partitions in the semantic embedding space. To test the paraphrastic robustness of watermarking algorithms, we propose a "bigram paraphrase" attack that produces paraphrases with small bigram overlap with the original sentence. This attack is shown to be effective against existing token-level watermark algorithms, while posing only minor degradations to SemStatamp. Experimental results show that our novel semantic watermark algorithm is not only more robust than the previous state-of-the-art method on various paraphrasers and domains, but also better at preserving the quality of generation.1

Footnote 1: Equal Contribution. All three are corresponding authors.

+
Footnote †: We release code in [https://github.com/bohanhou14/SemStatmp](https://github.com/bohanhou14/SemStatmp)

## 1 Introduction

This work focuses on algorithms for detecting machine-generated text via _watermarked generation_--adding signatures during text generation which are algorithmically detectable, yet are imperceptible to human eye Atallah et al. (2001). This problem is of extreme importance now that large language models (LLMs) such as GPT-4 OpenAI, 2023] generate realistic text, increasing risks of LLM misuse, such as generation of misinformation, impersonation, and copyright infringements Weidinger et al. (2021); Ippolito et al. (2022); Pagnoni et al. (2022); House (2023).

The dominant body of recent works on watermarked generation operate by injecting token-level signatures during decoding time Kuditipudi et al. (2023); Yoo et al. (2023); Wang et al. (2023); Christ et al. (2023); Fu et al. (2023), _i.a._). As a representative example, Kirchenbauer et al. (2023) propose a watermarked generation algorithm that invjects watermark signals that are extracted based on the previously generated _tokens_. Despite its efficiency, follow-up work has shown that corrupting the generated text, especially paraphrasing, could weaken its robustness Krishna et al. (2023); Sadasivan et al. (2023); Kirchenbauer et al. (2023).

We propose SemStatamp, a _semantic watermark algorithm_ that is robust to sentence-level paraphrase attacks (SS2.2). Depicted in Figure 1, our core intuition is that while paraphrasing alters the surface-form tokens, the sentence-level semantics are unchanged. Thus, instead of partitioning the vocabulary, our watermark operates on the semantic space of sentence embeddings, partitioned by locality-sensitive hashing LSH; Indyk and Motwani (1998); Charikar (2002). We develop two key components--a sentence encoder trained with contrastive learning CL; Wieting et al. (2022) and a margin-based constraint--to enhance paraphrastic robustness.

To stress-test the robustness of watermarking algorithms, we develop a novel attack method that minimizes bigram overlap during paraphrasing, and name it the bigram paraphrase attack (SS2.3). Experimental results (SS3) demonstrate that our proposed semantic watermark remains effective while token-level watermarks suffer significantly from the bigram attack.

We summarize our main contributions as follows. First, we propose a sentence-level semantic watermark for LLMs and show that it is robust to paraphrasing and more quality-preserving than a token

level watermark algorithm. Second, we develop a novel attack method for watermarking algorithms, namely the bigram paraphrase attack, which can effectively weaken token-level watermarking but only poses minor degradations to our semantic watermark. Third, we fine-tune a paraphrase-robust sentence encoder with a contrastive learning objective and develop a rejection margin constraint to enhance the paraphrastic robustness of our semantic watermark algorithm.2

Footnote 1: [https://github.com/facebook/sentiment-language-transformers](https://github.com/facebook/sentiment-language-transformers)

## 2 Approach

### Preliminaries

Text Generation from Autoregressive LMsAn autoregressive LM, denoted by \(P_{\text{LM}}\), models the conditional distribution of the next token over the vocabulary \(V\). Given a token history \(w_{1:t}=w_{1},\dots,w_{t}\) where each token \(w_{i}\in V\), the next token is generated by sampling \(w_{t+1}\sim P_{\text{LM}}(\cdot|w_{1:t})\). We introduce a sentence-level notation: \(s^{(t+1)}\sim P_{\text{LM}}(\cdot|s^{(1)}\ldots s^{(t)})\) refers to the sampling of the next sentence given sentence history \(s^{(1)}\ldots s^{(t)}\).

Detecting Machine-Generated Text through WatermarkingThe goal of watermarked generation Kuditipudi et al. (2023); Zhao et al. (2023), _i.a._) is to facilitate the detection of machine-generated text. A watermarked generation algorithm adds a statistical signal during the decoding stage of LLMs. The watermarked text is then provided to the user. At the detection stage, a piece of text is classified as machine-generated if the watermark is detected. Because malicious users could postprocess LLM-generated texts before detection, it is crucial that the watermark remains detectable under various text perturbations attacks, including text insertion, substitution, deletion, and paraphrasing.

Token-Level Watermarking and its Susceptibility to Paraphrase AttacksKirchenbauer et al. (2023) propose a watermark that is injected at the token level. At each time step of the generation, the vocabulary \(V\) is pseudorandomly partitioned into a "green list" and a "red list". The random seed for partition is computed by a hash of the previously generated token. A globally fixed bias parameter \(\delta>0\) is added to the logit of each green list token so that the LLM is induced to generate more green list tokens. The watermark is detected by conducting one proportion \(z\)-test (detailed in SSB) on the number of green list tokens in the generated text.

Because of the token-level nature of the watermark algorithm, perturbing a token \(w_{t}\) in a generated sequence \(w_{1:T}\) through paraphrasing would change the green list for token \(w_{t+1}\). As a result, a green token \(w_{t+1}\) might be considered red, which undermines the detectability of the watermark Krishna et al. (2023). Moreover, because the watermark changes logits directly, it can degrade the quality of generated text Fu et al. (2023).

Locality-Sensitive HashingWe will use LSH Indyk and Motwani (1998) to partition the semantic embedding space. It hashes similar inputs into similar signatures, thereby reducing the dimensionality and providing a similarity measure for a high-dimensional input space \(\mathbb{R}^{h}\). Given an LSH dimension \(d\), we adopt the cosine-preserving method from Charikar (2002) which produces a \(d\)-bit binary signature through random hyperplane projections, and each hyperplane is represented by a random normal vector \(n^{(i)}\) drawn from the \(h\)-dimensional Gaussian distribution.3 The LSH signature for an embedding vector \(v\in\mathbb{R}^{h}\) is then determined by the sign of the dot product be

Figure 1: An overview of the proposed SemStamp algorithm. **Left**: During generation, the watermark is injected by mapping candidate sentences into embeddings through a robust sentence encoder, dividing the semantic space through locality-sensitive hashing, and rejection sampling from the LM to generate sentences with valid region embeddings. **Right**: Detection is determined by the number of valid sentences in a candidate generation.

 

tween the candidate vector and the normal vectors: \({\texttt{Lsh}_{i}}:\mathbb{R}^{h}\mapsto\left\{0,1\right\}\) which gives the \(i\)-th digit signature, is defined by \({\texttt{Lsh}_{i}}(v)=\mathbbm{1}\big{(}n^{(i)}\cdot v>0\big{)}\lx@note{ footnote}{$1$}(\cdot)\) is the indicator function.

```
Input: language model \(P_{\text{LM}}\), prompt s\({}^{(0)}\), number of sentences to generate \(T\). Params: sentence embedding model \(M_{\text{embd}}\) with embedding dimension \(h\), maxout number \(N_{\text{max}}\), margin \(m>0\), valid region ratio \(\gamma\in(0,1)\), LSH dimension \(d\), a large prime number \(p\). Output: generated sequence \(s^{(1)}\ldots s^{(T)}\). procedure SemStamp initlsh\((\cdot)\), randomly initialize \(d\) vectors \(n^{(1)}\ldots n^{(d)}\in\mathbb{R}^{h}\), to create \(2^{d}\) semantic subspaces. for\(t=1,2,\dots,T\)do  1. Compute the LSH signature of the previously generated sentence, Sig\((s^{(t-1)})\), and use \([\textsc{Sig}(s^{(t-1)})]_{10}\cdot p\) as the seed to randomly divide the space of signatures \(\{0,1\}^{d}\) into a "valid region set" \(G^{(t)}\) of size \(\gamma\cdot 2^{d}\) and a "blocked region set" \(R^{(t)}\) of size \((1-\gamma)\cdot 2^{d}\).
2. repeat Sample a new sentence from LM, until the signature of the new sentence is in the "valid region set", Sig\((s^{(t)})\in G^{(t)}\) and the margin requirement Margis\((s^{(t)},m)\) is satisfied. or has repeated \(N_{\text{max}}\) times
3. Append the selected sentence \(s^{(t)}\) to context. endfor return\(s^{(1)}\ldots s^{(T)}\) endprocedure
```

**Algorithm 2**SemStAMP subroutines

At each generation step, given a sentence history \(s^{(0)}\ldots s^{(t-1)}\), we first produce the LSH signature of the previously generated sentence Sig\((s^{(t-1)})\), where Sig\((\cdot)\) encodes and LSH-hashes the sentence, as defined in Alg. 2. Next, we pseudorandomly divide the LSH partitions into a set of "valid" regions \(G^{(t)}\) and a set of "blocked" regions \(R^{(t)}\), where the masking is seeded by Sig\((s^{(t-1)})\).5 To produce the watermarked next sentence, we sample with rejection a new sentence \(s^{(t)}\) from the LM until its embedding lies in the "valid" region in the semantic space.6

Footnote 4: \(\mathbbm{1}(\cdot)\) is the indicator function.

To detect the SemStamp watermark, we conduct a one-proportion \(z\)-test on the number of validation sentences in the generated text. Since this detection is similar to Kirchenbauer et al. (2023a), we defer the details to SSB.

Because a proper paraphrase should retain the meaning of the original sentence, we hypothesize that the LSH signature is likely to remain the same after paraphrasing (Figure 4 provides empirical re 

sults). Therefore, the valid region partition for the next sentence would not change, ensuring the watermark is still detectable after the paraphrase attack. Below we explain each core component of SemStamp in detail.

Footnote 6: [https://github.com/facebook/semStatmp-2018/](https://github.com/facebook/semStatmp-2018/)

Paraphrase-Robust Sentence EncoderA requirement for SemStamp is a semantic encoder to map sentences into semantic embeddings. Our encoder is built upon Sentence-BERT (SBERT; Reimers and Gurevych, 2019), a fine-tuned siamese network trained to produce sentence embeddings whose cosine similarity mirror the semantic similarity of the STS benchmark Cer et al. (2017).

To enhance the encoder's robustness to paraphrase, we further fine-tune the SBERT model using contrastive learning Wieting et al. (2022). For each sentence \(s_{i}\) in a corpus, we first produce its paraphrase \(t_{i}\) using an off-the-shelf paraphrasing model, Pegasus Zhang et al. (2020).7 Next, we sample a random sentence \(t^{\prime}_{i}\) from the corpus that is not a paraphrase of \(s_{i}\) to serve as the negative example. The objective promotes the original sentence to be more similar to the paraphrase than the negative example by a margin of \(\delta>0\):

Footnote 7: Link to Pegasus paraphraser.

\[\min_{\theta}\sum_{i}\max\Bigl{\{}\delta-f_{\theta}(s_{i},t_{i})+ f_{\theta}(s_{i},t^{\prime}_{i}),0\Bigr{\}}, \tag{1}\]

where \(f_{\theta}\) is the cosine similarity between the embedded sentences, \(f_{\theta}(s,t)=\cos\bigl{(}M_{\theta}(s),M_{\theta}(t)\bigr{)}\), and \(M_{\theta}\) is the encoder model with parameter \(\theta\).

Semantic Space Partitioning through LSHDuring the initialization of watermarked generation, normal vectors \(n^{(1)}\dots n^{(d)}\) are randomly drawn from the \(h\)-dimensional Gaussian distribution to represent \(d\) LSH hyperplanes in the semantic space \(\mathbb{R}^{h}\). The hyperplanes are fixed during generation and detection to serve as the basis for partitioning. As introduced in SS2.1, this induces a \(d\)-bit binary signature \(\textsc{Lsh}(v)\) for a vector \(v\in\mathbb{R}^{h}\). Consequently, we use each of the \(2^{d}\) signatures \(c\in\{0,1\}^{d}\) to represent a region in the semantic space consisting of points with signature \(c\).

During the generation of a new sentence \(s^{(t)}\), we apply a watermarking "mask" on the semantic space by pseudorandomly partitioning the space of signatures \(\{0,1\}^{d}\) into a valid region set \(G^{(t)}\) of size \(\gamma\cdot 2^{d}\) and a blocked region set \(R^{(t)}\) of size \((1-\gamma)\cdot 2^{d}\), where \(\gamma\in(0,1)\) determines the ratio of valid regions. The masking is seeded by the LSH signature of the last sentence \(s^{(t-1)}\) and thus varies for each time-step \(t\). Specifically, we convert the binary signature \(\textsc{Sig}(s^{(t-1)})\) to decimal and use \([\textsc{SIG}(s^{(t-1)})]_{10}\times p\) to seed the randomization. Here \(p\) is a large prime number and \([.]_{10}\) an operator that casts binary numbers to decimal numbers. The condition for rejection sampling is that the LSH signature of the new sentence must fall into one of the valid regions, i.e., \(\textsc{Lsh}(M_{\text{embd}}(s^{(t)})\in G^{(t)}\).

Margin-Based Constraint for RobustnessFor the SemStamp algorithm to be robust, the LSH signature of the sentences should remain the same under paraphrase attack. Empirically, we found the robustness from contrastive learning (Eq. 1) is not strong enough to preserve consistent LSH signature under paraphrasing. Therefore, we add an additional rejection sampling requirement that the sampled sentence \(s^{(t)}\) must have the absolute value of cosine similarity with each normal vector \(n^{(i)}\) larger than a margin \(m>0\):

\[\min_{i=1,...,d}|\cos(n^{(i)},v_{t})|>m, \tag{2}\]

where \(v_{t}=M_{\text{embd}}(s^{(t)})\) is the embedding of the candidate next sentence.8

Footnote 8: We discuss additional details on the condition for consistent LSH signature in §E.

Visually, this is akin to rejecting sentences whose embeddings lie near the boundaries of an LSH hyperplane. We illustrate this in Figure 2. In our experiments (SS3), we show that this margin-based rejection requirement can effectively increase the LSH signature robustness under paraphrasing.

### The Bigram Paraphrase Attack

We develop a strong "bigram" paraphrase attack with the following intuition. Because existing

Figure 2: An illustration for margin-based rejection. Sentence embeddings at LSH hyperplane boundaries are rejected (highlighted in red).

 

token-level watermark algorithms hash the last generated token to determine the watermarking signature (Kirchenbauer et al., 2023), any choice of token at position would affect the watermark of position . Therefore, we hypothesize that token-level watermarks might be especially sensitive to bigram (two adjacent tokens) perturbation.

Motivated by this intuition, we propose and explore the bigram paraphrase attack, a simple yet effective variant of the basic sentence-level paraphrase attack. Specifically, given a neural paraphrase model, we first decode a large number of top-raking sequences with beam search, obtaining paraphrase candidates. Next, we select the candidate that has the smallest bigram overlap with the original sentence. Moreover, to preserve the paraphrasing quality, we constrain the paraphrase attack with BERTScore (Zhang et al., 2019) between paraphrases and original sentences:

where denotes the original sentence, is a simple counting of overlapped bigrams between sequences and , denotes the BERTScore between sequence and , and is the BERTScore threshold ratio. See Figure 5 for an example in action.

## Experiments

### Experimental Setup

DatasetsWe conduct experiments to validate the detection robustness and quality of SemStamp on the RealNews subset of the C4 dataset (Raffel et al., 2020), BookSum (Kryscinski et al., 2021), and Reddit-TIFU (Kim et al., 2019), a dataset with informal text style. We further analyze the detection results and generation quality on 1000 random samples.

MetricsWe use binary classification metrics: (1) area under the receiver operating characteristic curve (_AUC_), and (2) the true positive rate when the false positive rate is % or % (_TP@1%_, _TP@5%_), i.e., the percentage of machine-generated text (the "positive" class in the classification setting) that is correctly detected when 1% and 5% of human texts (the "negative" class) are misclassified as machine-generated texts. A piece of text is classified as machine-generated when its score exceeds a threshold chosen based on a given false positive rate, which we explain in detail in SSB. Differing from KGW algorithm (Kirchenbauer et al., 2023), our algorithm treat sentences as the unit during -score computation.

To evaluate generation quality, we measure the perplexity (_PPL_) with OPT-2.7B (Zhang et al., 2022). Generation diversity is measured with trigram text entropy (Zhang et al., 2018) (_Ent-3_), i.e., the entropy of the trigram frequency distribution of the generated text. We also evaluate generations with _Sem-Ent_(Han et al., 2022), an automatic metric for semantic diversity. Following the setup in Han et al. (2022), we use the last hidden states of OPT-2.7B models on generations as their semantic representation and perform -means clustering. Sem-Ent is the entropy of semantic cluster assignments of test generations. We evaluate the quality of paraphrases using BERTScore (Zhang et al., 2019) between original generations and their paraphrases.

Training, Generation, and BaselinesFor contrastive learning of SBERT, we paraphrase 8k paragraphs of the RealNews dataset (Raffel et al., 2020) using the Pegasus paraphraser (Zhang et al., 2020) through beam search with 25 beams. We then fine-tune an SBERT model9 with an embedding dimension on this subset for 3 epochs with a learning rate of , using contrastive learning objective (Eq. 1). We set the contrastive learning margin which is tuned from the dev set.

Footnote 9:sentence-transformers/all-mpnet-base-vl

For watermarked generation, we use a fine-tuned version of OPT-1.3B (Zhang et al., 2022) as our base model to produce text with shorter length per sentence and conduct sampling at a temperature of 0.7 following Kirchenbauer et al. (2023) with a repetition penalty of 1.05. Setting as the prompt length, we let 200 be our default generation length but also experiment on various different lengths (Fig. 3). To generate from SemStamp, we sample at a LSH dimension with valid region ratio with rejection margin . See SS3.2 for the impact on hyperparameter choices.

We choose the popular watermarking algorithm Kirchenbauer et al. (KGW; 2023) as our main baseline. In the paraphrase attack phase, we paraphrase generations by SemStamp and KGW and compare their post-hoc detection rates after attacks. We also experiment with a distortion-free watermark by Kuditipudi et al. (KTH; 2023) and Unigram-Watermark (Zhao et al., 20 

but preliminary results show that KTH performs poorly compared to both KGW and SemStamp against our paraphrase attacks for the AUC metric. Unigram-Watermark also demonstrates strong robustness against paraphrase, but it is vulnerable to being reverse-engineered (also see SSD).

**Paraphrase Attack** For paraphrase attack experiments, watermarked generations are paraphrased sentence-by-sentence with the Pegasus paraphraser Zhang et al. (2020), the Parrot paraphrase used in Sadasivan et al. (2023), and GPT-3.5-Turbo OpnAI, 2022). We use beam search with 25 beams for both Pegasus and Parrot. For GPT-3.5-Turbo, we provide the sentences before the current sentence as the context and prompt the model to paraphrase via the OpenAI API.10 A detailed description of prompts is included in SSE.

Footnote 10: [https://platform.openai.com/playground/](https://platform.openai.com/playground/)

To implement the bigram paraphrase attack, we prompt the GPT-3.5-Turbo to return 10 paraphrases of the same sentence. For the Pegasus and Parrot paraphrsers, we select the candidate sentence with the least bigram overlap among the 25 beams from beam-search, subject to a BERTScore constraint of dropping no more than 10% of the score from the first beam. For GPT-3.5-Turbo, the paraphrase sample with the highest BERTScore is treated as the first beam.

### Results

DetectionTable 1 shows detection results under different paraphrases and the bigram attack at generation length 200. SemStamp **is more robust to paraphrase attacks than KGW across the Pega

\begin{table}
\begin{tabular}{l c|c c c} \hline \hline  & & RealNews & BookSum & Reddit-TFU \\ \cline{4-6} _Paraphrase_ & _Algorithm_ & _AUC_\(\uparrow\) & _TP@I\%_\(\uparrow\) & _TP@5\%_\(\uparrow\) \\ \hline No Paraphrase & KGW & 99.6 1 99.9 1 99.3 & 98.4 1 99.4 1 97.5 & 98.9 1 99.5 1 98.1 \\  & SSTamp & 99.2 1 99.7 1 99.7 & 93.9 1 98.8 1 97.7 & 97.1 1 99.1 1 98.2 \\ \hline \hline Pegasus & KGW & 95.9 1 97.3 1 94.1 & 82.1 1 89.7 1 87.2 & 91.0 1 95.3 1 87.2 \\  & SSTamp & **97.8 1 99.2 1 98.4** & **83.7 1 90.1 1 92.8** & **92.0 1 96.8 1 95.4** \\ Pegasus-bigram & KGW & 92.1 1 96.5 1 91.7 & 42.7 1 56.6 1 67.2 & 72.9 1 85.3 1 67.6 \\  & SSTamp & **96.5 1 98.9 1 98.0** & **76.7 1 86.8 1 89.0** & **86.0 1 94.6 1 92.9** \\ \hline Parrot & KGW & 88.5 1 94.6 1 79.5 & 31.5 42.0 1 22.8 & 55.4 1 75.8 1 43.3 \\  & SSTamp & **93.3 1 97.5 1 90.2** & **56.2 1 70.3 1 56.2** & **75.5 1 88.5** 1 **70.5** \\ Parrot-bigram & KGW & 83.0 1 93.1 1 82.8 & 15.0 1 39.9 1 27.6 & 37.4 1 71.2 1 49.7 \\  & SSTamp & **93.1 97.5 1 93.9** & **54.4 1 71.4 1 71.8** & **74.0 1 89.4 1 82.3** \\ \hline GPT3.5 & KGW & 82.8 1 87.6 1 84.1 & 17.4 1 71.2 2 27.3 & 46.7 1 52.1 1 50.9 \\  & SSTamp & **83.3 1 91.8 18 87.7 & **33.9 1 55.0 1 47.5** & **52.9 1 70.8 1 58.2** \\ GPT3.5-bigram & KGW & 75.1 1 77.1 1 79.8 & 5.9 1 4.4 1 19.3 & 26.3 1 27.1 1 41.3 \\  & SSTamp & **82.2 1 90.5 1 87.4** & **31.3 1 47.4 1 43.8** & **48.7 1 63.6 1 55.9** \\ \hline \hline \end{tabular}
\end{table}
Table 1: Detection results under different paraphraser settings. All numbers are in percentages. \(\uparrow\) indicates higher values are preferred. The numbers in parenthesis show the changes over our baseline. SemSTAMP **is more robust than KGW on multiple paraphrasers, datasets, and both the regular and bigram paraphrase attacks.**

Figure 3: Detection results (AUC) under different generation lengths. SemStamp **is more robust than KGW across length 100-400 tokens.**



sus, Parrot, and GPT-3.5-Turbo paraphrasers, as measured by AUC, TP@1%, and TP@5%. Although we only fine-tune the SBERT model on data from the Pegasus paraphraseer, SemStamp algorithm generalizes its robustness to different paraphrasers (Parrot, GPT-3.5-Turbo) and works on texts from different domains.

**The bigram paraphrase attack effectively weakens the token-level KGW algorithm while SemStamp is relatively unaffected.** Pegasus bigram attack can lower KGW's AUC by 7.9% and TP@5% by 27.1% on RealNews, whereas SemStamp only decreases by 3.5% and 13.2%. Furthermore, the BERTScore for bigram paraphrase does not change drastically compared to the regular paraphrases (Table 4 in SSD), showing that the bigram paraphrase attack still preserves paraphrase quality due to the BERTScore constraints we add. Kirchenbauer et al. (2023b) propose several alternative hashing schemes to the KGW algorithm. We conduct paraphrase attack experiments on a recommended scheme named SelfHash, and do not find visible improvements to KGW, thus omitting the results for brevity.

QualityTable 2 compares quality metrics of non-watermarked generations with KGW and SemStamp generations. **While KGW notably degrades perplexity due to the token-level noise added to logits, the perplexity of SemStamp generation is on par with the base model without watermarking.** This confirms our hypothesis that the sentence-level nature of SemStamp is less disruptive of token selections and preserves the generation quality. Figure 5 and 6 provide qualitative examples of SemStamp generations and the bigram paraphrase attack. Compared to non-watermarked generation, the sentences are equally coherent and contextually sensible. **SemSTamp also preserves token and semantic diversity of generation compared to non-watermarked generation and KGW generation**, as measured by the Ent-3 and Sem-Ent metrics, respectively.

Generation LengthFigure 3 highlights that SemStamp is robust to both regular and bigram paraphrase attacks across different generation lengths as measured by the number of tokens. SemStamp has consistently higher AUC than KGW (Kirchenbauer et al., 2023a).

AnalysisFigure 4 shows that increasing margin size \(m\) will increase the consistency of LSH signatures (_LSH consistency_), the ratio of sentences that remain in the same valid region after being paraphrased. A higher rejection margin will ensure the sampled generations are further away from the region boundary, thus less likely to shift to a different region after paraphrasing. However, a larger margin will result in a slower generation speed, and we find \(m=0.02\) works well empirically.

We also compare the LSH consistency before and after fine-tuning SBERT with contrastive learning in Figure 4. Fine-tuning the encoder on Pegasus-paraphrasead data improves the LSH consistency across different margins.

Applying the masking of semantic space partitions and the rejection margin, SemStamp makes a trade-off between watermark detection accuracy and generation speed. For our current hyperparameter setting, 13.8 sentences are needed on average to sample one valid sentence. As we explain in the Limitations and Discussion section, this limitation can be mitigated if we conduct batched sampling of next sentences.

## 4 Related Work

Machine-generated text detection, aiming at distinguishing LLM-generated texts from human-written ones, can be categorized into proactive and post-hoc methods. Our focus, watermarked generation, belongs to the first category.

Watermarked GenerationEarly approaches to watermarking include text-meaning representation tree for information hiding Atallah et al. (2002), and a watermarking scheme for machine translation using an output selector that considers hashing operation Venugopal et al. (2011). Watermarked language generation, specifically pertinent to LLMs, is a renewed trend of proactive machine

Figure 4: Rejection margin and contrastive fine-tuning effectively improve LSH Consistency.

 

generated text detection. The scheme works by adding signatures imperceptible to humans during decoding time to enable stable detection at a later time. Kirchenbauer et al. (2023) propose a watermarking algorithm by adding token-level bias (reviewed in SS2). Kuditipudi et al. (2023) proposes a distortion-free watermark that preserves the original distribution of LM during watermarking. Yoo et al. (2023) embeds multi-bit information into watermark and enhances performance against corruption through a robust infilling model. They inject the watermark via word replacement after initial generation, which is incorporated into one-stage watermarked generation by Wang et al. (2023). Christ et al. (2023) propose a watermarking scheme that is computationally undetectable without the secret key in theory.

Importantly, these existing works employ a token-level design and focus on span-level corruption such as editing and cropping, which renders the watermarks susceptible to paraphrase attacks.

More related to our focus on paraphrase attack, Krishna et al. (2023) propose a retrieval-based method that requires saving all previously-generated sequences, and Kirchenbauer et al. (2023) empirically shows that Kirchenbauer et al. (2023) is more robust under longer generation length. Contemporary to our work, Zhao et al. (2023) improves robustness via a cryptographic-free watermark without hashing previous tokens, which is more robust to editing and paraphrasing attacks. To the best of our knowledge, our work is the first sentence-level semantic watermark algorithm targeted against paraphrase attacks.

**Post-Hoc Detection of Machine-Generated Text** In post-hoc methods, applying binary classification models is the most straightforward approach (Zellers et al., 2019; Jawahar et al., 2020; Liu et al., 2022; Mireshghallah et al., 2023; Pu et al., 2023). These methods are applicable to black-box generators but need sufficiently large corpus for fine-tuning. Another type of post-hoc detection is based on statistical patterns within generation, including token likelihood (Gehrmann et al., 2019), rank (Solaiman et al., 2019), entropy (Ippolito et al., 2020), and likelihood gap at perturbation (Mitchell et al., 2023; Su et al., 2023). These methods have better interpretability but are reliable only with white-box access to generators. Sadasivan et al. (2023) question the theoretical reliability of detection while Chakraborty et al. (2023) support detection is achievable.

We defer further related works on LSH, watermarking for copyright, and contrastive learning to SSA due to space reasons.

## 5 Conclusion

We introduce SemStamp, a novel sentence-level semantic watermark for LLMs. The watermark is injected by mapping candidate sentences into embeddings with a paraphrase-robust encoder, partitioning the semantic space through LSH, and rejection sampling to generation sentences with valid region embeddings. Empirical results show that

Figure 5: Generation Examples. Paraphrase examples are based on SemStamp generations. Additional examples are presented in Figure 6 in the Appendix. **SemStamp generations are equally coherent and contextually sensible compared to non-watermarked generations.**



SemStamp is not only robust to paraphrase attacks but also more quality-preserving than a token-level baseline watermark algorithm. We also propose a bigram paraphrase attack which effectively weakens the token-level watermark while only causing minor performance deterioration to SemStamp. We hope SemStamp can serve as an effective tool for regulating the proliferation of machine-generated texts.

### Limitations and Discussion

Robustness to Stronger AttacksSince SemStamp operates on the sentence level, it is not robust against attacks on the inter-sentence level. For example, a recently proposed paraphraser Dipper [16] includes sentence reordering. Our algorithm is also less effective when the machine text is embedded in a relatively large portion of human text. We leave the exploration of stronger attacks to future work.

Semantic Constraint from LSHWhile the LSH partitioning divides the full semantic space into sub-regions, enforcing the "valid region" requirement during generation may potentially reduce the generation flexibility. Interestingly, we use a small LSH dimension (\(d=3\)) and we do not observe a visible quality degradation. A potential explanation is that with a smaller LSH dimension, the valid partition also becomes larger, which does not impose a strong semantic constraint and provides enough diversity for generations, as we found in our experiments (SS3.2).

SpeedDue to the nature of rejection sampling, text generation with SemStamp is slower than non-watermarked generation by a factor of 20.9 with LSH dimension \(d=3\) and margin \(m=0.02\) (SS3.2), and by a factor of 5.26 when \(d=3\) and \(m=0\) (Table 3). However, since candidate sentences for rejection sampling have the same LM context, it is possible to conduct batch sampling of candidate next sentences, which speeds up watermarked generation while increasing the memory overhead. We see the additional computation cost for SemStamp as a cost for robustness: adding the watermark on the semantic space trades-off speed for better detection accuracy under paraphrase attacks. A potential mitigation to speed up is through parallel decoding multiple sentences across multiple GPUs, which makes our algorithm a viable option for cloud servers with ample computing. For future work, it would be exciting to make this procedure more efficient with techniques from controlled generation [48; 70; 16].

Reverse EngineeringSince our sentence encoder and LSH hyperplanes are not public, it is not straightforward for a curious attacker to reverse engineer the configurations and we leave it for future work to explore. The difficulty of reverse engineering can also be increased by using a larger LSH dimension, while the watermark could be less robust to paraphrase attack.

Bigram Paraphrase Attack ControlWe control the "intensity" degree of bigram paraphrase attack by constraining the paraphrase candidate selection with a BERTScore constraint. Removing the constraint will more forcefully lower AUROC at the expense of paraphrase quality.

Finally, due to lack of space we defer discussions on ethical impacts to SSF.

## Acknowledgement

This research is supported in part by the Office of the Director of National Intelligence (ODNI), Intelligence Advanced Research Projects Activity (IARPA), via the HIATUS Program contract #2022-22072200004. This material is also funded by the DARPA Grant under Contract No. HR001120C0124. We also gratefully acknowledge support from NSF CAREER Grant No. IIS2142739, NSF Grants No. IIS2125201, IIS2203097, and the Alfred P. Sloan Foundation Fellowship. The views and conclusions contained herein are those of the authors and should not be interpreted as necessarily representing the official policies, either expressed or implied, of ODNI, IARPA, or the U.S. Government. The U.S. Government is authorized to reproduce and distribute reprints for governmental purposes notwithstanding any copyright annotation therein.

## References

* Atallah et al. (2001) Mikhail J. Atallah, Victor Raskin, Michael Crogan, Christian Hempelmann, Florian Kerschbaum, Dina Mohamed, and Sanket Naik. 2001. Natural language watermarking: Design, analysis, and a proof-of-concept implementation. In _Information Hiding_, pages 185-200, Berlin, Heidelberg. Springer Berlin Heidelberg.
* Atallah et al. (2002) Mikhail J. Atallah, Victor Raskin, Christian F. Hempelmann, Mercan Karahan, Radu Sion, Umut Topkara, and Katrina E. Triezenberg. 2002. Natural language 

watermarking and tamperproofing. In _International Workshop on Information Hiding_, pages 196-212.
* Cer et al. (2017) Daniel Cer, Mona Diab, Eneko Agirre, Inigo Lopez-Gazpio, and Lucia Specia. 2017. SemEval-2017 task 1: Semantic textual similarity multilingual and crosslingual focused evaluation. In _Proceedings of the \(I1th\) International Workshop on Semantic Evaluation (SemEval-2017)_, pages 1-14, Vancouver, Canada. Association for Computational Linguistics.
* Chakraborty et al. (2023) Souradip Chakraborty, Amrit Singh Bedi, Sicheng Zhu, Bang An, Dinesh Manocha, and Furong Huang. 2023. On the possibilities of ai-generated text detection. _arXiv preprint arXiv:2304.04736_.
* Charikar (2002) Moses S Charikar. 2002. Similarity estimation techniques from rounding algorithms. In _Proceedings of the thiry-fourth annual ACM symposium on Theory of computing_, pages 380-388.
* Choi et al. (2022) Seungtaek Choi, Myeongho Jeong, Hojae Han, and Seung won Hwang. 2022. C2l: Causally contrastive learning for robust text classification. In _AAAI Conference on Artificial Intelligence_.
* Christ et al. (2023) Miranda Christ, Sam Gunn, and Or Zamir. 2023. Undetectable watermarks for language models. _ArXiv_, abs/2306.09194.
* Fang et al. (2020) Hongchao Fang, Sicheng Wang, Meng Zhou, Jiayuan Ding, and Pengtao Xie. 2020. Cert: Contrastive self-supervised learning for language understanding. _ArXiv_, abs/2005.12766.
* Fu et al. (2023) Yu Fu, Deyi Xiong, and Yue Dong. 2023. Watermarking conditional text generation for ai detection: Unveiling challenges and a semantic-aware watermark remedy. _ArXiv_, abs/2307.13808.
* Gao et al. (2021) Tianyu Gao, Xingcheng Yao, and Danqi Chen. 2021. Simcse: Simple contrastive learning of sentence embeddings. _arXiv preprint arXiv:2104.08821_.
* Gehrmann et al. (2019) Sebastian Gehrmann, Hendrik Strobelt, and Alexander M Rush. 2019. Gltr: Statistical detection and visualization of generated text. _arXiv preprint arXiv:1906.04043_.
* Gu et al. (2022) Chenxi Gu, Chengsong Huang, Xiaoqing Zheng, Kai-Wei Chang, and Cho-Jui Hsieh. 2022. Watermarking pre-trained language models with backdooring. _arXiv preprint arXiv:2210.07543_.
* Guu et al. (2018) Kelvin Guu, Tatsunori B. Hashimoto, Yonatan Oren, and Percy Liang. 2018. Generating sentences by editing prototypes. _Transactions of the Association for Computational Linguistics_, 6:437-450.
* Hadsell et al. (2006) R. Hadsell, S. Chopra, and Y. LeCun. 2006. Dimensionality reduction by learning an invariant mapping. In _2006 IEEE Computer Society Conference on Computer Vision and Pattern Recognition (CVPR'06)_, volume 2, pages 1735-1742.
* Han et al. (2022) Seungju Han, Beomsu Kim, and Burua Chang. 2022. Measuring and improving semantic diversity of dialogue generation. In _Findings of the Association for Computational Linguistics: EMNLP 2022_.
* House (2023) The White House. 2023. FACT SHEET: BidennHarris Administration Secures Voluntary Commitments from Leading Artificial Intelligence Companies to Manage the Risks Posed by AI.
* Indyk and Motwani (1998) Piotr Indyk and Rajeev Motwani. 1998. Approximate nearest neighbors: Towards removing the curse of dimensionality. In _Proceedings of the Thirtieth Annual ACM Symposium on Theory of Computing_, STOC '98, page 604-613, New York, NY, USA. Association for Computing Machinery.
* Ippolito et al. (2020) Daphne Ippolito, Daniel Duckworth, Chris Callison-Burch, and Douglas Eck. 2020. Automatic detection of generated text is easiest when humans are fooled. In _Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics_, pages 1808-1822, Online. Association for Computational Linguistics.
* Ippolito et al. (2022) Daphne Ippolito, Florian Tramer, Milad Nasr, Chiyuan Zhang, Matthew Jagielski, Katherine Lee, Christopher A Choquette-Choo, and Nicholas Carlini. 2022. Preventing verbatim memorization in language models gives a false sense of privacy. _arXiv preprint arXiv:2210.17546_.
* Jawahar et al. (2020) Ganesh Jawahar, Muhammad Abdul-Mageed, and Laks Lakshmanan, V.S. 2020. Automatic detection of machine generated text: A critical survey. In _Proceedings of the 28th International Conference on Computational Linguistics_, pages 2296-2309, Barcelona, Spain (Online). International Committee on Computational Linguistics.
* A Conditional Transformer Language Model for Controllable Generation. _arXiv preprint arXiv:1909.05858_.
* Kim et al. (2019) Byeongchang Kim, Hyunwoo Kim, and Gunhee Kim. 2019. Abstractive Summarization of Reddit Posts with Multi-level Memory Networks. In _NAACL-HLT_.
* Kim et al. (2021) Taeuk Kim, Kang Min Yoo, and Sang goo Lee. 2021. Self-guided contrastive learning for bert sentence representations. _ArXiv_, abs/2106.07345.
* Kirchenbauer et al. (2023a) John Kirchenbauer, Jonas Geiping, Yuxin Wen, Jonathan Katz, Ian Miers, and Tom Goldstein. 2023a. A watermark for large language models. _arXiv preprint arXiv:2301.10226_.
* Kirenbauer et al. (2020) John Kirenbauer, Jonas Geiping, Yuxin Wen, Manli Shu, Khalid Saifullah, Kezhi Kong, Kasun Fernando, Aniruddha Saha, Micah Goldblum, and Tom Goldstein. 2023b. On the reliability of watermarks for large language models 

* Klein and Nabi (2020) Tassilo Klein and Moin Nabi. 2020. Contrastive self-supervised learning for commonsense reasoning. _ArXiv_, abs/2005.00669.
* Krishna et al. (2023) Kalpesh Krishna, Yixiao Song, Marzena Karpinska, John Wieting, and Mohit Iyyer. 2023. Paraphrasing evades detectors of ai-generated text, but retrieval is an effective defense. _arXiv preprint arXiv:2303.13408_.
* Kryscinski et al. (2021) Wojciech Kryscinski, Nazneen Rajani, Divgansh Agarwal, Caiming Xiong, and Dragomir Radev. 2021. Booksum: A collection of datasets for long-form narrative summarization. _arXiv preprint arXiv:2105.08209_.
* Kuditipudi et al. (2023) Rohith Kuditipudi, John Thickstun, Tatsunori Hashimoto, and Percy Liang. 2023. Robust distortion-free watermarks for language models. _ArXiv_, abs/2307.15593.
* 2022 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP)_, pages 8237-8241.
* Liu et al. (2022) Xiaoming Liu, Zhaohan Zhang, Yichen Wang, Yu Lan, and Chao Shen. 2022. Coco: Coherence-enhanced machine-generated text detection under data limitation with contrastive learning. _ArXiv_, abs/2212.10341.
* Liu et al. (2023) Yixin Liu, Hongsheng Hu, Xuyun Zhang, and Lichao Sun. 2023. Watermarking text data on large language models for dataset copyright protection. _ArXiv_, abs/2305.13257.
* Logeswaran and Lee (2018) Lajanugen Logeswaran and Honglak Lee. 2018. An efficient framework for learning sentence representations. _arXiv preprint arXiv:1803.02893_.
* Mireshghalah et al. (2023) Fatemehsadat Mireshghalah, Justus Mattern, Sicun Gao, R. Shokri, and Taylor Berg-Kirkpatrick. 2023. Smaller language models are better blackbox machine-generated text detectors. _ArXiv_, abs/2305.09859.
* Mitchell et al. (2023) Eric Mitchell, Yoonho Lee, Alexander Khazatsky, Christopher D. Manning, and Chelsea Finn. 2023. Detectopt: Zero-shot machine-generated text detection using probability curvature. In _International Conference on Machine Learning_.
* OpenAI (2022) OpenAI. 2022. ChatGPT.
* OpeAI (2023) OpenAI. 2023. GPT-4 Technical Report.
* Pagnoni et al. (2022) Artidoro Pagnoni, Martin Graciarena, and Yulia Tsvetkov. 2022. Threat scenarios and best practices to detect neural fake news. In _Proceedings of the 29th International Conference on Computational Linguistics_, pages 1233-1249, Gyeongju, Republic of Korea. International Committee on Computational Linguistics.
* Pu et al. (2023) Xiao Pu, Jingyu Zhang, Xiaochuang Han, Yulia Tsvetkov, and Tianxing He. 2023. On the zero-shot generalization of machine-generated text detectors. In _Findings of the Association for Computational Linguistics: EMNLP 2023_, pages 4799-4808, Singapore. Association for Computational Linguistics.
* Raffel et al. (2020) Colin Raffel, Noam Shazeer, Adam Roberts, Katherine Lee, Sharan Narang, Michael Matena, Yanqi Zhou, Wei Li, and Peter J Liu. 2020. Exploring the limits of transfer learning with a unified text-to-text transformer. _Journal of Machine Learning Research_ (JMLR).
* Ravichandran et al. (2005) Deepak Ravichandran, Patrick Pantel, and Eduard Hovy. 2005. Randomized algorithms and NLP: Using locality sensitive hash functions for high speed noun clustering. In _Proceedings of the 43rd Annual Meeting of the Association for Computational Linguistics (ACL'05)_, pages 622-629, Ann Arbor, Michigan. Association for Computational Linguistics.
* Reimers and Gurevych (2019) Nils Reimers and Iryna Gurevych. 2019. Sentence-BERT: Sentence embeddings using Siamese BERT-networks. In _Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing and the 9th International Joint Conference on Natural Language Processing (EMNLP-IJCNLP)_, pages 3982-3992, Hong Kong, China. Association for Computational Linguistics.
* Sadasivan et al. (2023) Vinu Sankar Sadasivan, Aounon Kumar, Sriram Balasubramanian, Wenxiao Wang, and Soheil Feizi. 2023. Can ai-generated text be reliably detected?
* Solaiman et al. (2019) Irene Solaiman, Miles Brundage, Jack Clark, Amanda Askell, Ariel Herbert-Voss, Jeff Wu, Alec Radford, and Jasmine Wang. 2019. Release strategies and the social impacts of language models. _ArXiv_, abs/1908.09203.
* Su et al. (2023) Jinyan Su, Terry Yue Zhuo, Di Wang, and Preslav Nakov. 2023. Detectllm: Leveraging log rank information for zero-shot detection of machine-generated text. _arXiv preprint arXiv:2306.05540_.
* Durme and Lal (2010) Benjamin Van Durme and Ashwin Lall. 2010. Online generation of locality sensitive hash signatures. In _Proceedings of the ACL 2010 Conference Short Papers_, pages 231-235, Uppsala, Sweden. Association for Computational Linguistics.
* Venugopal et al. (2011) Ashish Venugopal, Jakob Uszkoreit, David Talbot, Franz Och, and Juri Ganitkevitch. 2011. Watermarking the outputs of structured prediction with an application in statistical machine translation. In _Proceedings of the 2011 Conference on Empirical Methods in Natural Language Processing_.
* Wang et al. (2023) Lean Wang, Wenkai Yang, Deli Chen, Haozhe Zhou, Yankai Lin, Fandong Meng, Jie Zhou, and Xu Sun. 2023. Towards codable text watermarking for large language models. _ArXiv_, abs/2307.15992.
* Weidinger et al. (2017) Laura Weidinger, John Mellor, Maribeth Rauh, Conor Griffin, Jonathan Uesato, Po-Sen Huang, Myra 

Cheng, Mia Glaese, Borja Balle, Atoosa Kasirzadeh, et al. 2021. Ethical and social risks of harm from language models. _arXiv preprint arXiv:2112.04359_.
* Weir et al. (2020a) Nathaniel Weir, Joao Sedoc, and Benjamin Van Durme. 2020a. COD3S: diverse generation with discrete semantic signatures. In _Proceedings of the 2020 Conference on Empirical Methods in Natural Language Processing (EMNLP)_.
* Weir et al. (2020b) Nathaniel Weir, Joao Sedoc, and Benjamin Van Durme. 2020b. COD3S: Diverse generation with discrete semantic signatures. In _Proceedings of the 2020 Conference on Empirical Methods in Natural Language Processing (EMNLP)_, pages 5199-5211, Online. Association for Computational Linguistics.
* Wieting et al. (2022) John Wieting, Kevin Gimpel, Graham Neubig, and Taylor Berg-kirkpatrick. 2022. Paraphrastic representations at scale. In _Proceedings of the 2022 Conference on Empirical Methods in Natural Language Processing: System Demonstrations_, pages 379-388, Abu Dhabi, UAE. Association for Computational Linguistics.
* Yang and Klein (2021) Kevin Yang and Dan Klein. 2021. FUDGE: Controlled text generation with future discriminators. In _Conference of the North American Chapter of the Association for Computational Linguistics_ (NAACL), pages 3511-3535.
* Yao et al. (2023) Hongwei Yao, Jian Lou, Kui Ren, and Zhan Qin. 2023. Promptcare: Prompt copyright protection by watermark injection and verification. _ArXiv_, abs/2308.02816.
* Yoo et al. (2023) Kiyoon Yoo, Wonhyuk Ahn, Jiho Jang, and No Jun Kwak. 2023. Robust multi-bit natural language watermarking through invariant features. In _Annual Meeting of the Association for Computational Linguistics_.
* Zellers et al. (2019) Rowan Zellers, Ari Holtzman, Hannah Rashkin, Yonatan Bisk, Ali Farhadi, Franziska Roesner, and Yejin Choi. 2019. Defending against neural fake news. In H. Wallach, H. Larochelle, A. Beygelzimer, F. d'Alche-Buc, E. Fox, and R. Garnett, editors, _Advances in Neural Information Processing Systems 32_, pages 9054-9065. Curran Associates, Inc.
* Zhang et al. (2020) Jingqing Zhang, Yao Zhao, Mohammad Saleh, and Peter Liu. 2020. Pegasus: Pre-training with extracted gap-sentences for abstractive summarization. In _International Conference on Machine Learning_ (ICML).
* Zhang et al. (2022) Susan Zhang, Stephen Roller, Naman Goyal, Mikel Artetxe, Moya Chen, Shuohui Chen, Christopher Dewan, Mona Diab, Xian Li, Xi Victoria Lin, et al. 2022. OPT: Open Pre-trained Transformer Language Models. _arXiv preprint arXiv:2205.01068_.
* Zhang et al. (2019) Tianyi Zhang, Varsha Kishore, Felix Wu, Kilian Q Weinberger, and Yoav Artzi. 2019. Bertscore: Evaluating text generation with bert. In _International Conference on Learning Representations_ (ICLR).
* Zhang et al. (2018) Yize Zhang, Michel Galley, Jianfeng Gao, Zhe Gan, Xiujun Li, Chris Brockett, and William B. Dolan. 2018. Generating informative and diverse conversational responses via adversarial information maximization. In _NeurIPS_.
* Zhao et al. (2023) Xuandong Zhao, Prabhanjan Ananth, Lei Li, and Yu-Xiang Wang. 2023. Provable robust watermarking for ai-generated text. _arXiv preprint arXiv:2306.17439_.

 



## Appendix A Additional Related Works

Locality-Sensitive Hashing in NLPThe application of locality-sensitive hashing (Indyk and Motwani, 1998; Charikar, 2002) in NLP dates back to Ravichandran et al. (2005), where LSH is used for high-speed noun clustering. Van Durme and Lall (2010) show that the LSH method of Charikar (2002) can enable fast approximated online computation of cosine similarity. Guu et al. (2018) use LSH to efficiently compute lexically similar sentences in a prototype-then-edit sentence generation model. Closely related to our work, Weir et al. (2020b) generate semantically diverse sentences by conditioning a sequence-to-sequence model on the LSH signature of sentence embeddings.

Watermarked Natural Language Data for CopyrightWatermarked generation can be further applied for data copyright protection. Gu et al. (2022) embed backdoor trigger words as black-box watermarks into LLMs. Liu et al. (2023) propose a novel watermark via backdoor-based membership inference, where backdoor watermarked texts poison unauthorized training models. Yao et al. (2023) focus on protecting the copyright of prompts through inserting the secret key into the prompt optimization stage. These works mainly apply watermark techniques for data copyright protections , whereas our work focuses on exploring the robustness of watermark against paraphrasing.

Contrastive Learning in NLPT contrastive learning (Hadsell et al., 2006) aims at improving the distinguishability of representation by pulling over positive pairs and pushing off negative pairs. In the NLP domain, contrastive learning can be applied to sentence embedding Logeswaran and Lee (2018), and further used in downstream tasks like natural language inference Li et al. (2022), understanding Fang et al. (2020), reasoning Klein and Nabi (2020), classification Choi et al. (2022) etc. Logeswaran and Lee (2018) apply unsupervised contrastive learning between current sentence candidates and context sentences to effectively learn sentence representation. Gao et al. (2021) further apply supervised contrastive learning in sentence embedding by using annotated pairs from natural language inference. Kim et al. (2021) propose a self-guided contrastive learning between embeddings from a fixed model and a fine-tuned model.

## Appendix B Watermark Detection

Kirchenbauer et al. (2023a) proposes using a one-proportion \(z\)-test on the number of green list tokens to detect watermarks, assuming the following null hypothesis:

\[H_{0}:\] _The text is not generated (or written) knowing a watermarking green list rule._

The null hypothesis is rejected when the \(z\)-score computed based on the number of green tokens in a piece of text \(T\) exceeds a given threshold \(M\):

\[z_{\text{KGW}}=\frac{N_{G}-\gamma N_{T}}{\sqrt{\gamma(1-\gamma)N_{T}}}, \tag{3}\]

where \(N_{G}\) denotes the number of green tokens, \(N_{T}\) refers to the total number of tokens contained in the given piece of text \(T\), and \(\gamma\) is a chosen ratio of green tokens. During detection time, the number of green tokens in each piece of text will be counted. According to Eq. 3, a higher ratio of detected green tokens means a higher \(z\)-score, determining with more confidence that the text is machine-generated.

We adapt this one proportion \(z\)-test to SemStamp, modifying the null hypothesis and using sentence as our basic unit:

\[H_{0}:\] _The text is not generated (or written) knowing a rule of valid and blocked partitions in the semantic space._

\[z_{\textsc{SemStAMP}}=\frac{S_{V}-\gamma S_{T}}{\sqrt{\gamma(1-\gamma)S_{T}}}, \tag{4}\]

where \(S_{V}\) refers to the number of valid _sentences_, \(\gamma\) is the ratio of valid sentences out of the total number of sentences \(S_{T}\) in a piece of text \(T\).

During detection time, we first break a piece of texts into individual sentences and detect the number of valid sentences \(S_{V}\) to calculate \(z_{\textsc{SemSTAMP}}\). We detect a machine-generated text when \(z_{\textsc{SemSTAMP}}>M_{r}\), where \(M_{r}\) is located according to a given false positive rate \(r\): We define machine-generated as the positive class in classical classification setting and non-machine-generated as the negative class. We iterate through a range of possible \(m\in[0,6.0]\) until there is a \(M_{r}=m\) such that \(r\) percentage of human (negative-class) texts is misclassified as machine-generated. For example, we let \(r=0.05\) for the **TP@5%** metric in Table 1



## Appendix C Effect of LSH dimension \(d\)

In Table 3, we discover that fewer LSH dimensions will make a sentence more likely to stay in the same region after being paraphrased. We define LSH Consistency as the ratio of paraphrased sentences that have the same LSH signature as the original sentence over the total number of paraphrased sentences. A higher consistency ratio indicates better robustness.

Geometrically, when the LSH dimension is lower, there are fewer partitioned semantic regions, each having a larger space. A paraphrase will have a similar representation with its source sentence in the semantic space, which will be more likely to remain in the same semantic region if each region is larger.

On the other hand, lowering the number of LSH dimensions will also slightly increase the average number of sentences sampled to produce one valid sentence (Average Number of Sentences Sampled). We ultimately decide on a minor sacrifice in speed for the gain of accuracy and choose \(d=3\). We choose \(\gamma=0.25\) following Kirchenbauer et al. [2023a], where the authors show that larger greenlist ratios will lower the \(z\)-score.

## Appendix D Additional Experimental Results

We include additional experimental results on paraphrase quality, i.e., the BERTScore between original and paraphrased generations under different settings, in Table 4.

We provide paraphrased detection results of the KTH algorithm [Kuditipudi et al., 2023] in Table 6 and Unigram-Watermark[Zhao et al., 2023] in Table 5. We find that the KTH watermark performs poorly against KGW and SemStamp.

Although Unigram-Watermark enjoys strong robustness against paraphrasing attacks, it has the crucial limitation of being readily backed by an adversary. Since Unigram-Watermark can be understood as a variant of KGW [Kirchenbauer et al., 2023a] but with only one fixed greenlist initialized at the onset of generation. An adversary can reverse-engineer this greenlist by brute-force submissions to the detection API of \(|V|\) times, where each submission is repetition of a token drawn with out replacement from the vocabulary \(V\) of the tokenizer. Therefore, upon each submission to the detection API, the adversary will be able to determine if the submitted token is in the greenlist. After \(|V|\) times of submission, the entire greenlist can be reverse-engineered. On the other hand,

Figure 6: Additional Generation Examples. Non-Watermarked refers to the original model without adding any watermark. Baseline Watermark refers to [Kirchenbauer et al., 2023a]. Paraphrase examples are based on SemStamp generations.

\begin{table}
\begin{tabular}{c c c} \hline \hline LSH Dim (_d_) & Average \# of Sentences Sampled \(\downarrow\) & LSH Consistency \(\uparrow\) \\ \hline
3 & 5.26 & **.720** \\
4 & 4.53 & .666 \\
8 & 4.26 & 508 \\
16 & **4.14** & .335 \\ \hline \hline \end{tabular}
\end{table}
Table 3: Effects of Increasing LSH Dimensions at margin \(m=0.0\). The sampling rate is the average number of sentences sampled to produce one valid (watermarked) sentence.



such hacks are not applicable to SemStamp, since SemStamp does not fix the list of valid regions and blocked regions during generation, and the adversary needs to have access to the private sentence embedder and LSH hyperplanes to hack SemStamp.

In summary, despite having strong robustness against various paraphrase attacks, UnigramWatermark has a notable vulnerability that may limit its applicability in high-stake domains where adversaries can conduct reverse-engineering.

Computing Infrastruture and BudgetWe run sampling and paraphrase attack jobs on 8 A40 GPUs, taking up a total of around 100 GPU hours.

## Appendix E Additional Details

Condition for consistent LSH signatureFor robustness, the SemStamp algorithm would need the LSH signature of the paraphressed sentence to be unchanged from the signature of the original sentence. This requires that for each LSH digit \(i\), the sign of the dot product between the embedded sentence and the normal vector \(n^{(i)}\) should not change before and after paraphrasing:

\[\begin{split}\mathbbm{1}\big{(}n^{(i)}\cdot v_{\text{orig}}>0 \big{)}=&~{}\mathbbm{1}\big{(}n^{(i)}\cdot v_{\text{para}}>0 \big{)},\\ &\forall i\in\{1\dots d\},\end{split} \tag{5}\]

where \(v_{\text{orig}}=M_{\text{embd}}(s^{(t)})\) and \(v_{\text{para}}=M_{\text{embd}}(G(s^{(t)}))\) are the embeddings for the original and paraphrased sentences, respectively, and \(G\) is the paraphraser.

Cosine SimilarityIn SS2.2, we slightly abuse the notation and use \(\cos(\mathbf{x},\mathbf{y})\) to denote the _cosine similarity_ between two vectors \(\mathbf{x}\) and \(\mathbf{y}\). That is,

\[\cos(\mathbf{x},\mathbf{y})=\frac{\mathbf{x}\cdot\mathbf{y}}{|\mathbf{x}||\mathbf{y}|}. \tag{6}\]

Sentence DelimitationDuring generation time, a full candidate next sentence is considered generated if the language model has generated a new delimiter punctuation, i.e., a comma, period, question mark, or exclamation mark.

Data PreprocessingWe separate the data points, which are paragraphs of news (RealNews) and book summaries (BookSum), into sentences using nltk.sent_tokenize. Additionally, we add a period mark to every sentence that does not end in a comma, period, question mark, or exclamation mark.

Prompt for GPT-3.5-Turbo ParaphraseTo use GPT-3.5-Turbo as a paraphraser, we provide the following prompt:

Previous context: {context} \n Currentsentence to paraphrase: {sent} We define sent to be the target sentence to be paraphrased, and context as the list of sentences before the target sentence.

For the bigram paraphrase attack, we provide the following prompt:

Previous context: {context} \n Paraphrase in {num-beams} different ways and return a numbered list : {sent} where num-beams specifies the number of candidate sentences. A higher num-beams will strengthen the bigram paraphrase attack but also at the cost of more computational resources.

## Appendix F Ethical Impacts

As language models become increasingly capable of generating realistic texts, the risk of misusing language model generations, such as spreading misinformation, practicing plagiarism, and violating copyrights, has become imminent. Furthermore, on a fundamental level, the inability to distinguish humans from machines poses threats to establishing the basic level of mutual understanding and trust that bonds society. Robust detection of machine-generated text is crucial for preventing the misuse of large language models by properly attributing the source of online texts. Although current LLMs are often exposed to users as API endpoints, malicious users can still postprocess and paraphrase the API-generated response to escape the injected watermark. This motivates us to study watermark robustness against paraphrasing in this work. We hope that the proposed SemStamp algorithm can mitigate the risk of LLM misuse by providing a reliable method to counter paraphrasing attacks on watermarked generations 



\begin{table}
\begin{tabular}{c|c c c} \hline \hline  & \multicolumn{3}{c}{BookSum} \\ _Algorithm_ & _AUC_\(\uparrow\) & _TP@17\%_\(\uparrow\) & _TP@9.5\%_\(\uparrow\) \\ \hline KGW & 95.9 & 82.1 & 91.0 \\ KTH & 51.7 & 5.0 & 5.8 \\ SemStamp & **97.8** & **83.7** & **92.0** \\ \hline \end{tabular}
\end{table}
Table 6: Paraphrased detection results on the BookSum dataset. The paraphraser used is Pegasus. We find that the KTH watermark performs poorly against KGW and SemStamp.

\begin{table}
\begin{tabular}{c|c c c|c c} \hline \hline  & \multicolumn{4}{c}{RealNews} & \multicolumn{2}{c}{BookSum} \\ _Algorithm\(\downarrow\)__Paraphrase\(\rightarrow\)_ & Pegasus & Parrot & GPT3.5 & Pegasus & Parrot & GPT3.5 \\ \hline KGW & 71.0 / 66.6 & 57.1 / 58.4 & 54.8 / 53.3 & 71.8 / 69.3 & 62.0 / 61.8 & 60.3 / 56.7 \\ SSTAmp & 72.2 / 69.7 & 57.2 / 57.4 & 55.1 / 53.8 & 72.7 / 70.2 & 62.9 / 62.4 & 61.8 / 58.4 \\ \hline \hline \end{tabular}
\end{table}
Table 4: BERTScore between original and paraphrased generations under different settings. All numbers are in percentages. The first number in each entry is under vanilla paraphrase attack while the second number is under the bigram paraphrase attack. **Bigram paraphrase attack poses only minor degradation on semantic similarity with original sentence compared to vanilla paraphrase attack.**

\begin{table}
\begin{tabular}{c|c c c} \hline \hline  & \multicolumn{2}{c}{RealNews} & \multicolumn{2}{c}{BookSum} \\ _Algorithm\(\downarrow\)__Paraphrase\(\rightarrow\)_ & Pegasus & Parrot & Pegasus & Parrot \\ \hline KGW & 95.9 / 92.1 & 88.5 / 83.0 & 97.3 / 96.5 & 94.6 / 93.1 \\ SSTAmp & 97.8 / 96.5 & 93.3 / 93.1 & 99.2 / 98.9 & 97.5 / 97.5 \\ Uniform-Watermark & **99.1** / **98.4** & **98.9** / **98.7** & **99.4** / **99.7** & **99.5** / **99.6** \\ \hline \hline \end{tabular}
\end{table}
Table 5: Paraphrased detection results of Unigram-Watermark. We find that Unigram-Watermark demonstrates strong robustness against paraphrase attacks, but has the vulnerability to being reverse-engineered, which we discuss in §D.