k-SEMSTAMP : A Clustering-Based Semantic Watermark for Detection of Machine-Generated Text (2024)

Abstract: Recent watermarked generation algorithms inject detectable signatures during language generation to facilitate post-hoc detection. While token-level watermarks are vulnerable to paraphrase attacks, SEMSTAMP (Hou et al., 2023) applies watermark on the semantic representation of sentences and demonstrates promising robustness. SEMSTAMP employs localitysensitive hashing (LSH) to partition the semantic space with arbitrary hyperplanes, which may lead to a suboptimal trade-off between robustness and speed. We propose k-SEMSTAMP, a simple yet effective enhancement of SEMSTAMP, utilizing k-means clustering as an alternative of LSH to partition the embedding space with awareness of inherent semantic structure. Experimental results indicate that k-SEMSTAMP saliently improve its robustness and sampling efficiency while preserving the generation quality, advancing a more effective tool for machine-generated text detection.

heading: 1 Introduction
text: To facilitate the detection of machine-generated text (Mitchell et al., 2019), recent watermarked generation algorithms usually inject detectable signatures (Kuditipudi et al., 2023; Yoo et al., 2023; Wang et al., 2023; Christ et al., 2023; Fu et al., 2023; Hou et al., 2023, i.a.). A major concern for these approaches is their robustness to potential attacks, since a malicious user could attempt to remove the watermark with text perturbations such as editing and paraphrasing (Wang et al., 2024; Krishna et al., 2023; Sadasivan et al., 2023; Kirchenbauer et al., 2023b; Zhao et al., 2023). Hou et al. (2023) propose SEMSTAMP, a paraphrase-robust and sentence-level watermark which assigns signatures to each watermarked sentence according to the locality sensitive hashing (LSH) (Indyk and Motwani, 1998) partitioning of semantic space (see 2.1). While demonstrating promising robustness
against paraphrase attacks, SEMSTAMP arbitrarily partitions the semantic space by a set of random hyperplanes, possibly splitting semantically similar sentences into different partitions (see Fig.1).
This limitation motivates our proposed method, k-SEMSTAMP (detailed in §2.2), which partitions the space via k-means clustering (Lloyd, 1982) on the semantic structure of a given text domain (e.g. news, narratives, etc.). In §3, we show that the clustering-based partitioning in k-SEMSTAMP greatly improves its robustness against sentencelevel paraphrase attacks and sampling efficiency.1

heading: 2 Approach
text: We first review the existing watermark algorithms for machine-generated text detection (§2.1) and introduce our proposed watermark (§2.2).

heading: 2.1 Preliminaries
text: Token-Level Watermark Kirchenbauer et al. (2023a) develop a notable token-level watermark
1We have released the code for reproducibility. Corresponding authors: Abe Hou, Jingyu Zhang, and Tianxing He.
1706
Prompt: Jackie woke up.
Language Model
It was 3pm.
She smiled.
She felt good.
Decode
Robust Sentence Encoder
embed
embed
embed
Semantic Space
Cluster centroid with index
Blocked sentence embedding
Valid sentence embedding
k-means cluster boundary
Semantic Space
Figure 2: An overview of the proposed k-SEMSTAMP algorithm. k-means clustering partitions the semantic space into semantically similar regions. The sentence generation is accepted if the closest cluster of its sentence embedding corresponds to a "valid" region in the semantic space.
algorithm. Given a token history w1:t−1, the vocabulary V is pseudo-randomly divided into a “green list” G(t) and a “red list” R(t), where a hash of the previous token wt−1 is used as the seed of the partition. The algorithm then adds a bias to the logits of all tokens in the green-list and sample the next token with an increased probability from the greenlist. For a given piece of text, the watermark can be detected by conducting one proportion z-test (detailed in §C) on the number of green list tokens.
SEMSTAMP Under the intuition that common sentence-level paraphrase modifies tokens but preserves sentence meaning, Hou et al. (2023) introduce SEMSTAMP to apply watermark on sentence semantics by partitioning the embedding space with locality sensitive hashing (LSH).
To initialize the LSH partitioning, d normal vectors are randomly sampled from a Gaussian distribution to specify d hyperplanes in the semantic space Rh. For an embedding vector v ∈ Rh, a d-bit binary LSH signature is assigned, where each
digit specifies the position of v in relation to each hyperplane. Each signature c ∈ {0, 1}d indexes a region consisting of all vectors with signature c.
During generation, given a sentence history denoted by s(0) . . . s(t−1), the space of signatures is pseudorandomly partitioned into a set of “valid” regions G(t) and a set of “blocked” region R(t). The LSH signature of the last generated sentenceis used as the random seed to control randomness. A new sentence generation, s(t), will be accepted and if its embedding belongs to any valid region, and rejected otherwise. To detect the watermark in a given piece of text, a one-proportion z-test is performed on the number of sentences whose signatures belong to valid regions (see §C).
2.2 k-SEMSTAMP As discussed earlier, SEMSTAMP partitions the semantic space with random planes, which could potentially separate semantically similar sentences into two different regions, as shown in Fig.1. Paraphrasing sentences near the margins of regions may shift their sentence embeddings to a nearby region, resulting in suboptimal watermark strength. This weakness motivates our proposed k-SEMSTAMP, a simple yet effective enhancement of SEMSTAMP that partitions the semantic space with k-means clustering (Lloyd, 1982).
To initialize k-SEMSTAMP , we assume the language model generates text in a specific domain D (e.g., news articles, scientific articles, etc.). We aim to model the semantic structure of D and partition its semantic space into k regions. Concretely, we first randomly sample a large number of data from D. We obtain their sentence embeddings with a robust sentence encoder fine-tuned on D with contrastive learning (detailed in §A). We cluster the sentence embeddings into K clusters with k-means (Lloyd, 1982) and save the cluster centroids. We index a region with i ∈ {1, ...,K} representing the
Algorithm 1 k-SEMSTAMP text generation algorithm and subroutines
Input: language model PLM, prompt s(0), the text domain D, the number of sentences to generate T . Params: sentence embedding model fine-tuned on D, MDembd with embedding dimension h, maxout number Nmax, margin m > 0, valid region ratio γ ∈ (0, 1), the number of k-means clusters K, a large prime number p, an integer N . Output: generated sequence s(1) . . . s(T ).
procedure k-SEMSTAMP CK ← INITIALIZE(D,K) to initialize K cluster centroids based on D. for t = 1, 2, . . . , T do
1. Find the index of the closest cluster centroid of the previously generated sentence, q(t−1) ← ASSIGN(s(t−1), CK), and use q(t−1) · p as the seed to randomly divide the index set of clusters CK into a “valid region set” G(t) of size γ ·K and a “blocked region set” R(t) of size (1− γ) ·K. 2. repeat Sample a new sentence from LM, until the index of the closest cluster centroid of the new sentence, q(t), is in the “valid region set”, and the margin requirement MARGIN(s(t),m) is satisfied or sampling has repeated over Nmax times.
3. Append the selected sentence s(t) to context. end for return s(1) . . . s(T )
end procedure
function INITIALIZE(D,K) D′N ∼ D // sample N sentences from D CK ← K-MEANS(D ′ N ,K) // obtain k cluster centroids
return CK end function function ASSIGN(s, CK ) // find the index of the closest centroid by cosine distance
return argmini=1,...,K dcos(v, ci), where ci ∈ CK end function
set of all vectors assigned to the i-th centroid. The generation process is analogous to SEMSTAMP (Hou et al., 2023), as illustrated in Fig.2: given a sentence history s(0) . . . s(t−1), K regions are pseudorandomly partitioned into a set of valid regions G(t) of size γ ·K and a set of blocked regions R(t) of size (1− γ) ·K, where γ ∈ (0, 1) is the ratio of valid regions. The cluster assignment of s(t−1), C(s(t−1)), seeds the randomness of the partition at time step t, where C(.) returns the cluster index by finding the closest cluster centroid of the input sentence embedding. We then conduct rejection sampling and only sentences whose embeddings fall into any valid regions (i.e., C(s) ∈ G(t)) are accepted while the rest are rejected. If no valid sentence is accepted after a preset maxout number (Nmax) of tries, the last decoded sentence will be chosen. The full algorithm is presented in Algo 1.
Algorithm 2 k-SEMSTAMP detection algorithm Input: a piece of text T , saved k-means cluster centroids CK Params: sentence embedding model finetuned on D, MDembd, z-threshold range Z, human-written texts H , a large prime number p, valid region ratio γ ∈ (0, 1), number of k-means clusters K. Output: a z-score based on the ratio of detected sentences. procedure DETECT(T,CK )
s1, ..., sN ← SENTENCE-TOKENIZE(T) q(1) ← ASSIGN(s1, CK) seed← q(1) · p G(1) ← RANDOM-SAMPLE(seed,K, γ) // pseudorandomly sample a set of cluster centroid indices of size K · γ, where the randomness of sampling is controlled by seed.
for t = 2, . . . , N do q(t) ← ASSIGN(st, CK) if q(t) ∈ G(t−1) then
SV += 1 end if
SEED ← q(t) · p G(t) ← RANDOM-SAMPLE(seed,K, γ)
end for end procedure z ← SV −γN√ γ(1−γ)N return z
Cluster Margin Constraint To prevent the sampled sentences from being assigned to a nearby cluster after paraphrasing, we propose a cluster margin constraint similar to (Hou et al., 2023). We constrain the sentence embeddings to be sufficiently away from the cluster boundaries (visualized in Fig.1). Concretely, the cosine distance (dcos) of the candidate sentence embedding (v) to the closest centroid (cq) needs to be smaller than other cluster centroids by at least a margin m:
dcos(v, cq) < min i∈{1,...,K}\q dcos(v, ci)−m, (1) where q is the index of the closest cluster centroid to v, i.e., q = argmini=1,...,K dcos(v, ci), and v = Membd(s
(t)) is the embedding of the generated sentence at time step t by a robust sentence embedder Membd.
The detection procedure of k-SEMSTAMP is analogous to SEMSTAMP which uses oneproportion z-test on the number of sentences belong to valid regions, explained in §C and Algo 2.

heading: 3 Experiments

heading: 3.1 Experimental Setup
text: Following Hou et al. (2023), we conduct paraphrase attack experiments and compare the detection robustness of watermarked generations.
Task and Metrics We evaluate 1000 watermarked generations after paraphrase, respectively
on the RealNews subset of the C4 dataset (Raffel et al., 2020) and on the BookSum dataset (Kryściński et al., 2021). We paraphrase watermarked generations sentence-by-sentence with the Pegasus paraphraser (Zhang et al., 2020), Parrot used in Sadasivan et al. (2023), and GPT-3.5-Turbo (OpenAI, 2022). We also implement the strong bigram paraphrase attack as detailed in Hou et al. (2023). Detection robustness of paraphrased watermarked generations is measured with area under the receiver operating characteristic curve (AUC) and the true positive rate when the false positive rate is at 1% and 5% (TP@1%, TP@5%).2 Generation quality is measured with perplexity (PPL) (using OPT2.7B (Zhang et al., 2022)), trigram text entropy (Zhang et al., 2018) (Ent-3), i.e., the entropy of the trigram frequency distribution of the generated text, and Sem-Ent (Han et al., 2022), an automatic metric for semantic diversity. Following the setup in Han et al. (2022), we perform k-means clustering (k = 50) with the last hidden states of OPT-2.7B on text generations, and Sem-Ent is defined as the entropy of semantic cluster assignments of test generations. We also measure the paraphrase quality with BERTScore (Zhang et al., 2019) between original generations and their paraphrases.
Generation We use OPT-1.3B (Zhang et al., 2022) as our base autoregressive LM. To obtain robust sentence encoders specific to text domains for k-SEMSTAMP generations, we fine-tune two versions of Membd, respectively on RealNews (Raffel et al., 2020) and on BookSum (Kryściński et al., 2021) datasets (see §A for specific procedure and parameter choices).
Following Hou et al. (2023) and Kirchenbauer et al. (2023a), we sample at a temperature of 0.7 and a repetition penalty of 1.05, with 32 being the prompt length and 200 being the default generation length. Results with various lengths are included in Fig. 4. For k-SEMSTAMP , we perform k-means clustering on embeddings of sentences in 8k paragraphs, respectively on RealNews and BookSum. We keep k = 8 and a valid region ratio γ = 0.25, which is consistent with the number of regions in SEMSTAMP, and we use a rejection margin m = 0.035.
2We denote machine-generated text as the “positive” class and human text as the “negative” class. A piece of text is classified as machine-generated when its z-score exceeds a threshold chosen based on a given false positive rate. See §C.
Baselines Our baselines include popular watermarking algorithms Kirchenbauer et al. (2023a), SEMSTAMP, UNIGRAM-WATERMARK (Zhao et al., 2023), and the Semantic Invariant Robust (SIR) watermark in Liu et al. (2023), implemented with their recommended setups.

heading: 3.2 Results
text: Detection Detection results in Table 1 show that k-SEMSTAMP is more robust to paraphrase attacks than KGW (Kirchenbauer et al., 2023a) and SEMSTAMP across Pegasus, Parrot, and GPT3.5-Turbo paraphrasers and their bigram attack variants, as measured by AUC, TP@1%, and TP@5%. In particular, k-SEMSTAMP demonstrates considerable robustness against GPT-3.5, in which none of SEMSTAMP and KGW performed strongly. While UNIGRAM-WATERMARK (Zhao et al., 2023) also demonstrates strong robustness against paraphrase, it has a critical vulnerability to reverse-engineering attacks. We discuss its vulnerability and experimental results in §D. The BERTScores of paraphrases are presented in Table 5.
Domain Shifts Since k-SEMSTAMP finetunes sentence-embedder from a specified text domain, we investigate the robustness of the fine-tuned sentence-embedder inputs from a different domain. In Table 2, we show that k-SEMSTAMP experiences a drop in robustness when using a cross-domain sentence-embedder. Nevertheless, k-SEMSTAMP is able to retain some robustness compared to KGW and SIR, staying especially resilient against Pegasus-bigram attacks.
Sampling Efficiency k-SEMSTAMP not only demonstrates stronger paraphrastic robustness, but also generates sentences with higher sampling efficiency. To produce the results on BookSum (Kryściński et al., 2021) in Table 1, k-SEMSTAMP samples 13.3 sentences on average to accept one valid sentence, which is 36.2% less compared to the average 20.9 sentences sampled by SEMSTAMP. We analyze the reasons of candidate sentences for being rejected respectively by k-SEMSTAMP and SEMSTAMP, discovering that around 42.0% and 80.7% of the sentences are rejected due to the margin requirements. Since k-SEMSTAMP determines the cluster centroids by k-means clustering on the semantic structure of a given text domain, the embeddings of most candidate sentences generated in this text domain are closer to the centroids and
away from the margins, and they are less likely to relocate to a blocked region after paraphrase.
Quality Table 3 shows that the perplexity, text diversity, and semantic diversity of both SEMSTAMP and k-SEMSTAMP generations are on par with the base model without watermarking, while KGW and SIR notably degrade perplexity. Qualitative examples of k-SEMSTAMP are presented in Figure 3 and 5. Compared to non-watermarked generation, k-SEMSTAMP convey the same level of coherence and contextual sensibility. The Ent-3 and Sem-Ent metrics also show that k-SEMSTAMP preserves token and semantic diversity of generation compared to non-watermarked generation.
Generation Length As shown in Fig. 4, kSEMSTAMP has higher AUC than Kirchenbauer et al. (2023a) and than SEMSTAMP across most generation lengths by number of tokens.

heading: 4 Conclusion
text: We propose k-SEMSTAMP, a simple but effective enhancement of SEMSTAMP. To watermark generated sentences, k-SEMSTAMP maps embeddings of candidate sentences to a semantic space which is partitioned by k-means clustering, and only accept sampled sentences whose embeddings fall into a valid region. This variant greatly improves the paraphrastic robustness and sampling speed.
Limitations A core component of k-SEMSTAMP is performing k-means clustering on a particular text domain and partitioning the semantic space according to the semantic structure of the text domain. However, this requires specifying the text domain of generation to initialize k-SEMSTAMP . If the k-means clusters and the sentence embedder are not specific to the text domain, k-SEMSTAMP suffers from a minor drop in paraphrastic robustness (see Table 2 for experimental results with k-SEMSTAMP using a sentence embedder trained on RealNews).
Ethical Considerations The proliferation of large language models capable of generating realistic texts has drastically increased the need to detect machine-generated text. By proposing k-SEMSTAMP, we hope that practitioners will use this as a tool for governing modelgenerated texts. Although k-SEMSTAMP shows promising paraphrastic robustness, it is still not perfect for all kinds of attacks and thus should not be solely relied on in all scenarios. Finally, we hope this work motivates future research interests in not only semantic watermarking but also general adversarial-robust methods for AI governance.

heading: Acknowledgement
text: We would like to thank Brian Lu and following members of the Intelligence Amplification Lab: Yining Lu, Nikil Sharma, Jiefu Ou, and Tianjian Li for their support and constructive feedback to this work. We are also grateful for the insightful advice from the broader JHU CLSP community and our anonymous reviewers and senior members at ACL.

heading: A Contrastive Learning and Sentence Encoder Fine-tuning
text: To make sentence encoders robust to paraphrase, we fine-tune following the procedure in Hou et al. (2023) and Wieting et al. (2022).
First, we paraphrase 8000 paragraphs from RealNews (Raffel et al., 2020) and BookSum (Kryściński et al., 2021) using the Pegasus paraphraser (Zhang et al., 2020) through beam search with 25 beams. We then fine-tune two SBERT models3 with an embedding dimension h = 768 for 3 epochs with a learning rate of 4 × 10−5, using the contrastive learning objective with a margin δ = 0.8:
min θ
∑
i
max { δ − fθ(si, ti) + fθ(si, t′i), 0 } , (2)
where fθ measures the cosine similarity between sentence embeddings, fθ(s, t) = cos ( Mθ(s),Mθ(t) ) , and Mθ is the sentence encoder parameterized by θ that is to be fine-tuned.

heading: B Algorithms
text: The algorithms of k-SEMSTAMP are presented in Algorithm 1.

heading: C Watermark Detection
text: The detection of both SEMSTAMP and kSEMSTAMP follows the one-proportion z-test framework proposed by Kirchenbauer et al. (2023a). The z-test is performed on the number of green-list tokens in Kirchenbauer et al. (2023a), assuming the following null hypothesis:
Null Hypothesis 1. A piece of text, T, is not generated (or written by human) knowing a watermarking green-list rule.
The green-list token z-score is computed by:
z = NG − γNT√ γ(1− γ)NT , (3)
where NG denotes the number of green tokens, NT refers to the total number of tokens contained in the given piece of text T , and γ is a chosen ratio of green tokens.
The z-test rejects the null hypothesis when the green-list token z-score exceeds a given threshold M . During the detection of each piece of text, the
3sentence-transformers/all-mpnet-base-v1
number of the green tokens is counted. A higher ratio of detected green tokens after normalization implies a higher z-score, meaning that the text is classified as machine-generated with more confidence.
Hou et al. (2023) adapts this z-test to detect SEMSTAMP, according to the number of valid sentences rather than green-list tokens.
Null Hypothesis 2. A piece of text, T, is not generated (or written by human) knowing a rule of valid and blocked partitions in the semantic space.
z = SV − γST√ γ(1− γ)ST , (4)
where SV refers to the number of valid sentences, γ is the ratio of valid sentences out of the total number of sentences ST in a piece of text T . To detect SEMSTAMP, the given piece of text, T , is first broken into sentences and the number of valid sentences SV is counted to calculate the z-score. Likewise, the null hypothesis 2 is rejected when the z-score exceeds a threshold M.
The detection procedure of k-SEMSTAMP is analogous to SEMSTAMP. We break a text into sentences and count the number of valid sentences to calculate the z-score, where only the determination of whether a sentence falls into a valid region is different. k-SEMSTAMP assigns the sentence generation to its closest cluster centroid and consider if the index of the cluster centroid belongs to a valid partition. See the full detection algorithm in Algorithm 2.
For a comprehensive evaluation of detection robustness, we consider a range of possible threholds Mf ∈ [0, 6.0], where each Mf is determined by a given false positive rate rf , and consider machinegenerated text as "positive" and human text as "negative" in a traditional classification setting. We estimate a false positive rate of corresponding Mf by computing the misclassification rate of human text samples. We let rf = 0.01 and rf = 0.05 to respectively measure TP@1% and TP@5% metrics in Table 1.

heading: D Additional Experimental Results
text: Table 4 shows the detection results of UNIGRAMWATERMARK (Zhao et al., 2023) against paraphrase attacks, demonstrating more robustness compared to SEMSTAMP and k-SEMSTAMP . However, UNIGRAM-WATERMARK has the key vulnerability of being readily reverse-engineered by
AUC / TP@1% / TP@5% Algorithm Domain Pegasus Pegasus-bigram Parrot Parrot-bigram
UNIGRAM-WATERMARK RealNews 99.1 / 92.2 / 96.4 98.4 / 87.9 / 94.3 98.9 / 82.7 / 94.0 98.7 / 79.6 / 91.5 BookSum 99.4 / 96.4 / 99.0 99.7 / 91.6 / 98.2 99.5 / 91.6 / 97.7 99.6 / 87.8 / 97.2
Table 4: Detection results of UNIGRAM-WATERMARK in Zhao et al. (2023)
an adversary. Since UNIGRAM-WATERMARK can be understood as a variant of the watermark in Kirchenbauer et al. (2023a) but with only one fixed greenlist initialized at the onset of generation. An adversary can reverse-engineer this greenlist by brute-force submissions to the detection API of |V | times, where each submission is repetition of a token wi, i ∈ {1, ..., |V |} drawn without replacement from the vocabulary V of the tokenizer. Therefore, upon each submission to the detection API, the adversary will be able to tell if the submitted token is in the greenlist or not. After |V | times of submission, the entire greenlist can be reverse-engineered. On the other hand, such hacks are not applicable to SEMSTAMP and k-SEMSTAMP , since both algorithms do not fix the list of valid regions and blocked regions during generation. In summary, despite having strong robustness against various paraphrase attacks, UNIGRAM-WATERMARK has a notable vulnerability that may limit its applicability in high-stake domains where adversaries can conduct reverse-engineering.
Computing Infrastruture and Budget We ran sampling and paraphrase attack jobs on 8 A40 and 4 A100 GPUs, taking up a total of around 200 GPU hours.

heading: Future Work
text: Finally, we hope this work motivates future research interests in not only semantic watermarking but also general adversarial-robust methods for AI governance