G umbel S oft: Diversified Language Model Watermarking via the G umbel M ax-trick (2024)

Abstract: Large language models (LLMs) excellently generate human-like text, but also raise concerns about misuse in fake news and academic dishonesty. Decoding-based watermark, particularly the watermark based on the GumbelMax trick (GM watermark), is a standout solution for safeguarding machine-generated texts due to its notable detectability. However, GM watermark encounters a major challenge with generation diversity, always yielding identical outputs for the same prompt, negatively impacting generation diversity and user experience. To overcome this limitation, we introduce a new type of GM watermark, the Logits-Addition watermark, as well as three variants that aim to enhance diversity, particularly the GumbelSoft watermark (i.e., the softmax variant of the Logits-Addition watermark). When assessed for detectability in high diversity settings, our Gumbelsoft demonstrates superior performance, with its AUROC score exceeding those of the two alternative variants by a margin of 0.1 to 0.3 and outperforming other decoding-based watermarking methods by a minimum of 0.1.

## introduction the emergence of large language models (llms), exemplified by @xcite , has enabled the generation of remarkably human-like content, facilitating tasks such as writing (shanahan and clarke, 2023), coding @xcite , and fostering creativity. however, this technological advancement brings forth the potential for malicious applications, including social engineering @xcite , fake news fabrication @xcite , and academic dishonesty. consequently, the need for effective detection of machine-generated texts has become increasingly critical. various strategies have been proposed to distinguish machine-generated texts from human-figure 1 : one significant limitation of gm watermark lies in their production of identical responses to the same queries. such determinism can lead to user dissatisfaction, as individuals may become frustrated with llm recommending the same outcomes for repeated prompts. this issue primarily stems from the deterministic nature of both the pseudo-random function and the decoder function. to address this concern, we propose three solutions: solutions i and ii aim to introduce variability into the decoder function, whereas solution iii seeks to inject uncertainty into the pseudo-random function. to address the challenge of generating diverse outputs of the gm watermark, our analysis delves into the core mechanism of decoding-based watermarks. we discover that these watermarks share a cohesive framework, as illustrated in figure 3 . the primary cause of uniform completions for identical prompts is traced back to the deterministic nature of both the decoder and pseudo-random functions in the gm watermark. to mitigate this, we propose two strategies to introduce variability into the decoder function and one strategy to the pseudorandom function: 1) implement a drop mechanism with a predefined probability d p , enabling direct sampling from the language model without watermark insertion. 2) replace the "argmax" operation in gumbelmax watermark with "sampling from softmax" with temperature τ . 3) adjust the watermark key, derived from the pseudo-random function, by cyclically shifting it r positions-a method to effectively randomize the watermark key. a critical aspect of this exploration is balancing detectability with diversity. integrating a dropout probability and shifting the watermark key boosts diversity but also reduces detectability. we propose replacing the argmax operation with "sampling from softmax" to enhance diversity without significantly compromising the watermark's integrity. this approach ensures that even though selections diverge from "argmax", they still achieve high pertoken scores, preserving the statistical foundation of the watermark. further investigation into gm watermark leads us to question the necessity for an exponential transformation in the gumbelmaxtrick for embedding watermarks, a technique outlined by @xcite . instead, we employ the gumbelmax-trick directly for watermark embedding and propose a distinct type of gm watermark, termed the logits-addition watermark. our experiments reveal that the gumbelsoft watermark, the softmax variant of the logits-addition watermark, consistently outperforms other gm watermark diversified variants in the auroc metric, achieving a margin of 0.1 to 0.3 in high diversity settings. additionally, the gumbelsoft watermark surpasses other decoding-based watermarks in au-roc by at least 0.1 on qa tasks, while maintaining low perplexity. for a clearer understanding of these findings, we have illustrated the relationships among the gumbelmax-trick, the gm watermark (including exponential and logits-addition), and their diversified variants in figure 2 . in conclusion, our gumbelmax-trick logits-addition watermark exponential watermark two gumbelmax trick based watermarks three variants to encourage diversity @xmath0.2 sample directly from llm @xmath1.3 replace "argmax" with "softmax" contributions are threefold: ## related work machine-generated text detection can be roughly categorized into three approaches. zero-shot methods. this approach, known as "model self-detection," necessitates full access to the language model and utilizes statistical measures such as perplexity and entropy. noteworthy contributions include gehrmann et al. (2019)'s gltr, vasilatos et al. (2023)'s perplexity analysis, and yang et al. (2023a)'s n-gram overlaps. mitchell et al. (2023) introduced a perturbationbased method, while deng et al. (2023) proposed a bayesian surrogate model. recent studies encompass krishna et al. (2023), who advocate for retrieval against paraphrase attacks, and su et al. (2023), who leverage log-rank ratios. additionally, solaiman et al. (2019) employ log probability, bao et al. (2023) focus on conditional probability curvature, and venkatraman et al. (2023) utilize uniform information density for improved detection. the primary limitation of zero-shot methods lies in their requirement for complete access to the language model. training-based methods. these involve classifiers trained to distinguish between machinegenerated and human-written texts. chen et al. (2023 2023 ) applying positive unlabeled training for classifier development. @xcite . a drawback of training-based methods is their potential over-fitting to specific datasets and models. watermarking techniques. recent advancements in hidden signal watermarking in texts can be categorized into post-edited and decoding-based watermarking. post-edited watermarking involves text formatting or lexical changes @xcite @xcite , while decoding-based watermarking in the era of large language models (llms) embeds statistical signals during decoding. notable techniques in this domain include @xcite 's red-green list and @xcite 's robust watermarking. unbiased watermarks, which preserve the original token distributions, have been explored by @xcite and @xcite . additionally, multi-bit watermarking, which embeds complex information, is examined by @xcite and @xcite . several techniques for embedding watermarks include text formatting, as demonstrated by @xcite and @xcite , and context-aware lexical substitution, as explored by @xcite . syntactic modifications are discussed by @xcite and @xcite . training data watermarking is addressed by @xcite and @xcite . a publicly detectable watermark is proposed by @xcite , while leveraging semantic meaning for robustness is examined by @xcite . @xcite propose pf-watermark to further improve the text perplexity. there are also some surveys on machine-generated content detection @xcite ) and text watermarking @xcite . ## method in this section, we will first provide an overview of the decoding-based watermark framework and the gumbelmax-trick. following this, we'll delve into the application of the gumbelmax-trick in text watermarking and examine their limitations. concluding the section, we will present our recommended watermark scheme, specifically crafted to overcome these identified limitations. ## experiment this section presents a comparative study of three diversified variants (refer to figure 1 ) of both exponential and logits-addition watermarks, with an emphasis on aspects such as detectability, diversity, and quality. following this, the optimal diversified variant of the gm watermark, the gumbelsoft watermark, is identified and compared against several existing decoding-based watermark schemes (see appendix a for details). ## conclusion we observed that the gumbelmax-trick-based watermark(gm watermark) produces identical responses to identical queries due to the deterministic nature of both the decoder and pseudo-random functions. to address this, we introduce three diversified variants aimed at enhancing gm watermark diversity. furthermore, we question the need for an exponential transformation @xcite in watermark embedding and propose a new approach named logits-addition watermark. our experiments across these variants for both exponential and logits-addition watermarks identified gumbelsoft, a softmax-based logits-addition variant, as the optimal choice. comparative analysis with other decoding-based watermarks demonstrated that gumbelsoft surpasses in detectability, maintains lower perplexity, and ensures higher robustness.