Low-Entropy Watermark Detection via B ayes’ Rule Derived Detector (2025)

Abstract: Text watermarking, which modify tokens to embed watermark, has proven effective in detecting machine-generated texts. Yet its application to low-entropy texts like code and mathematics presents significant challenges. A fair number of tokens in these texts are hardly modifiable without changing the intended meaning, causing statistical measures to falsely indicate the absence of a watermark. Existing research addresses this issue by rely mainly on a limited number of high-entropy tokens, which are considered flexible for modification, and accurately reflecting watermarks. However, their detection accuracy remains suboptimal, as they neglect strong watermark evidences embedded in low entropy tokens modified through watermarking. To overcome this limitation, we introduce Bayes’ Rule derived Watermark Detector (BRWD), which exploit watermark information from every token, by leveraging the posterior probability of watermark’s presence. We theoretically prove the optimality of our method in terms of detection accuracy, and demonstrate its superiority across various datasets, models, and watermark injection strategies. Notably, our method achieves up to 50% and 70% relative improvements in detection accuracy over the best baselines in code generation and math problem-solving tasks, respectively. Our code is available at https://github.com/cczslp/BRWD.

## introduction text watermarking is an effective technique for differentiating machine-generated text from humanwritten content that subtly injects an invisible marker, i.e. watermark, into text. it serves as a safeguard against unauthorized or malicious use of large language models, such as creating fake news @xcite and election manipulating @xcite . † corresponding author. 1 our code is available at https://github.com/cczslp /brwd . non-watermarked code watermarked code if @xmath0: return false if n % 2 < 1: return false for j in range(3, n, 2): if n % @xmath1: return false return true if @xmath2: return false if n % 2 == 0: return false for i in range(3, n, 2): if n % @xmath3: return false return true this position is probably red but appears green entropy-based detector low-entropy neglected underestimated! our detector strong evidence! entropy-based detector high-entropy emphasized overestimated! our detector moderate evidence! this position is arbitrarily red or green low-entropy tokens high-entropy tokens a token in red list a token in green list generative watermark, which integrates watermark during llm's generation process, generally exhibits superior detectability and robustness. a generative watermarking method injects a watermark through perturbing tokens' output distribution during text generation, and subsequently computes a score indicating its presence for detection. for instance, the kgw method @xcite partitions the model's vocabulary into green and red tokens at each generation step, and increases the output probability of green tokens, resulting in higher proportion of green tokens in a watermarked text. subsequent detection is performed by computing the z-score of green token occurrences in the text. however, text watermarking exhibits suboptimal performance in low-entropy scenarios, such as code generation or mathematical problem solving, where a fair number of tokens are unmodifiable without compromising output quality. statistics, such as the z-score computed for these tokens, suggest the absence of a watermark, providing contrary evidence of its presence and thus diminishing the effectiveness of the watermark detector. this limitation poses a substantial barrier to detecting malicious or unauthorized activities in software de-velopment, academic exams @xcite , and job interviews @xcite , raising concerns about social equality and ethical use of technology. subsequent works @xcite attempt to enhance detection accuracy by prioritizing high-entropy tokens, which are considered more adjustable to watermark perturbations, thus more indicative of watermark's presence. in contrast, low-entropy tokens receive little weight during detection. nevertheless, their detection accuracy remains suboptimal, as they fail to leverage the substantial information in highly indicative low-entropy tokens, which are actually modified through watermarking. we identified a misalignment between the entropy-based mechanism and the goal of detection: entropy inadequately captures the modifiability of a token, nor does it account for the actual modification introduced by watermarking. to illustrate the misalignment, consider a low-entropy position where the probability is concentrated on a red token but presents a green one. it serves as a strong evidence of the watermark's presence. however, entropy-based detectors would assign a small weight to this position and neglect this evidence. to address this misalignment, we propose bayes' rule-derived-watermark dector (brwd), which quantifies the impact of watermark injection on every token. we calculate the posterior likelihood of every token altered by watermark injection with bayes' rule, then aggregate it into a total score. a token that deviates more from its original distribution and consequently shows stronger evidence of watermark injection has a greater impact on the score. in this way, we extract more information from every token, especially highly indicative and low-entropy tokens, instead of neglecting them. we prove that brwd achieves optimal true positive rate (tpr), given any false positive rate (fpr) limit. our method is compatible with various watermark injection techniques. extensive experiments across multiple language models, generative tasks and watermark injection schemes, under scenarios with or without prompts, demonstrate the superior performance of our approach. in particular, under a 1% fpr constraint in mathematical contexts, brwd boosts the tpr from below 60% to over 90%. furthermore, brwd demonstrates adaptability to general high-entropy texts and exhibits robustness against removal attacks. ## related works text watermarking emerges as a promising solution to identifying machine-generated contents. they can be categorized into two types @xcite : generative watermarking methods and watermarking applied to existing text. ## preliminaries as we focus on generative watermarking due to its superior detectability and robustness, the term "watermarking" will henceforth refer to generative watermarking. in this section, we present the preliminaries of watermark injection and detection. notations used throughout the paper are provided in appendix a. ## method in this section, we first illustrate our motivation by examining the limitations of current watermark detection methods and then present our approach. ## experiments we conduct experiments across different datasets, models and watermark injection methods. tasks and datasets. we select two tasks: code generation and math problem solving. for code generation, we use humaneval @xcite and mbpp @xcite datasets which contain python programming problems and reference answers. for math problem solving, we select 500 samples from gsm8k @xcite , which contains mathematical problems in english. in all experiments, we use texts longer than 15 tokens for detection. ## results in this section, we present and analyze experimental results from multiple settings and perspectives. ## conclusion in this study, we identify the misalignment in entropy-based watermark detectors and propose a detector that addresses this problem. our detector utilizes bayes' rule to fully leverage the distribution given by language model. we provide theoretical analysis on its optimality and empirically verify its superiority in low-entropy scenario. across various models and watermark injection methods, under scenarios with or without prompts, our method demonstrates superior accuracy and robustness. notably, compared to the best baseline, our method achieves up to 1.5 times accuracy on the code datasets and up to 1.7 times accuracy on the mathematics dataset.