Improving Detection of Watermarked Language Models
Authors: Dara Bahri
John Frederick Wieting
Venue: Accepted by TMLR ()

Abstract: Watermarking has recently emerged as an effective strategy for detecting the generations of large language models (LLMs). The strength of a watermark typically depends strongly on the entropy afforded by the language model and the set of input prompts. However, entropy can be quite limited in practice, especially for models that are post-trained, for example via instruction tuning or reinforcement learning from human feedback (RLHF), which makes detection based on watermarking alone challenging. In this work, we investigate whether detection can be improved by combining watermark detectors with \emph{non-watermark} ones. We explore a number of \emph{hybrid} schemes that combine the two, observing performance gains over either class of detector under a wide range of experimental conditions.

Published in Transactions on Machine Learning Research (01/2026)
Improving Detection of Watermarked Language Models
Dara Bahri, John Wicting Google Mind
Reviewed on OpenReview: https: // openreview. net/ forum? id= 6nmz tBgngB
Abstract
Watormarking has recently emerged as an effective stratogy for detecting the generations of large language models (LLMs). The strength of a walermark lypically depends strongly on the entropy afforded by the language model and the set of input prompts. However, entropy can be quite limited in practice, especially for models that are post-trained, for example via instruction tuning or reinforcement learning from human feedback (RLHF), which makes detection based on watermarking alone challenging. In this work, we investigate whether detection can be improved by combining watermark detectors with non-watermon wateraaermark ones. We explore a number of hybrid schemes that combine the two, observing performance gains over either class detector under a wide range of experimental conditions.
1 Introduction
General usage of large language models (LLMs) has increased dramatically in recent years, and so has the need for identifying texts generated by LLMs, or Al-generated content (AGC), in the wild. For example, an academic institution may wish to know whether students are using an LLM to do their assignments, or an LLM provider may want to understand where and how their model is being used. One may want to detect whether the text was generated by a specific model or any model. Moreover, in the former case, the detecting party may or may not have white-box access (concretely, an ability to compute log- probabilities) to the model they wish to test against. Parties with white-box access typically the model's owners, and we refer to this sctting as first-party (1P) detection. In contrast, we refer to the detection of one white-box access is not available as Uhird-porly (3P) detection.
models where
The aim of watermarking is to bias the model so that first-party detection becomes more tractable. Most schemes do not modify the LLM's training procodure but instead inject the watermark signal at inference time as part of the autoregressive decoding loop (Aaronson, 2023; Bahri Wieting, 2024; Kuditipudi el al., 2023; Kirchenbauer et al., 2023; Dathathri et al., 2024). Meanwhile, non-watermark detection has mostly been viewed as binary classification task, with the goal of discriminating more models. Strategies here include training a binary
class from another typically
text from text authored by classifier on labeled text samples or computing uncertainty-based scores, such as likelihood, under a specific LLM (Zellers et al., 2019; Solaiman et al., 2019; Gehrmann et al., 2019; Su et al., 2023; Mitchell et al., 2023). The focus of our work is on improving first-party detection by intelligently combining watermark- and non-watermark- based detection approaches.
The entropy of an LLM's distribution over possible responses conditioned on a specific prompt is a key quantity in watermark detection. For example, Bahri & Wieting (2024) provides lower-bound the detection ROC-AUC as function of the entropy of the sampled next-token distribution. In all schemes, watermark performance improves with more available entropy. As a concrete example, consider two input prompls to a watermarked LLM: (1) "Where is Mount Whilney?" and (2) "Wrile me a dreamy haiku about the John Muir Troil in California." " The first prompt is a based inquiry whereas the second more open-ended nature, so the entropy will likely be higher for the second prompt than for the first and the generated haiku will be easier detect via the watermark. The role of entropy for non- watermark detection is less studied. We show how non-watermark detectors, when configured in hybrid setup, can substantially

---

Published in Transactions on Machine Learning Research (01/2026)
bolster watermarking in low entropy regimes. For example, training logistic regression model watermark scores and scores from RoBERTa-based AGC classifier can boost watermarking test accuracy from 75% to over 95%, a 20% absolute improvement, on the 20% of test prompts that afford the least amount of entropy. We study how these two approaches to detection can be combined effectively for better predictive performance than either part as well as computational advantages, and we recommend practical algorithms for deployment.
2
Related Work
We provide concise overview of prior work watermarking and AGC detection.
2.1 Watermarking
Although watermarking, sometimes referred to linguistic steganography, has storied past, interest in its application to modern large language models grew substantially after the seminal works of Kirchenbauer et al. (2023) and Aaronson (2023). Many successful techniques employ pseudo-random functions (PRFs) and cryptographic hashes and are applied token-by-token during the autoregressive decoding loop. Kirchenbauer et al. (2023) uses a PRF to bias the next-token probability distribution so that certain tokens known only the watermarker are made more probable. The degree of bias is a hyper-parameter that trades off text, distortion and watermark strength. Aaronson (2023) proposes clever strategy that selects the token in the next-token distribution that has a high PRF value and high likelihood in way that makes the process appear distortion-fres free. Kuditipudi et al. (2023) applies scheme similar to Aaronson (2023) but to increase robustness to attacks such paraphrasing, the pseudo-random numbers are determined by cycling through a secret, predetermined sequence of values rather than by n-grams. Meanwhile, SYNTUID (Dathathri et al., 2024) samples a large number of tokens from the next-loken distribution and pits them head-to-head in a series tournaments, the winner of which is selected as the next token. Lee et al. (2023) adapts Kirchenbauer et al. (2023)'s schome for the code-generation task by applying the watermark only at decoding steps that have sufficient entropy.
Black- box schemes that only require way to sample sequences from the LLM have been devised (Yang et al., 2023a; Giboulot & Furon, 2024; Chang et al., 2024; Bahri & Wieting, 2024). Most notably, Bahri & Wicting (2024) uses elements from Aaronson (2023) to construct a general framework for black-box distortion-free watermarking, which can be used effectively when white-box access is available.
Ways of evading, spoofing, and even stealing watermarks have been extensively studied (Zhang et al., 2023; Gu et al., 2023; Jovanović et al., 2024; Cheng et al., 2025). To address the weakness of many schemes to attacks such token substitution or paraphrasing, watermarking based on semantics invariant features has also been proposed (Liu et al., 2023; Hou et al., 2023; Ren et al., 2023; Yoo et al., 2023). Fernandez et al. (2023) tests various watermarking schemes on classical NLP benchmarks and introduces new statistical tests for example, they suggest skipping duplicate n-grams during testing. Liang et al. (2024) conducts a comprehensive asscssment of watermarking strategics, finding that incorporating a non-watermark RoBERTa-based detector like the way we do can help boost robustness to allacs. The purpose of our work to generalize this focused insight by proposing and then carefully evaluating various hybrid schemes.
2.2 AGC Detection
A powerful paradigm to detect AGC is finetune pretrained LM on sizable datasets for classification (Zellers et al., 2019; Solaiman et al., 2019); notably, Hu et al. (2023) and Tian et al. (2023) do so via adversarial and positive-umlabeldd (PU) learning respectively, while Tay et al. (2020) applies it to the task of discriminating different LM generators from one another.
A second camp of approaches focuses on capturing statistical patterns of AGC using little to no training data such as the intrinsic dimensionality of generated text (Tulchinskii et al., 2024), perplexity rank log-rank (Gehrmann et al., 2019; Su et al., 2023), perplexity curvature (Mitchell et al., 2023), and n-gram patterns (Yang et al., 2023b).
2

---

Published in Transactions on Machine Learning Research (01/2026)
Mireshghallah et al. (2024) shows, interestingly, that small LM generators make for better zero-shot detectors than larger ones. Mao et al. (2024) leverages the observation that LLMs change more words when tasked to rewrite human text than to rewrite AGC. Zhang et al. (2024) shows that existing detectors are inadequate for detecting Al-revised human-writte writnn tezt, text that a human wrote with assistance from an LLM. Intcrestingly, Russell et al. (2025) found that humans who use LLMs for writing tasks can outperform many commercial and open source detectors even in the presence of paraphrasing and humanization.
As usage of large language models is still nascent, it remains to be seen precisely how they are used in the wild, be it with good or nefarious intentions. Bahri et al. (2021) conducts large-scale study to this end by running GPT-2 detector on half billion web pages.
Ways to evade detection have been studied. For example, both Lu et al. (2023) and Kumarage et al. (2023) propose a prompting strategy that encourages LLMs to produce more human- -lonang text, while Krishna et al. (2024) presents a paraphrasing model called DIPPER that can successfully fool detectors.
3 Experimental Setup
The task of detection is not without inherent ambiguities. For example, suppose LLM regurgitates text from its (human) training corpus verbatim - should we consider this sample human or AGC? Arguments can be made for both sides; for the purposes of our work, we consider it AGC.
As watermarking is most commonly studied in the IP setting, our evaluation is restricted to the IP setting as well. However, we consider both IP and 3P detectors in our hybrid scheme, as even 3P detector can provide lift for 1P detection when the negative class consists of human samples. We treat the problem as binary classification task, where positive samples come from model, denoted Mo, and negative samples are either human or generations from different model, denoted M, (theirs). We report performance metrics for both the human and model negative classes separately, a distinction often missing in prior works, Each dataset consists of prompts and human responses. For each prompt, we generate responses under Me and Mo, where the latter equipped with various walermarking methods. We now discuss our choice of datasets, watermarks, and detection strategies.
3.1 Models, Datasets, Hyperparameters, and Compute
We consider two distinct settings: when M, is DEMMA--BBINSTRUUS (Team et al., 2024b)' and M, is MISTRRA--7B-INSTRUCE (Jiang et al., 2023)² as well as the reverse assignment.
We use two test datasets: (atabricks-dolly--55t (Conover et al., 2023), an open source dataset of instruction- following examples for brainstorming, classification, closed QA, generation, information extraction, open QA, and summarization. Exactly replicating Bahri & Wieting (2024), we use prompts from the brainstorming, generation, open QA (i.e. general QA), and summarization categorics, whose human responses are at least 50 tokens long (save one example, which was removed because the prompt was extremely long); this amounts of 5,233 prompts total.
We use the training and test splits of eli5-category' (Fan et al., 2019). Prompts are formed by concatenating the prompt and selftitle fields. Only examples whose prompt field is non non-empty and contains a ? are kept for a total of 83,089 train and 4,855 test samples.
We always decode by random sampling with temperature 1 and do not employ top-p or top-k strategies. For each prompt in both test datasets, generate four wot-mnaerdrr responses, along with a watermarked one under each scheme. We always force a minimum (maximum) of 250 (300) new tokens by disabling the stop token for the first 250 tokens, re-enabling it, and slopping the generation al 300, regardless of whether the stop token was encountered. For both the train and test split of catt-cate the human response is
3

---

Published in Transactions on Machine Learning Research (01/2026)
taken to be the one with the highest score. Also, for the train split only, we run non-watermarkod generation enforcing maximum of 300 tokens but no minimum number of tokens.
To simulate real-world use, we de-tokenize the outputs to obtain plain text, and re-tokenize them during scoring. We study performance as a function of token length T v 250 by truncating samples to their first T tokens.
Experiments were run on SOGB A100 or H100 GPUs using PyTorch and HuggingFace's Transformers models with bfloat16 quantization for total compute cost of about 2,000 GPU hours.
3.2 Entropy
Entropy is a key quantity in analysis. With æ denoting the input prompt and y Pm( x)) the sampled response of length T from model M, the response entropy H (x) is,
B, - log Par x) = E, Σ log (M Vcr.z) = Σ E vel
E
- log Par ves,
=H (2,8e))
-H((x)
where (x, yei) is the entropy of the next-token distribution given prompt x and partial response yei (of length i i- 1) and H,(x) this quantity averagod over partial responscs. In other words, H,(x) is the entropy of the next-token distribution for token position i that we see on average.
If we condone sample reuse for the sake of computational efficiency, then Hx(x) and (x) can be estimated
with i.i.d. samples (y²,...,,) y"} from PM((- x) as,
" (1(()) = -
T Σ Σ 14 (2, and A(xx) Σ Al(x)). j=1
We bucket prompts x in our test set by their estimated response entropy (x) (where the responses used for the estimate are sampled from the model sans watermarking) and analyze delection performance for each bucket separately so as to directly observe the role of entropy on performance. We choose T = 100 and n = 4.
3.3 Watermarks
The watermark schemes we consider here operate at a token level. To facilitate describing them below, let p the next-token probability distribution. Moreover, if F is a cumulative distribution function (CDF), let F[s] be a single draw from a pseudorandom number generator (PRNG) for seeded with integer seed 8, F(z) F evaluated at x, and F& the CDF for the sum of k i..d. random variables where each is distributed F. Furthermore, let named distributions represent their CDFs; for example, U 1)(0.5) is the CDF for the standard uniform distribution evaluated al 0.5. Let our LLM have vocabulary V of size V and h be a cryptographic hash function (e.g. SHA-256) from 2 to E. K E Z is used to denote the secret integer key that is known only to the watermarking party. For both watermark and non won-mmatrrark detection, higher scores indicate higher confidence that the test query watermarked / AGC.
Aaronson. At each step, Aaronson (2023) computes a pscudorandom number for each token í E as, where w is the preceding (n - 1)-gram, and | denotes concatenation. Token i" is selected, where i" = argmax, u; 1/p. At test time, r-grams [w:) are extracted from the test query and the detection score is given by.
" -O, ()((K))
Σ 1=1
log (1 - Ra), where R, = (0, (M(K)))))))
Bahri & Wicting (2024) notes that SA is not length-aware so that a single decision threshold across scores involving various lengths results in poor performance. To remedy this, they propose length-aware score
4

---

Published in Transactions on Machine Learning Research (01/2026)
SAC Gamma(TT (sAa) = (2r(2s)) We use this length-aware score with set to 4; this choice strikes good balance between generation quality / diversity and robustness to attacks.
Bahri. Bahri & Wieting (2024) recently proposed a distortion-free watermarking scheme that requires only black-box access to an LLM. The scheme works by autoregressively sampling m sequences {Q,} from the LLM, each roughly tokens long- Let {X;} be the set of unique sequences from {Qi); (x( represent the set of n-grams for sequence X, and S; the set of seeds {h(K)w) Jee..))) that have had duplicates across the X,'s removed and where random unseen seed is added, if necessary, to ensure all Sy's nonempty. Then X₃- is returned, where,
i" = arrgmx, m/e and G = Σ 1Q; = XXi,, and u, Fisil
Σ F(s) 1
The score sB for sequence X with unique (i.e. de-duplicated) seeds S {h(K) (((K(/))) weE.. is given 8B = Fisi (Eees [(s)). We use the settings optimal for 1P deteccion where while-box access is available their flat scheme with sequence length k = and F (O, We set m 1024 and n
Kirchenbauer et al. (2023) uses the last n tokens to pseudorandomly partition the vocabulary for the next token into two lists: green list of size yV and red list consisting of the remainder. A positive bias of 8 is added to the logits of the green list tokens while those of the red list are left unchanged. This has the effect of modifying so that green list tokens are more probable. The score for a text consisting of We incorporate the latest updates to the algorithm," such as including the current token in the n-gram and skipping duplicate n- n-grams at test time. The scheme modifics the model probability so it's not distortion-fiee but a good balance between watermark strength and generation quality can often be achieved. We set 4, 7 0.25, and 8 E 2,3..
Kirchenbauer.
T tokens, Tg of which were found to be green is SKB = (Tg - - (1 /TT->////TT(1-->>
Kuditipudi. Using the last tokens as the basis for the PRF has the downside that modifying just one of the tokens changes the oulput and subsequently hurts detection. Kuditipudi et al. (2023) addresses this limitation, which we describe detail in the Appendix. We follow their methodology and use 256 seeds. To expedite the permutation test, we precompute 5,000 reference values for the secret list. These values are obtained by sampling snippets from the training set of C4-realneullike (Raffel et al., 2019) across all evaluated target lengths.
3.4 Detectors
We consider the following competitive IP and 3P detection strategies. We omit techniques like Detect- GPT (Mitchell et al., 2023), Ghostbuster (Verma et al., 2023), and DNA-GPT (Yang et al., 2023b) as the baselines covered here, like RADAR (Hu et al., 2023) and Binoculars (Hans et al., 2024), were shown to outperform them.
Log-Likelihood and Rank. We compute per-token log-likelihood (LLh) and rank features of the target text under model M (taken to be our model, Mo), done in prior work (Gehrmann et al., 2019). We note that when scoring response y. standard practice throughout the literature has been to use (M @) as the likelihood, where & is the null or empty prompt. This is incorrect; the true likelihood is: Par()) = Ex-f, Par(y xx), where fz is the distribution over prompts. Since the focus of this work is not on new detection techniques but in novel combinations of existing watermark and detection strategies we continue to define likelihood using the empty prompt. Our fcatures are then:
lyl
lvl
LM =
log Par (Vil Vei). RM( = 1y
RM (vs Vee)
1=1
5

---

Published in Transactions on Machine Learning Research (01/2026)
Rad (3ke (ee)) is the absolute rank of yi in Par(- yes) - the rank is (V) when y4 is the most likely (least likely) token in the next-token distribution. Meanwhile, La represents the log-likelihodd. AGC should have high likelihood and low rank.
DetectLLM. Su et al. (2023) proposes a novel combination of zero-shot likelihood and rank features. Specifically, they propose the Log- Likelhhood Log- Rag-Rank Ratio (IRR) as:
LR = -i= log Pa (Yi yes)
-i=1 log RM (y/ ye1)
where Mo is used for M.
Binoculars. Hans et al. (2024) proposes a 3P detection scheme that leverages uncertainty scores from two LLMs. The Binoculars score is:
log PM, (ys Vee)
B (v)
81 PMr (eE)) log PMa Mee)
1=1
We use the recommended settings: PALCON-TB-INSTRES
for M1 and FALCON-7B for M2.
RADAR. Hu et al. (2023) proposes an adversarial technique to jointly train an AGC detector with a paraphraser. The paraphraser is trained to generate content that will evade the detector while the detector is trained to discern the paraphrascr's outputs. They show the mothod outperforms basclines on 4 datascts and 8 LLMs. We use their open-source RoBERTa-large based detector."
RoBERTa Classifier. As done in prior work, fine-tune RoBERTa (Liu et al., 2019) model for binary classification. To detect Mo. positive samples are random non-warte won-waaerrrreed generations under M. and negative ones are human responses and non-watermarkey generations under M,. To train the classifier, we use the entire train split of cate-oryy (one positive and two negative samples for each prompt), fine-tuning RoBERTa-large with Adam (Kingma Ba, 2014) for epoch using batch size 32, weight decay 0.01 and a linear learning rate schedule with no warm-up and initial rate 5e-5. To make the detector robust shorter and truncated texts, the train set is augmented by additionally including truncated version of each example (to half the number of tokens).
3.5 Hybrid Detection
We explore a number of designs for hybrid detection. For methods that are trainable or require calibration, we use held-out samples from whichever dataset not used for testing. That is, when evaluating on databricks- dolly-15k we use eli5-category for calibration (and vice-versa). This ensures that the new detector does not overfit to the test distribution. For all non-cascade models below, scikit-learn (Buitinck et al., 2013) (version 1.5.1) is used with the default hyperparameter settings.
One-sided WM + DET Cascade (1S). Here, we cascade the watermark and non-watermark detectors together. If the sequence-level watermark score, denoted 8 equals or exceeds threshold A, we predict positive otherwise we predict positive if and only the sequence-level detector score Sa equals or exceeds threshold Aa. Since most watermark detection schemes (at least the ones we evaluate here) are significantly more computationally elficient than non- non-walermark ones that involve LM inference, by positioning them first in the cascade, save compute as the latter is only run residual samples. We quantify our savings later. The hypothesis here is that when s is large, the sample is likely positive, but when it's small, it could be either negative one or positive one for which there was insufficient entropy to embed a strong watermark, in which case we defer to the non-watermark classifier.
Two-sided WM + DET Cascade (2S). Here, if sn 2 A (h for high) we predict positive and if sw. S X (1 for low) we predict negative, otherwise we predict positive if and only if sa 2 Xa. This extends the aforementioned one-sided cascade by baking in the hypothesis that while positives with low entropy have small w,, it is still not small as those of negatives.
htttps://github wwt.ec/we//////eeee IBM/RADAR

---

Published in Transactions on Machine Learning Research (01/2026)
WM
Det
WM Only
Det Only
1S
1S
28
28 +
LR
LR +
LLh
90.4
91.7
93.9
2.2+0.4
94.3
2.6+0.4
96.8
5.1+0.0
LLR
90.4
88.0
92.2
1.8+1.1
92.8
2,4+11
94.2
3,8+1.0
Aaronson
RoBERTa
90.4
86.0
94.7
4.3+1.0
94.7
4.3+1.
98.3
8.0+0.9
Binoculars
90.4
93.1
94.8
1.7
95.0
2.0
96.8
3.7
Radar
90.4
72.9
85.4
-5.0
90.7
0.3
91.5
1.1
LLh
91.5
+1.7
89.8
+1.7
91.4
-0.1
LLR
58.0
88.1
87.9
-0.2
87.9
-0.2
88.2
0.1+0.3
Kir. (0.5)
RoBERTa
58.0
85.9
93.8
7.9
93.8
7.9
96.2
10.3
Binoculars
58.0
93.4
92.6
-0.810.a
92.6
-0.81o.
93.4
-0. 1.1
Radar
58.0
72.9
72.9
0.02g.0
72.9
0.0
72.9
0.0
LLh
80.6
91.7
93.3
1.520.4
93.3
1.620.
94.1
2.420.5
LLR
80.6
87.5
88.6
1.1
88.6
1.1i0.
91.3
3.810.
Kir. (2)
RoBERTa
80.6
85.1
92.1
7.0
92.1
7.0
96.7
11.641.0
Binoculars
80.6
92.9
93.9
1.140.4
94.0
1.1
95.0
2.1
Radar
80.6
72.1
75.0
-5.6+
81.8
1.2
83.4
2.8+1.a
LLh
90.2
90.0
93.5
3.3
92.6
2.4
94.1
3.920.
LLR
90.2
83.1
90.4
0.2
92.8
2.7
94.3
4.1
Kir. (3)
RoBERIa
90.2
82.2
89.3
-0.8+1.1
89.3
-0.8+11
98.2
8.0+0.9
Binoculars
90.2
91.0
94.6
3.6+a.e
94.7
3.7+o.r
95.9
4.9+0.0
Radar
90.2
72.1
89.9
-0.3
90.8
0.611.2
91.2
1.0
LLh
89.9
2.4+0.4
95.0
2.8+o.5
95.3
3.1+0.6
LLR
89.9
87.4
93.6
3.7+1.1
94.3
1.4+11
94.5
4.6+1.1
Bahri
RoBERTa
89.9
92.9
3.0
92.9
3.0+1.
98.6
8.7
Binoculars
89.9
93.2
95.1
1.9
95.1
1.9
96.0
2.8
Radar
89.9
72.4
85.4
-4.4
90.4
0.5
90.3
0.4
LLh
61.0
91.3
90.0
-1.4+a.a
89.9
-1.4o..
91.3
0.0+o.3
L.R
61.0
89.2
89.4
0.31
89.4
0.210
89.9
0.7
Kuditipudi
RoBERTa
61.0
87.6
95.4
7.810.s
95.3
7.710.
96.0
8.510.
Binoculars
61.0
93.9
94.1
0.1
94.0
0.1
94.3
0.4
Radar
61.0
72.1
72.2
0.1
72.2
0.1
72.4
0.3+1.1
Table 1: Main table of accuracy results when human examples are taken as negatives, with a target length of 100 tokens. 18 and 2S stand for the one-sided and two-sided cascades respectively. 18 + is the percent improvement conferred by 1S over the watermark and non-watermark detector, whichever is better. Firstly, we observe that it is not uncommon for the non-watermark detectors to outperform the watermark ones on their own. For example, under Aaronson, Binoculars gives 93.1% 93.1% accuracy whereas the watermark detector obtains 90.4%. When combined with the two-sided cascade (LR), performance boosts to 95.0% (96.8%). Watermarking is not a silver bullet and non-watermark detection strategies can provide substantial complementary value. While one and two-sided cascades provide performance boost, the best gain had with simple learnable model like logistic regression.
is applied to databricks-dolyy dolll--k- and
Logistic Regression (LR). An alternative to manually designing a cascade based on intuition is to have model learn the combination. To that end, train logistic regression model cellearn.iiner__nne Logistticeeegeesssiiooo wtw.aeowwwoooeeoooooooeeceeeeeee
z-score normalized {s sa} features.
MLP. To see whether higher capacity parametric model confers gains over logistic regression, we also train a ReLU network (sklearn neural network. MLPClassifier) with hidden layers of 100 units on {sw: sa) features.
neural_network.
Decision Tree. We also experiment with a decision tree with a max depth of and Gini criterion for splitting (sklearn. tree. Deciistaa/TreeClsssiiierrrr

---

Published in Transactions on Machine Learning Research (01/2026)
05.0
accuracy
accuracy
accuracy
900
09.01
Aaronson
823
Bohr
25
000
LUh
UR
65.0
Lth
UR
75-
LUh
125
entropy
entropy
entropy
54-
accuracy
accuracy
accuracy
07.
0.0
Aaronson RoBERTs
Bahri RoBERTh
in
25
Ki. (2) RoBERTa
25 LR
03
64
25
86
37
0a
03
34
35
28
57
0S
30
11
03
24
65
08
07
10
entropy
entropy
entropy
Figure 1: Detection accuracy function of average response entropy of prompts. The response entropy of each prompt is estimated using 4 non-watermarker generations and the prompis are partitioned based on 20% percentiles. For example, accuracy at entropy x is computed on the 20% of prompts with the largest entropy that is less than or equal to x. GEMMA-7B-NSTRUCC is applied to databricks- databricks-dolly- dolly-15k under Aaronson, Kirchenbauer, and Bahri watermarking schemes. MISTRAL-7BB-INSTIEE INSTRUCT generations are taken as negalives with a target length of 100. Likelihood (LLh) and RoBERTa detectors are shown. We see that watermarking performance improves with entropy, as we expect. While LLh also improves with entropy, the RoBERTa classifier is strong and also fairly flat, which the hybrid approaches successfully leverage to significantly improve watermarking in the low entropy regime. More results are in the Appendix.
3.6 Attacks
In practice, users of our model M, may act adversarially and attempt to evade detection. Following the methodology of Bahri & Wieting (2024), we study how detectability of our approaches degrades under two attack strategies random token replocement and paraphrasing, While more advanced attacks exist, both these attacks are very casy to deploy and furthermore prior work has shown simple paraphrasing to be highly effective.
While some may argue it does not make sense to evaluate this setting because, quite simply, the text we are trying to detect no longer text produced from our model per se, we include it nonetheless. We corrupt only the positive test samples (i.e. M,'s watermarked generations). leaving the negative test ones untouched. The calibration dataset used for fitting parameters or thresholds corrupted the same way as the one used for testing.
Random Token Replacement. Here, corrupt random p-percent of watermarked tokens by replacing each with random different token. is taken to be (10, 20, 30, 40). This attack strategy is cheap for the adversary to carry out, but it will significantly degrade the quality of the text.
Paraphrasing. In this attack, the adversary attemptss to evade detection by paraphrasing the watermarked text. We use Gemini-1.5-Pro (Team et al., 2024a) paraphrase each non-truncated watermarkod generation using the prompt: "Paraphrase the following: (RESPONSE)" We skip examples for which Gemini does not return response, for instance, for safety reasons.

---

Published in Transactions on Machine Learning Research (01/2026)
100
accuracy
accuracy
accuracy
05
Aaronson
25
Aaronson
25
Aaronison
25
50
100
20
206
58
100
201
256
response length
response length
response length
accuracy
accuracy
Aaranson RoBERT
25
66
Aaronsan RADAR
25
LA
LR
50
300
150
200
50
100
ate 35
response length
response length
Figure 2: Delection accuracy of the cascades and logistic regression models function of the text length T (in tokens) used for detection. The first T tokens of each text is used and two standard error bars are shaded. GEMMA-7B-INSTRUCC is applied to databricks-dolly- 15k under Aaronson and human negatives. We observe that for the watermark detector and all non-watermark detectors except the RoBERTa classifier, performance improves sharply with more test. tokens, as we expect. RoBERTa's strong performance at low token count is noteworthy and likely due the training procedure which explicitly incorporates texts of varying lengths. We find that cascades and LR combinations boost performance over either detector fairly consistently across lengths, providing assurance to the practitioner that these combinations confer benefits no matter the length of the test text.
3.7 Evaluation Criteria
Detection Performance. Our main metrics are centered solely around detection performance (as opposed to the quality of the watermarked generations) with and without adversarial corruption, as that is the focus of our work. We present two metrics.
(1) Partial ROC-AUC. We use partial ROC-AUC (pAUC) up to max FPR, as utilized in Bahri & Wieting (2024). Since cascades (IS and 28) have multiple thresholds, sweep increments based pees)ntiles), record the (FPR, TPR) pairs that result from each threshold setting, and then determine the ROC's Pareto front, which is subsequently used to calculate pAUC using the trapezoidal rule. All other methods use a single decision threshold and for them we check every test datapoint in generating the (FPR, TPR) pairs, as is the standard practice (e.g. sklearn's implementation). It crucial note that the granularity of the threshold grid search for cascades has a significant impact on the metric (especially when the region of the ROC we are interested in slim); performance should only increase the more finely grained the search is, and when every datapoint is checked, it should never underperform either detector since both cascades can represent any decision rule involving single threshold on watermark non- watermarra scores alike. In other words, any losses for 1S and 2S under this metric should be ignored.
grid of thresholds (in 1%
Our main one-number summary is PAUC with FPR < 1%. The test. datasets are always class balanced (i.e. 50/50 split), where negatives are either human responses or M,'s 'ss generations.
In the Appendix, we discuss detail how this methodology can give an unfair advantage techniques with more tuneable decision thresholds and propose alternative approach that attempts to remedy it.

---

Published in Transactions on Machine Learning Research (01/2026)
accuracy
accuracy
accurre
accuuT
15
35
35
% corrupted
% corrupted
% corrupted
% corrupted
Figure Detection accuracy of cascades and logistic regression as function of the percentage of tokens corrupted. Two standard error bars shaded. GRMMA-7B-INSTRUCT is applied to databricks-dolle 15k under different watermarking schemes using the RoBERTa classifier with human negatives and 100 tokens. We observe that while watermark detectors all degrade with more corruption, the RoBERTa classifier maintains consistently strong performance and that cascades and LR offer consistent improvements over either at all corruption levels. More results are in the Appendix.
We continue to use it, notwithstanding, as only at most two additional thresholds are being tuned (so the advantage conferred is small) and the alternative approach is not without its own disadvantages.
(2) Accuracy. We sidestep the concerns of (1) and the alternative approach discussed in the Appendix by finding the thresholds that optimize the accurucy on the calibration dataset and reporting lest set accuracy using these thresholds. Since we are always balanee, accuracy here is equivalent to the arithmetic mean of TPR and TNR. Other single scalar scores such as G-mean score (the geometric mean TPR and TNR) or F-score are also suitable, but we find no compelling reason to use them over simple accuracy, especially since the observed trends are similar.
Due to the critical role of text length, we also study performance the positive and negative samples to their first T E (25, 50, 75, 100, 150, 200, 250) tokens.
function of text length by truncating
It is important to bear in mind that random text snippets (e.g. from wikipedia) are often used as negative samples other work, whereas our negatives are model or human answers to the same prompts used to generate positive samples. The consequence is that there are far more semantic and syntactic similarities between the two classes our work, which makes our classification task intrinsically harder. Concretely, positive and negative samples here will have more overlapping n-grams, the basis for many watermarking schemes, and discrimination via the watermark will be more challenging.
Computational Performance. Our proposed cascading approach offers computational benefits over non- watermark detection alone, as watermark scoring is usually significantly cheaper than its counterpart. To that end, we report the fraction of test samples that are caught by the watermark detector (i.e. whose prediction does not depend on the non-wald non-watermark detector). If Cw, (y), Ca(y), Ce(y) represent the computational cost (c.g. FLOPs) of detection based on watermark, non-watermark and cascade for sample y, respectively, then C.(y) (w) + (1 (1--)) (a,
All standard errors reported are obtained by bootstrap resampling each class of the test set separately (500 times), to ensure that the resampled test datasets remain class-balance #2 standard errors are shown in all tables and figures. In the Appendix, we estimate the sensitivity of our models' performance on the calibration data, again via bootstrapping.
4 Experimental Results
Tables 1, 3 (Appendix), 10 (Appendix), and 11 (Appendix) show the performance of various watermarking schemes, detectors, and hybrid approaches, when GRAMA-7B-INSTRUCT is applied to databricks-dolle human text taken as the negative class, and the target length is 100 tokens. Unless indicated otherwise, results discussed under this setting. Many results are deferred to the Appendix.
10

---

Published in Transactions on Machine Learning Research (01/2026)
Standalone non-watermark detectors can outperform watermark ones. We first note that wa- termarking is not a silver bullet; in fact, we see that non-watermark detectors can outperform watermark ones in many cases, For example, under Aaronson, Binoculars and tikelihood-basee detection (LLh) achieve 93.1% and 91.7% accuracy respectively whereas watermark can only obtain 90.4%. The gains are even larger for Kuditipudi, which perforins rather poorly at 61%. Thus, we should keep in mind that sometimes simply scoring the text using log-likelihood can outperform watermark-specific scoring.
Hybrid approaches boost performance across the board. We see that hybrid detection lifts perfor- mance above either detection approach across the board. Observing We generally see marginal improvements going from one-sided to two-sided cascades (with the exception of RADAR whose baseline performance at 72.9% for Aaronson is quite low) but much larger ones moving to the learned logistic regression model. For example, LR with RoBERTa boosts Kirchenbauer (2) accuracy from 85.1% to 96.7% (11.6% gain). The same effective combination boosts Bahri from 89.9% to 98.6% (8.7% improvement).
Cascade hit rates can be substantial and improve the more trustworthy watermarking is. In Table 3 (Appendix) we see that the hit rates grow as the strength of the watermarking increases. For example, under LLh, as Kirchenbaucr's ō is cranked up 0.5 + + 3, we see 18 y inercase 0% + 8.5% 22.2%. Furthermore, the hit rates are large (i.e. the cascades prefer using the watermark scores for classification) when the non-watermark detectors are less reliable. For example, for Aaronson, 1S for Radar is 42.9% vs. 21.6% for LLh. This confirms our expectation that cascades learn to rely more on the watermarking signal the more trustworthy it is. Furthermore, we observe that hit rates for two-sided cascades are generally higher than those for one-sided cascades, and this is anticipated as it has more degrees of freedom (an additional lcarnable threshold on the watermark scores).
For Aaronson, Bahri, and Kirchenbauer, 7 generally hovers around 20-40%. 7 under Kuditipudi is very low since its watermark detector is not much better than random. If the cost of watermark detection is negligible compared to its counterpart, as it often is, then cascading improves non-watermark computational efficiency by 20-40%.
MLP and Tree generally offer gains similar to LR but risk overfitting. Among the learnable approaches, we find not much benefit in going from simple LR. to deep MLP the tree - the gains, if any, are generally small, but there can be losses as well due to potential overfitting on the calibration dataset (more parameters to learn). For example, under Aaronson, MLP improves over LR by an absolute 0.6% (96.8% vs. 97.4%) but Tree loses by 2.7% (94.1%). Broadly, the tree under the hyperparameter setting we explored was more at risk of overfitting and realizing losses than MLP. These insights all suggest that the watermark and watermaraa scores are quite discriminative on their own and do not greatly benefit from excessive learning. Thus, among the learnable options, we recommend simple LR to the practitioner. In the Appendix, we analyze the decision boundaries learned by LR for the various non- non-watermark methods.
Weaker non-watermark detectors can offer stronger performance in a hybrid setup. Interestingly, we observe that for each watermarking scheme, although the RoBERTa detector is never the best among non-watermark detectors when evaluated alone, it confers the biggest gains when leveraged in hybrid detection. For example, for Aaronson, log- likelihood (LLh) and RoBERTa achieve 91.7% and 86.0% respectively in isolation, but RoBERTa makes LR 8% better than either detector whercas LLh only provides 5.1% boost. The difference is even starker for Binoculars (93.1% alone but only a 3.7% boost). This suggests that the RoBERTa classifier provides more complementary signal than the other approaches. One factor here is its strong performance in low entropy regimes, as discussed shortly.
Stronger watermarking can hurt non-watermark detection but generally improves hybrid detection. We can observe the effect of watermarking on non-walermark detection by studying Kirchenbauer. As the strength of its watermark ó increases 0.5 + 3, watermark performance increases 58% + 90.2% as expected. However, non-watermark performance can degrade a bit; for example, I.Lh degrades 91.5% - 90.0% while RoBERTa drops more noticeably 85.9% + 82.2%. The reason for this drop is that stronger watermarks distort the generated text more, pushing away from the original language model distribution that the non-
11

---

Published in Transactions on Machine Learning Research (01/2026)
watermark detectors were either scoring with or trained under. However, despite this, the hybrid LR approach performs increasingly better with a stronger watermark; e.g. LR with RoBERTa goes 96.2% + 98.2%.
PAUC shows mostly consistent trends. Our observed trends are largely similar when pAUC (with a max FPR of 1%) replaces accuracy as our metric. For example, under Aaronson, 2S improves LLh by 17% and Binoculars by 14,, as shown Table 10.
Hybrid approaches can significantly outperform watermarking alone when available entropy is low. Figures 1 and 5 (Appendix) show the effect of entropy on performance. We see that watermarking performance indeed depends on the amount of entropy available. For example, accuracy on the 20% of prompts that afford the least. amount of response entropy (whose estimation we described earlier) is about 10% worse in absolute terms than the 20% affording the most entropy. While LLh also improves with entropy, the RoBERTa classifier is both strong and fairly flat, which the hybrid approaches successfully leverage. Concretely, the cascades and logistic regression models with RoBERT& provide a near constant performance in the high 90's across the entire entropy spectrum. Lastly, hybrid approaches post improvements at all entropy levels.
Hybrid approaches provide gains for short and long texts alike. An important research question is how the length of the text (i.c. the number of tokens observed) impacts the performance of our methods. Figure 2 studies this, and we find that for the watermark detector and all non-watermark detectors except the RoBERTa classifier, performance improves sharply with more tokens. RoBERTa's impressive performance at low token counts is likely due to its training boost performance over either detector consistently across target lengths, providing assurance that the hybrid confers benefits no matter the length of the text.
texts of various lengths. We find that cascades and LR
Paraphrasing attacks degrade non-watermark detectors but destroy watermark oncs and gains from hybrid are minimal. Tables 18 and 19 (Appendix) show accuracy under the paraphrasing allack when GRMMA-7B-NNTRUCC applied to databricks-dollyy-15 We find that paraphrasing effectively removes most of the watermarking signal, as watermark detection is near-random. As a result, the hybrid approaches rely mostly on the non-watermark signal (which alone achieves around 70-80%) and the overall performance boost is minimal. An intriguing exception is RoBERTa, where both LR and ML.Ps are able to squeese out. additional signal. For example, LR under Aaronson and RoBERIa confers an 8% gain.
Random token replacement attacks degrade watermark detectors but destroy non- non-watermark ones except for RoBERTa, Figure 3 shows the accuracy of the hybrid methods for the RoBERTa classifier deteriorating with higher levels of token corruption. We observe that the classifier is surprisingly robust to this kind of corruption despite having been trained on uncorrupted samples only and that hybrid methods offer consistent improvements at all corruption levels. Figure 6 (Appendix) reports the performance for all other detectors under Aaronson. Likelihood-basee detectors (LLh, LLR, and Binoculars) fail completely as even one bad loken can cause the likelihood of the whole LLM-generaled sequence to drop significantly, becoming even lower than that of human text. The cascades match the performance of the watermark detector here. However, LR does very well and this is an artifact of being trained on corrupted data specifically, it learns to flip the usual association and predict the negative class when the likelihood is high.
Observations carry over to when LLM generations comprise the negative class. Our insights largely carry over to the case where generations of a different LLM comprise the negative class. These results are presented in the Appendix. For instance, in Table 20 where GEMMA-7B-INSTEN databricks-dolly- 5k using MISTRAL--BB-NSTRUI
is applied to generations as negatives, we see that 2S (LR) improves LLh by au absolute 2.9% (5.2%) and RoBERTa by 1.7% (1.9%) under Aaronson.
5 Conclusion
In this work, we explore schemes for first-party Al-generated content detection that combine watermark with wonermatrmrrk detection. We observe that the two complement each other well, providing a substantial boost.
12

---

Published in Transactions on Machine Learning Research (01/2026)
in performance over either. For practitioners looking to boost performance while keeping computationally costs incurred by non-watermark detection low, recommend the two-sided cascade. For those looking for the best possible combination, we recommend the logistic regression model, which is less prone to overfitting than deep neural networks or trees but nearly equally performant. AGC detection is becoming harder but its societal implications and importance are only growing. We hope that this work will spur more interdisciplinary research on the topic and lead to effective real-world deployments.
References
Scott Aaronson. Watermarking of large language models. Large Language Models and Trunsformers Workshop Simons Institute for the Theory of Computing, 2023.
Dara Bahri and John Wieting. A watermark for black-box language models. arXiv preprint wrXiu:2410002099 2024.
Dara Bahri, Yi Tay, Che Zheng, Cliff Brunk, Donald Metzler, and Andrew Tomkins. Generative models are unsupervised predictors of page quality: A colossal-scale study. In Proceedings of the 14th ACM International Conference on Web Search and Data Mining, pp. 301-309, 2021.
Lars Buitinck, Gilles Louppe, Mathieu Blondel, Fabian Pedregosa, Andreas Mueller, Olivier Grisel, Vlad Niculae, Peter Prettenhofer, Alexandre Gramfort, Jaques Grobler, Robert Layton, Jake VanderPlas, Arnaud Joly, Brian Holt, and Gaäl Varoquaux. API design for machine learning software: experiences from the scikit-learn project. In ECML PKDD Workshop: Languages for Data Minông and Machine Learning, pp. 108-122, 2013.
Yapei Chang, Kalpesh Krishna, Amir Houmansadr, John Wieting, and Mohit lyyer. PostMark: A robust blackbox watermark for large language models. In Yaser Al-Onaizan, Mohit Bansal, and Yun-Nung Chen (eds.), Proceedings of the 2024 Conference on Empirical Methods in Natural Language Processing, pp. 8969-8987, Miami, Florida, USA, November 2024. Association for Computational Linguistics. doi: 10.18653/v 10/0555/v12000 2w00...mmlpmmmi.nn..mm
URL
Swantthoooog..oog222222222 org/2024. emmlp-main.500
Yixin Cheng, Hongcheng Guo, Yangming Li, and Leonid Sigal. Revealing weaknesses in text watermarking through self-information rewrite attacks arXiv preprint arxiv:2505. 2xX::2505500190, 2025.
Mike Conover, Matt Hayes, Ankit Mathur, Jianwei Xie, Jun Wan, Sam Shah, Ali Ghodsi, Patrick Wendell, Matei Zaharia, and Reynold Xin. Free dolly: Introducing the world's first. truly open instruction-tunet Ilm, 2023. URL https://ww. databri cwrriickkccommm wwwmblog//02////////// www.d//aaallll.oaaaoooooootoooio
dolly-first-open
Sumanth Dathathri, Abigail See, Sumedh Ghaisas, Po-Sen Huang, Rob McAdam, Johannes Welbl, Vandana Bachani, Alex Kaskasoli, Robert Stanforth, Tatiana Matejovicova, et al. Scalable watermarking for identifying large language model outputs. Nature, 634(8035):814 1888888834488 2024.
Angela Fan, Yacine Jernite, Ethan Perez, David Grangier, Jason Weston, and Michael Auli EL15: long form question answering. In Anna Korhonen, David R. Traum, and Lluís Márquez (eds.), Proceedings of the 57th Conference of the Association for Computational Linguistics, ACL 2019, Florence, Italy, July 28- August 2, 2019, Volume 1: Long Pupers, pp. 3558 3567. Association for Computational Linguistics, 2019. URL https://dot accoioorge www.oogg///0 10.g//10.110 111////11111///111111/1444
doi:
Pierre Fernandez, Antoine Chaffin, Karim Tit, Vivien Chappelier, and Teddy Furon. Three bricks to consolidate watermarks for large language modols. In 2023 IEEE International Workshop on Information Forensics and Security (WIFS), pp. 1 6. IEEE, 2023.
Sebastian Gehrmann, Hendrik Strobelt, and Alexander M Rush. Gltr: Statistical detection and visualization of generated text. arXiv preprint arXiv:1906. 040043, 2019.
Eva Giboulot and Teddy Furon. Watermax: breaking the 1lm watermark ddieetabblttyyoobstnnnssguuuu trade-off. Xiu preprint ar arXio:2408.04800 04808, 2024.
13

---

Published in Transactions on Machine Learning Research (01/2026)
Chenchen Gu, Xiang Lisa Li, Percy Liang, and Tatsunori Hashimoto. On the learnability of watermarks for language models. orXiv preprint (rXiv:3312.04460 2023.
Abhimanyu Hans, Avi Schwarzschild, Valeria Cherepanova, Hamid Kazemi, Aniruddha Saha, Micah Goldbhum, Jonas Geiping, and Tom Goldstein. Spotting Ilms with binoculars: Zero-shot detection of machine-generatel text. arXiv preprint 241.11001112700 2024.
Abe Bohan Hou, Jingyu Zhang, Tianxing He, Yichen Wang, Yung-Sug Chuang, Hongwei Wang, Lingfeng Shen, Benjamin Van Durme, Daniel Khashabi, and Yulia Tsvetkov. Semstamp: A semantic watermark with paraphrastic robustness for text generation. arXiv preprint arXiv:2310.09900 2023.
Xiaomeng Hu, Pin-Yu Yu Chen, and Tsung-Y Yi Ho. Radar: Robust ai-text detection via adversarial learning.
Advances in Newral Information Processing Systems, 36:15077-15095, 2023.
Albert Q Jiang, Alexandre Sablayrolles, Arthur Mensch, Chris Bamford, Devendra Singh Chaplot, Diego de las Casas, Florian Bressand, Gianna Lengyel, Guillaume Lample, Lucile Saulnier, et al. Mistral 7b. arXiv preprint arXiv:2310. 06825, 2023.
Nikola Jovanovié, Robin Staab, and Martin Vechev. Watermark stealing in large language models. arXiv preprint ar Xiv:2402. 19361, 2024.
Diederik P Kingma and Jimmy Ba. Adam: A method for stochastic optimization. arXiv preprint (xXi::142.69880 2014.
John Kirchenbauer, Jonas Geiping, Yuxin Wen, Jonathan Katz, Ian Miers, and Tom Goldstein. A watermark for large language models. arXiv preprint orXtt:2301.10266 2023.
Kalpesh Krishna, Yixiao Song, Marzena Karpinska, John Wieting, and Mohit lyyer. Paraphrasing evades detectors of ai-generated text, but retrieval is an effective defense. Advances in Neural Information Processing Systems, 36, 2024.
Rohith Kuditípudi, John Thickstun, Tatsunori Hashimoto, and Percy Liang. Robust distortion-free watermarks for language models. arXiv preprint or arXis:2307.15503 2023.
Tharindu Kumarage, Paras Sheth, Raha Moraffah, Joshua Garland, and Huan Liu. How reliable are ai-generated--extt detectors? assessment framework using evasive soft prompts. arXiv preprint arXiv:2310.08m 050990, 2023.
Taehyun Lee, Seokhee Hong, Jaewoo Ahm, Ilgee Hong, Hwaran Lee, Sangdoo Yun, Jamin Shin, and Gunhee Kim. Who wrote this code? watermarking for code generation. arXiv preprint 3rXin:2305.10000 2023.
Jiacheng Liang, Zian Wang, Lauren Hong, Shouling Ji, and Ting Wang. Waterpark: robustness assessment of language model watermarking. arXiv preprint 2rXie:2411.1333 131422, 2024.
Aiwei Liu, Leyi Pan, Xuming Hu, Shiao Meng, and Lijie Wen. A semantic invariant robust watermark for large language models. arXio preprint 2XXi:23100006356 2023.
Yinhan Liu, Myle Out, Naman Goyal, Jingfei Du, Mandar Joshi, Danqi Chen, Omer Levy, Mike Lewis, Luke Zettlemoyer, and Veselin Stoyanov. Roberta: A robustly optimized bert pretraining approach. arXiv preprint Xrx::11977. 1692, 2019.
Ning Lu, Shengcai Liu, Rui He, Qi Wang, Yew-Soon Ong, and Ke Tang- Large language models can be guided to evade ai-generated text detection. arXiv preprint arXiw:2305.108/ 2023.
Chengzhi Mao, Carl Vondrick, Hao Wang, and Junfeng Yang. Raidar: generative ai detection via rewriting. arXiv preprint 2rXiv:2401.12000 2024.
Niloofar Mireshghallah, Justus Mattern, Sieun Gao, Reza Shokri, and Taylor
Smaller language models are better zcro-shot machino-generated text detectors. In Proceedings of the 18th Conference of the European Chapter of the Association for Computational Linguistics (Volume 2: Short Papers), pp. 278 293, 2024.
14

---

Published in Transactions on Machine Learning Research (01/2026)
Eric Mitchell, Yoonho Lee, Alexander Khazatsky, Christopher D Manning, and Chelsea Finn. Detectgpt: Zero-shot machine-generated text detection using probability curvature. ar preprinl or 200: 11305, 2023.
Colin Raffel, Noam Shazeer, Adam Roberts, Katherine Lee, Sharan Narang, Michael Matena, Yanqi Zhou, Wei Li, and Peter J. Liu. Exploring the limits of transfer learning with a unified text-to-text transformer. arxi e-prints, 2019.
Jie Ren, Han Xu, Yiding Liu, Yinggian Cui, Shuaiqiang Wang, Dawei Yin, and Jiliang Tang. A robust. semantics-basee based watermark for large language model against paraphrasing, arXiv preprint ar Xr1:23311.,8,11, 2023.
Jenna Russell, Marzena Karpinska, and Mohit lyyer. People who frequently use ChatGPT for writing tasks are accurate and robust. detectors of Al-generated text. In Wanxiang Che, Joyce Nabende, Ekaterina Shutova, and Mohammad Taher Pilehvar (ods.), Proceedings of the 63rd Annual Meeting of the Association for Compulational Lingwistics (Volume 1: Long Pupers), pp. 5342 5373, Vienna, Austria, July 2025. Association for Computational Linguistics. ISBN 979-8-89116-251--0 doi: 10. 10.18653/v1 /2/////11222a./// acl- acl- long.267. wrw.:/ooooooorr/225555// aal-225.clllommm acl-long. 267/
URL
Trene Solaiman, Miles Brundage, Jack Clark, Amanda Askell, Ariel Herbert-Voss, Jeff Wu, Alec Radford, Gretchen Krueger, Jong Wook Kim, Sarah Kreps, et al. Release strategies and the social impacts of language models. arXiv preprint arXiv:1906 09200,, 2019.
Jinyan Su, Terry Yue Zhuo, Di Wang, and Preslav Nakov. Detectlim: Leveraging log rank information for zero-shot detection of pachine--gnnrater text. arXiv preprint arXiv:2306.055: 2023.
Yi Tay, Dara Bahri, Che Zheng, Clifford Brunk, Donald Metzler, and Andrew Tomkins. Reverse engineering configurations of neural text generation models. arXiv preprint arXiv:2044.0600 2020.
Gemini Team, Petko Georgiev, Ving lan Lei, Ryan Burnell, Libin Bai, Anmol Gulati, Garrett Tanzer, Damien Vincent, Zhufeng Pan, Shibo Wang, el al. Gemini 1.5: Unlocking multinodal understanding across millions of tokens of context. arXiv preprint arXiv:2403. 05530, 2024a.
Gemma Team, Thomas Mesnard, Cassidy Hardin, Robert Dadashi, Surya Bhupatiraju, Shreya Pathak, Laurent Sifre, Morganc Rivière, Mihir Sanjay Kalc, Julictte Love, al. Gemma: Open models based on gemini research and technology. aieie preprinl or arXiv:2403. 08295, 2024b.
Yuchuan Tian, Hanting Chen, Xulao Wang, Zheyuan Bai, Qinghua Zhang, Ruifeng Li, Chao Xu, and Yunhe Wang. Multiscale positive- unlabeled detection of ai-generated texts. arXiv preprint arXiv:2305.18149 2023.
Eduard Tulchinskii, Kristian Kuznetsov, Laida Kushnareva, Daniil Cherniavskii, Sergey Nikolenko, Evgeny Burnaev, Serguei Barannikov, and Irina Piontkovskaya. Intrinsie dimension estimation for robust detection of ai-generated texts. Advances in Neural Information Processing Systems, 36, 2024.
Vivek Verma, Eve Fleisig, Nicholas Tomlin, and Dan Klein. Ghostbuster: Detecting text ghostwritten by large language models. arXiv preprint 23.:2305.1044 2023.
Xi Yang, Kejiang Chen, Weiming Zhang, Chang Liu, Yuang Qi, Jie Zhang, Han Fang, and Nenghai Yu. Watermarking text generated by black-box language models. arXiv preprint arXiv:2305.0888. 2023a.
Xianjun Yang, Wei Cheng, Yue Wu, Linda Petzold, William Yang Wang, and Haifeng Chen. Dna-gpt: Divergent n- gram analysis for training-free free detection of gpt-generated text. arXiv preprint 2r3. 2023b.
KiYoon Yoo, Wonhyuk Ahn, Jiho Jang, and Nojun Kwak. Robust multi-bit natural language watermarking through invariant features. In Proceedings of the 61st Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), pp. 2092 21 2115, 2023.
15

---

Published in Transactions on Machine Learning Research (01/2026)
Rowan Zellers, Ari Holtzman, Hannah Rashkin, Yonatan Bisk, Ali Farhadi, Franziska Roesner, and Yejin Choi. Defending against neural fake news. Aduances in neural informalion processing systems, 32, 2019.
Hanlin Zhang, Benjamin Edelman, Danilo Francati, Daniele Venturi, Giuseppe Ateniese, and Boaz Barak. Watermarks in the sand: Impossibility of strong watermarking for generative models. arXiv preprint arXiv:231104998, 2023.
Qihui Zhang, Chujie Gao, Dongping Chen, Yue Huang, Yixin Huang, Zhenyang Sun, Shilin Zhang, Weiye Li, Zhengyan Fu, Yao Wan, et al. Lom-as-a-coauthor: Can mixed human- written and machine- -ennennee text be detected? Tn Findings of the Associalion for Compulational Linguislics: NA 2024, pp. 409 409-436, 2024.
16

---

Published in Transactions on Machine Learning Research (01/2020)
A Appendix
A.1 Omitted Discussions
Kuditipudi. We describe the distortion-free algorithm of Kuditipudi et al. (2023) in detail. Consider a secret, finite ordered list of seeds with length k. Begin watermarking by first selecting a position in the seed list uniformly at random and then apply the selection rule of Aaronson (2023) with the PRNG seeded to the current value. Advance to the next seed in the list (wrap-around if you are at the end) and repeat. Scoring is performed via a permutation test that evaluates how compatible the query text is with the specific list of seeds used during encoding versus any other random list of seeds of the same length. As the random starting position is not known during scoring, an alignment score based on the Levenshtein distance is formulated that considers alignments of various subsequenees of the text and seeds. The proposed method similar Aaronson (2023) with the difference of using a fixed list of seeds and a permutation test for scoring. The upside is robustness to attacks; the downside is significantly higher computational cost during scoring. Larger k offers more diversity and quality during generation but results in costlier and weaker detection.
ROC-AUC involving multiple thresholds. In the main text, we note that our methodology for computing ROC-AUC or pAUC favors methods with more tuneable thresholds; we now claborate further. To see this, imagine you have a large neural network and ROC-AUC is calculated by first recording (FPR, TPR) pairs on the test dataset as every combination of the network's parameters swept over. If AUC subsequently calculated by integrating under the Pareto front, then it would be misleadingly high, as you would have, in effect, fitted the network to the test dataset.
Now, consider an alternative approach. For each method, the set of thresholds {Ti} that achieve maximal TPR03-FPR on a (non-test) calibration dataset, for a uniform grid of B, is recorded; these are the thresholds corresponding to the ROC's Pareto front when it is computed on the calibration dataset. Then (FPR, TPR) pairs are computed on the test set using the afoeementioned set of thresholds (T)), and AUC / PAUC is estimated on these pairs using the trapezoidal rule. While more idcal, it has one crucial drawback: while {T:} gives (FPR, TPR) points that nicely cover the calibration ROC as the FPRs lie on a uniform grid on [0, I], this need not hold when the same thresholds are applied to the test set. In fact, we found that there can be large regions of the test ROC with little to no coverage, and this introduces unacceptably large errors in the estimation of AUC using the trapezoidal rule.
One might argue that we can just compare (FPR, TPR) pairs themselves rather than inaccurately estimated AUC, but the trouble with this that the (FPR, TPR)'s for different methods end up aligning on neither the FPR nor TPR axis, which makes direct comparisons challenging.
B Omitted Figures
Figure 4 shows the decision boundary our logistic regression model learns for a specific setting. We see that the learned model consistently puts more weight on the non-watermark detector scores than the watermark ones and this more pronounced for RoBERTa.
Figures 5 and 6 show, respectively, the effect of entropy and amount of token corruption on accuracy under the Aaronson scheme.
C Omitted Tables
Table 2 shows the standard deviation of the test accuracy sampling distribution for our hybrid models when the calibration dala is bootstrap resampled (100 times) and the test samples are held fixed. Results for each individual detector, 1S, 2S, and LR methods under Aaronson watermarking are depicted when GeMMAA-7B-NNTRUCC applied to databricks-dolly- 15k and human examples are taken as negatives, with a target length of 100 tokens. We see that the standard deviations are low, suggesting low sensitivity to the calibration data and fitting procedure.
17

---

Published in Transactions on Machine Learning Research (01/2026)
LR decision boundary
2.0
non wateemarrr z-score
LLh
Binoculars
1.5
LLR
Radar
RoBERTa
0.0
-0.5
-1.0
-2.0
-1.5
-1.0
-0.5
0.0
0.5
1.0
1.5
2.0
watermark Z-score
The decision boundary learned by logistic regression (LR) model when GEMMA-7B INSTRUCT is applied to databricks-dolly-I under the Aaronson scheme. Human responses are taken negatives and 100 lokens are used. Our LR models are trained on lop of z-score normalized watermark and non-walermark scores. The area >= x represents positíve predictions (i.e. the sample is from our model). = -x is shown for reference. By noticing that the slope magnitude less than one, we see the learned LR consistently puts more weight on the non-watermark detector scores than the watermark ones and this is more pronounced for RoBERTa.
Figure 4:
CENMRA-BB-NSTRUCT
915
05.
accuracy
accuracy
22.5
accuracy
Aaronson
+
25
05.0
Aaronson Binoculars
25
02.
85
00.0
11
10
entropy
entropy
entropy
675
accuracy
653
accuracy
63.5
Aaronson RADAR
25
Aa Arroon
853
ReqURts
635
entropy
entropy
Figure Detection accuracy of cascades and logistic regression as a function of average response entropy of prompts. GEMMA-7B-INSTHE responses are taken negatives and 100 tokens are used. Watermarking improves with entropy and hybrid methods boast gains across the board.
is applied to databricks-dollyy 15k under the Aaronson scheme. Human
18

---

Published in Transactions on Machine Learning Research (01/2026)
accuracy
accuracy
accuracy
Aaronson
25
11h
Binoculars
15
15
% corrupted
% corrupted
% corrupted
Aaronson RADAR
25
accuracy
accuracy
25
ROBERT
15
13-
10
32
15
20
33
19
% corrupted
% corrupted
Figure 6: Detection accuracy of cascades and logistic regrossion as function of the percentage of tokens corrupted. Two standard error bars are shaded. GRAMAA-7B-NNTRRCCT is applied to dalabricks-dolly- 15k under Aaronson using human negatives and 100 tokens for various non-watermark detectors. We observe strong performance from our hybrid approaches across corruption levels. Interestingly, logistic regression's 's performance improves with more corruption, for the likelihood--aase detectors. While this is counterintuitive, the explanation is that corrupting tokens of the positive samples via random flips will cause the likelihood far below those of the negative (human) samples. Since
scores of the positive samples to drop significantly the LR model is trained on corrupted data well, it learns to achieve near perfect accuracy simply by flipping its sign: high likelihood now indicates "negative" and low likelihood indicates "positive", the reverse of the non-corrupted case.
WM
Det
WM Only
Det Only
1S
2S
LR
LLh
0.150
1.169
1.022
0.924
0.598
LLR
0.150
0.627
1.079
0.839
0.842
Aaronson
RoBERTs
0.150
1.233
0.649
0.647
0.051
Binoculars
0.150
0.439
0.444
0.388
0.282
Radar
0.150
0.000
2.456
0.343
0.268
Table Standard deviation of the test accuracy sampling distribution when the calibration data bootstrap resampled 100 times and new hybrid models and decision thresholds are fit each time. Results are for each individual detector, 1S, 2S, and LR. methods under Aaronson watermarking when GEMMA-7B- -INSSRRUCT is applied to databricks-dolly 15k and human examples are taken as negatives, with a target length of 100 tokens. We find that the standard deviations are rather low, especially for logistic regression, suggesting that our findings are not too sensitive to randomness from the calibration data and fitting procedure.
Tables 3, 4, 5, 6, 7, 8, 9 report accuracy for human negatives and no corruption. Tables 10, 11, 12, 13, 14, 15, 16, 17 show pAUC for human negatives and no corruption. Tables 18 and 19 report accuracy for human negatives under the paraphrasing attack. Tables 20, 21, 22, 23 report accuracy for LLM negatives and no corruption.
19

---

Published in Transactions on Machine Learning Research (01/2026)
WM
Det
1S y
28 Y
MLP
MLP +
Tree
Tree
LLh
21.610.9
24.5+11
97.4
5.720.6
94.1
2.410.4
LLR
27.4:0.9
38.7213
95.6
5.2+1o
93.9
3.6
Aaronson
RoBERTa
24.2+1.0
24.211.0
97.8
7.5
87.0
-3.4
Binoculars
26.411.0
28.611
96.7
3.620.6
95.0
1.940.6
Radar
42.9+0.a
74.2+1.2
91.4
1.0+1.1
91.2
0.8+1.1
LLh
0.010.0
0.14a.
92.1
0.620.3
91.9
0.320.2
LLR
0.0
0.1
88.1
0.0go.4
86.9
-1.220.4
Kir. (0.5)
RoBERTa
0.0+o.9
0.1+a.1
92.4
6.5=n.r
86.3
0.4
Binoculars
0.0+o.a
0.1+0.1
93.5
0.1+0.2
93.2
-0.3
Radar
0.1
0.1
72.6
-0.3
72.9
0.0
LLh
8.5
8.620.7
93.9
2.1
94.9
3.1
LLR
16.7
16.7+1.
91.7
4.2+0.6
91.6
4.1
Kir. (2)
RoBERTa
6.6+0.r
6.7+a.r
96.4
11.3+0.9
87.1
2.0
Binoculars
12.1+o.a
12.1+o.a
95.1
2,2+0.6
94.6
1.8+0.4
Radar
16.710.a
40.611.4
82.0
1.321.5
79.4
-1.221.5
LLh
22.2+1.0
29.7+1.
96.1
5.9+0.7
93.5
3.4+o.s
LLR
33.1
51.021.
95.1
5.0
93.8
3.611.1
Kir. (3)
RoBERTa
22.0:1.0
22.111.
97.5
7.4.0.>
87.0
-3.2
Binoculars
30.5
36.3
95.7
2.7
94.6
3.5
Radar
51.420.s
77.4
90.4
0.3
89.7
-0.4
LLh 28.1
32.8
95.7
3.5
93.1
0.9
LLR
28.1
36.8119
95.4
5.5
93.8
3.9.1.1
Bahri
RoBERTa
21.4
21.421.
98.1
8.2
87.0
-2.821.2
Binoculars
26.4+1.0
26.4+10
96.2
3.0+0.6
96.4
3.2+0.a
Radar
44.5+o.s
80,1+11
89.9
-0.0+1.2
89.4
-0.4+1.2
LLh 0.610.2
0.7
92.3
1.0go.3
94.5
3.220.5
LLR
1.640.4
1.7+0.4
89.2
0.0
89.3
0.1
Kuditipudi
RoBERTa
0.640.2
0.7+0.2
93.9
6.3
87.4
-0.2
Binoculars
0.640.2
0.7
93.8
-0.2
93.9
-0.0
Radar
0.6+0.2
0.7+a.2
73.2
1.1
72.2
0.0
Table 3: Cascade hit rates and accuracies for MLP and Tree when GEMMA--B-INSTER databricks-dollg--55 15k with human negatives and 100 tokens. We see that the hit rates grow as the strength of the watermarking increases. For example, under LLh, Kirchenbauer's ő is cranked up from 0.5 + 2 3, see 1S increase 0% + 8.5% + 22.2%. Furthermore, the hit rates are large (i.e. the cascades prefer using the watermark scores for classification) when the non-watermark detectors are less reliable. For example, for Aaronson, 18 for Radar is 42.9% vs. 21.6% for LLh. We also observe that hit rates for the two-sided cascades are generally higher than those for one-sided cascade, and this is expected as has more degrees of freedom (an additional learnable threshold on the watermark scores). MLPs boost performance overall (they improve RoBERTa by 7.5% under Aaronson), though this is not always the case with Tree (3.4% drop for this same setting), and this due to overfitting on the calibration dataset.
is applied to
20

---

Published in Transactions on Machine Learning Research (01/2026)
WM
Det
WM Only
Det Only
1S
18
28
28
LR
LR +
LLh
90.3
96.6
97.9
1.320.a
98.2
1.640.5
98.5
1.920.8
LLR
90.3
93.8
96.4
2.7
96.2
2.5
97.5
3.720.7
Aaronson
RoBERTa
90.3
98.7
98.8
0.0#a.4
98.7
0.0
99.1
0.3
Binoculars
90.3
98.6
98.5
00.1
98.4
-0.2
99.1
0.5
Radar
90.3
85.8
90.4
0.1
93.0
2.7
92.5
2.2
LLh
54.5
96.4
96.5
0.0
96.5
0.0a
97.2
0.8
LLR
54.5
92.1
91.8
-0.
91.8
-0.310.2
93.0
0.81o.5
Kir. (0.5)
RoBERTa
54.5
98.5
98.5
0.0+a.0
98.5
0.0+o.8
-0.5+0.2
Binoculars
54.5
98.6
98.3
-0.3
98.3
-0,3+0.2
98.6
0.0+o.a
Radar
54.5
85.8
85.6
-0.2
85.1
-0.62..a
78.9
1
LLh
79.9
96.4
96.2
0.2
96.0
-0.4
96.6
0.3
LLR
79.9
91.1
93.1
2.1
92.6
1.640.9
95.2
4.1
Kir. (2)
RoBERTa
79.9
97.9
98.2
0.3
98.2
0.3
98.1
0.3
Binoculars
79.9
98.5
98.6
0.2+a.a
98.1
-0.4+0.4
98.9
0.4+0.4
Radar
79.9
86.0
80.0
-6. 11.5
88.1
2.1
86.3
0.3
LLh
90.7
95.0
96.9
1.9+a.7
97.3
2.3+o.r
98.6
3,6+0.7
LLR
89.0
94.7
4.0+1.1
95.3
4.6+1.1
97.1
6.4+1.0
Kir. (3)
RoBERTa
90.7
97.1
98.7
1.720.5
98.7
1.610.
98.8
1.820.5
Binoculars
90.7
97.5
98.0
0.5
98.4
0.92
98.7
1.220.3
Radar
90.7
85.8
90.7
-0.0
94.2
3.5
93.2
2.5
LLh
90.2
96.9
98.2
1.3
98.5
1.6
98.8
1.940.5
LLR
90.2
91.3
95.5
4.2
96.8
5.610.
98.5
7.3
Bahri
RoBERTa
90.2
97.7
98.4
0.7
98.8
1.140.
99.0
1.340.4
Binoculars
90.2
98.7
98.7
0.0
98.7
0.1
99.0
0.3
Radar
90.2
85.4
90.3
0.1+1.a
91.8
1.6+1.2
92.1
2.0+1.2
LLh.
97.7
96.9
-0.8
96.9
-0.810.3
96.6
-1.110.
LLR
62.1
92.7
91.8
-0.9+
91.8
-0.9
95.4
2.7+0.7
Kuditipudi
RoBERTa
62.1
98.9
98.9
-0.1
98.9
-0.1
98.9
-0.0+0.2
Binoculars
62.1
98.6
98.4
-0.3
98.4
-0.310.2
99.0
0.3
Radar
62.1
85.5
85.2
-0.3
84.1
-1.4aa.
78.7
-6.9
Table 4: Main table of accuracies when GEMMA-7B-INSTRRUCE is applied the test set of eli5 eli5-category with human negatives and 100 tokens. The trends here are similar to those for other LLMs and test datascts, with cascades and LR boasting improvements. There are sometimes anomalous losses, such as LR degrading 6.9% under Radar and Kuditípudi, which is due to overfitting since LR could achieve neutrality by zeroing out the watermark score and rely solely on the non-watermark onc.
21

---

Published in Transactions on Machine Learning Research (01/2026)
WM
Det
18 y
28 7
MLP
MLP +
Tree
Tree
LLh
24.5+1.0
48,5+16
98.5
1.9+a.e
98.2
1.++0.5
LLR
33.6+1.0
51.2+1.
96.7
2.9+a.7
96.4
2.6+0.s
Aaronson
RoBERTa
35.9+1.
55.7+14
99.4
0.6+a.a
99.5
0.8+o.3
Binoculars
35.921.0
54.321
99.1
0.5
96.5
20.5
Radar
50.1
88.620.
93.4
3.1
92.3
2.0
LLh
0.0
0.0
97.1
0.7
91.4
-5.020.8
LLR
0.0
0.0
93.6
1.420.s
91.9
-0.2
Kir. (0.5)
RoBERTa
0.0
0.0
98.6
0.0
98.3
-0.2
Binoculars
0.0
0.0
98.7
2.1
98.3
-0.310.3
Radar
1.620.4
5.820.7
84.6
-1.220.7
82.9
-2.92..5
LLh
8.910
13.811
96.6
0.3
94.8
-1.520.6
LLR
18
29.311.
95.7
4.7
91.0
-0.1
Kir. (2)
RoBERTa
12.
12.8+0.9
98.6
0.7+0.4
97.7
-0.2+0.1
Binoculars
10.3
20.7+1.2
0.1+0.4
99.0
0.5+0.5
Radar 15.8
59.5
88.9
2.9
89.7
3.620.9
LLh
27.6
50.1+1. a
98.7
3.7
98.1
3.1
LLR
35.0+1.0
59.3+1.
97.6
6.9
97.1
6.4+1.0
Kir. (3)
RoBERTa
32.1+1.0
50.1+1.
99.0
1.9+0.s
97.3
0.3
Binoculars
35.7+1.0
60.6+1.6
99.1
1.6+0.6
98.3
0.7+
Radar
49.1
83.611.1
94.5
3.
84.7
-6.021.5
LLh 30.3+1.0
48.1+1.6
2,2+0.6
95.7
-1.2+0.7
LLR
35.1+1.1
53.1+1.6
97.7
6,5+0.9
96.9
5.6+0.8
Bahri
RoBERTa 35.211.0
43.1
99.0
1.3
99.4
1.740.4
Binoculars
35.211.0
61.821a
99.4
0.7
99.3
0.620.4
Radar
47.620.9
86.6+1.
93.1
2.9
86.8
-3.3
LLh 0.7
0.7
97.2
-0.5
96.9
-0.7
LLR
2.540.4
3.1
95.3
2.6
91.4
-1.320.6
Kuditipudi
RoBERTa
0.710.2
0.7
99.0
0.1
99.2
0.34o.2
Binoculars
0.7
0.720.
98.7
0.1
97.8
-0.8
Radar 3.6+0.6
13.0+1.
85.0
-0.5+
85.4
-0.2+o.7
Table 5: Cascade hit rates and accuracies of MLP and Tree methods when GEXMA-7B-INSTRU is appliod the test set of elis-category with human negatives and 100 tokens. The trends here are similar those for other LLMs and datasets.
22

---

Published in Transactions on Machine Learning Research (01/2026)
WM
Det
WM Only
Det Only
18
+
28
28
LR
LLh
96.4
67.1
95.1
-1.2207
95.1
-1.2107
97.3
0.940.6
LLR
96.4
54.4
93.9
-2.5+a.a
97.2
0.9+a.8
95.2
-1.2+0.s
Aaronson
RoBERTa
96.4
62.1
83.6
-1.8+11
83.6
12.8+1.
79.2
-17.1+1.
Binoculars
96.4
75.0
97.6
1.2+0.6
97.6
1,2+0.6
97.3
0.9+0.7
Radar
96.4
69.1
93.8
-2.5
96.2
-0.110.7
96.8
0.4
LLh
61.0
67.4
71.5
4.1+10
71.5
4.1+1.
70.8
3.4+1.8
LLR
61.0
54.4
59.9
-1.1217
60.5
-0.5
61.5
0.411.
Kir. (0.5)
RoBERTa
61.0
66.2
4.0=0.6
4.0
76.3
14.1
Binoculars
61.0
73.4
76.7
3.3
76.7
3.3
76.8
3.4
Radar
61.0
69.7
69.8
0.1
70.6
0.9
67.5
-2.221.4
LLh
87.8
66.2
90.9
3.0+1.
94.0
6.2+a.9
95.0
7.1+0.s
LLR
87.8
52.6
84.8
-3.1
91.0
3.2
91.6
3.811.
Kir. (2)
RoBERTa
87.8
65.7
79.2
-8.721
79.2
-8.611.
83.1
-4.711.2
Binoculars
87.8
5.5+1.
6.3+1.
7.7+1.0
Radar
87.8
69.7
85.8
2.1+12
90.5
2.6+1.1
88.6
0.8
LLh
95.8
59.4
93.5
98.3
2.5
98.5
2.7
LLR
95.8
50.3
92.6
-3.220.8
97.3
1.5
97.5
1.7
Kir. (3)
RoBERTa
95.8
58.3
70.1
25.7711
95.5
-0.3
89.4
-6.410.
Binoculars
95.8
64.5
96.1
0.3+
97.7
1.9
98.4
2.6+0.6
Radar
95.8
67.1
94.7
-1.1+0.7
97.1
1.3+a.r
95.7
-0.1+o.7
LLh
97.1
0.4+0.6
1,2+o.6
97.3
0.6+o.5
LLR
96.7
55.7
93.5
-3.2+0.a
97.2
0.4+o.6
97.4
0.6+0.6
Babri
RoBERTa
16.4+1.c
-1..4+1.
-5.1
Binoculars
96.7
77.6
96.5
-0.220.
98.7
1.9
97.4
0.620.6
Radar
96.7
94.5
-23
0.4ta.
97.0
0.3
LLh
78.5
65.1
91.4
12
91.4
13.0
81.9
3.4
LLR
78.5
54.3
79.4
1.0=u.
79.6
1.1+1a
80.4
2.0+1.3
Kuditipudi
RoBERTa
78.5
66.6
77.5
-1.021
77.5
-0.941
80.4
2.021.a
Binoculars
78.5
73.3
94.1
15.621.
94.1
15.641.
86.0
7.621.2
Radar
78.5
68.8
77.7
-0.7
79.8
1.311.
79.5
1.021.3
Table 6: Main table of accuracies when MISTRAL-7B-INSTRUGE is applied to databricks-dolly- with human negatives and 100 tokens. The trends here similar those for other L.MMs and test. datasets. We sometimes observe loss for RoBERTa (e.g. 17.1% drop for LR under Aaronson). This is again due to overfitting on the calibration set. The watermakrkkkk detector performance accuracy on the test set of eli5-category) is better than on batabricks-dolly- 156klyy-5kkk-- (only 62.1%) and the model learns to put more weight on the non-watermark detector than it should. Note that this finding is partly an artifact of the evaluation procedure. While the cascades have the same loss pattern, they do not under PAUC since no
the calibration data (94.4%
calibration data used for them this case (see Table 14).
23

---

Published in Transactions on Machine Learning Research (01/2026)
WM
Det
18 Y
2S
MLP
MLP +
Tree
Tree +
LLh
43.9+0.a
44.0+a.e
97.3
0.9+0.e
97.9
1.+0.a
LLR
43.9+0.a
95.1+
96.1
-0.2+a.7
95.7
-0.7+0.7
Aaronson
RoBERTa
43.9+0.a
44.0+a.e
80.0
-16.4+10
80.2
-1..2+1.
Binoculars
43.910.6
44.020.6
97.4
1.020.7
97.8
1.520.6
Radar
43.910.8
95.110.5
96.7
0.320.7
93.9
-2.510.8
LLh
29.1
29.2
72.8
5.4110
70.6
3.2
LLR
43.3
48.8
62.3
1.221.
61.3
0.3
Kir. (0.5)
RoBERTa
0.1
0.2
74.6
12.4
74.0
11.820.8
Binoculars
22.841.1
22.9a1.1
77.1
3.620.0
77.3
3.8.0.s
Radar
3.920.5
21.9
72.8
3.120.9
69.7
0.020.0
LLh
37.01o.
72.011.a
94.7
6.820.9
90.2
2.321.1
LLR
37.010.
87.010.8
90.5
2.7
91.3
3.411.
Kir. (2)
RoBERTa
24.0+0.9
24. +0.9
81.0
-6.8+1.2
74.7
-13.1+1.2
Binoculars
37.0+0.9
81.2+1.0
95.5
7.6+1.0
94.5
6.7+1.0
Radar
37.0
87.020.s
91.1
3.2
87.2
-0.7
LLh
44.8
94.520..
98.4
2.6
97.9
2.140.4
LLR
44.8
94.52...
96.8
1.020.7
95.7
-0.1
Kir. (3)
RoBERTa
40.0+a.r
83.6+1.
79.8
-16.0+10
75.7
-20.1+1.0
Binoculars
44.8+0.a
83.4+1.0
98.3
2.5+0.6
95.6
-0.2+o.7
Radar
44.810.
96.81...
95.4
-0.410.7
95.6
-0.220.7
LLh
44.3+o.a
79.0+1.1
98.9
2.1+0.4
96.3
-0.5+0.7
LLR
44.3+0.a
90,6+0.s
97.4
0.7+0.6
93.9
+2.
Bahri
RoBERTa
37.910.
37.910.8
91.3
-5.120.<
80.2
-16.621.
Binoculars
44.520.
87.110.s
99.1
2.410.s
96.8
0.1
Radar
44.5
96. 1
97.4
0.620.6
97.3
0.520.6
LLh 25.8
28.1
92.8
14.3a1.
93.4
14.941.
LLR
25.841.0
28.111.0
83.2
4.7
78.8
0.321.
Kuditipudi
RoBERTa
24.510.
24.610.9
81.1
2.611.
74.4
-4.0a1.
Binoculars
25.840.a
25.910.8
95.3
16.821.
94.9
16.421.
Radar
28.2+0.9
92.3+
79.8
1.4+1.2
75.6
-2.9+1.s
Table Cascade hit rates and accuracies of MLP and Tree methods when MISTRAL-7B-INSTEER is appliod databricks-don dollss-ool--1150 15k with human negatives and 100 tokens. The trends here are similar to those for other LLMs and datasets.
24

---

Published in Transactions on Machine Learning Research (01/2026)
WM
Det
WM Only
Det Only
1S
1$
28
28
LR
LR +
LLh
99.1
62.9
99.4
0.3
99.8
0.7
99.9
0.810.3
LLR
99.1
54.1
99.0
-0.0
99.6
0.640
99.8
0.7
Aaronson
RoBERTa
99.1
94.4
99.4
0.3
99.8
0.7
99.9
0.810.3
Binoculars
99.1
67.9
99.8
0.7+
99.9
0.8+o.a
99.9
0.8+o.a
Radar
99.1
75.0
99.1
0.0+a.4
99.2
0.1+0.4
99.0
-0.0+0.4
LLh
63.6
59.0
66.2
2.6+1.2
66.2
2.6+1.2
62.6
-1,0+1.5
LLR
63.6
53.6
65.3
1.7+1.9
65.2
1.6+1.9
64.3
0.7#1.9
Kir. (0.5)
RoBERTa
63.6
94.7
94.5
-0.1
94.5
-0.1
97.4
2.7
Binoculars
63.6
64.5
69.1
4.6
69.1
4.6+1.1
68.6
4.1+10
Radar
63.6
73.1
75.3
2.3+a.6
75.3
2.2+0.9
72.3
-0.7+1.7
LLh
94.2
57.8
97.1
2.9+a.6
98.6
4.4+0.e
98.6
4.5+o.e
LLR
94.2
55.4
94.2
0.0+a.9
96.9
2.7+0.a
97.2
3.0+0.7
Kir. (2)
RoBERTa
94.2
92.8
97.4
3.2
98.8
4.62a.
99.7
5.5207
Binoculars
94.2
62.7
97.8
3.7
98.6
1.4107
98.6
4.520.7
Radar
94.2
73.3
93.9
-0.3
96.7
2.520.
95.0
0.8
LLh
98.9
55.4
98.9
0.0
99.7
0.8
99.9
1.020.3
LLR
98.9
53.3
98.9
0.0
98.9
-0.0
99.4
0.5
Kir. (3)
RoBERTa
98.9
86.5
99.3
0.4
99.6
0.7
99.9
1.020.3
Binoculars
98.9
57.8
99.3
0.4
99.5
0.64
99.6
0.7
Radar
98.9
70.9
98.5
-0.4+0.6
99.1
0.2+o.4
99.1
0.2+0.4
LLh
98.9
66.6
99.7
0.820.a
99.8
0.91
96.1
-2.8
L.R.
98.9
53.5
98.8
-0.0
99.5
0.7
98.8
-0.02o.
Bahri
RoBERTa
98.9
95.3
99.3
0.4+0.4
99.9
1.0+0.a
98.3
-0.6+..5
Binoculars
98.9
68.9
99.3
0.4+0.4
99.8
0.9
96.6
-2 2.2+
Radar
98.9
77.5
98.8
-0.0
99.0
0.1
98.6
-0.3
LLh
94.5
57.8
98.7
4.2+0.c
98.7
4.2+
84.5
-9.941.1
LLR
94.5
53.5
95.8
1.320.0
95.6
1.240.8
91.1
-3.341.1
Kuditipudi
RoBERTa
94.5
93.6
99.2
4.8+a.7
99.2
4.8+o.r
96.5
2.0+0.8
Binoculars
94.5
62.2
99.0
4.5+a.7
98.9
4.5+o.r
88,4
-6,0+1.2
Radar
94.5
71.5
90.7
-3.8
93.0
-1.441.0
92.8
-1.641.a
Table 8: Main table of accuracies when MISTRAL-7B-INSTROOE is applied to the test split of eli5-cate diit-categrry with human negatives and 100 tokens. The trends here are similar to those for other LLMs and datasets.
25

---

Published in Transactions on Machine Learning Research (01/2026)
WM
Det
1S
28 7
MLP
MLP +
Tree
Tree
LLh
50.4+0.a
91.8+o.
99.9
0.8+a.a
99.6
0.6+0.s
LLR
50.8+o.a
98.6+a.a
99.7
0.7+a.3
99.6
0.6+0.5
Aaronson
RoBERTa
50.4+0.2
92.0+o.7
99.9
0.8+a.a
99.7
0.6+0.3
Binoculars
49.8
91.720.
99.9
0.8
99.7
0.620.5
Radar
50.710.3
99.020.3
99.2
0.2
99.6
0.5
LLh
26.411.2
26.411.2
66.7
2.1
65.8
2.241.5
LLR
36.3
45.7
65.3
1.7
65.0
1.41199
Kir. (0.5)
RoBERTa
1.940.4
1.9
95.9
1.2
94.0
-0.7
Binoculars
17.521.0
17.5
69.7
5.2
70.0
5.5a1.a
Radar
17.521.0
18.621.
77.8
4.7
76.8
3.721.1
LLh
51.410.
77.011
98.9
2.7
98.5
4.440.6
LLR
55.210.6
93.210
96.9
2.7
98.0
3.8
Kir. (2)
RoBERTa
51.2+0.6
72.2+1.
99.4
5.2+0.7
98.4
4.2+0.7
Binoculars
50.4+0.6
77.1+1.1
4.5
98.3
4. 4.0.
Radar
55.410.6
91.820.7
97.1
2.9
97.2
3.1
LLh
51.0+0.a
95.7
99.9
1.0
99.4
0.5
LLR
51.0
96.7
99.6
0.7
99.4
0.5.0.4
Kir. (3)
RoBERTa
50.6
92.8+a.7
99.8
0.9+a.a
99.6
0.7
Binoculars
50.6+
92.8+0.7
99.7
0.8+a.a
99.4
0.5++.4
Radar
51.440.4
98.620
99.4
0.5
99.3
0.4
LLh 50.2+o.2
91.7+0.a
0.4+0.2
99.7
0.8+0.2
LLR
51.0+o.3
98,4+0.4
99.5
0.6+a.4
99.6
0.7+0.3
Bahri
RoBERTa
50.619.2
97.810.4
99.7
0.8
99.9
1.0.0.3
Binoculars
50.610.2
93.820
99.5
0.6go.3
99.7
0.8
Radar
51.02o.a
99.62
99.2
0.3
99.1
0.2
LLh 46.5
46.620.
98.6
4.1
99.0
4.5
LLR
48.910.6
58.720.
96.6
2.1
97.1
2.740.a
Kuditipudi
RoBERTa
46.010.6
46.1
99.2
4.7
99.2
4.840.7
Binoculars
46.5.0.6
46.620.6
99.1
4.6.0.7
98.8
4.340.s
Radar
48.9+0.a
71.9+1.1
94.1
-0.3+10
94.6
0.1+1.0
Table Cascade hit rates and accuracies of MLP and Tree methods when MISTRAL-7B-INSTREE is appliod the test set of elis-category with human negatives and 100 tokens, The trends here are similar those for other LLMS and datasets.
26

---

Published in Transactions on Machine Learning Research (01/2026)
WM
Det
WM Only
Det Only
18
1S +
28
28
LR
LR
+
LLh
78.9
72.3
88.4
9.6
95.9
17.0
94.4
15.5
LLR
78.9
59.8
81.2
2.3
93.2
14.3
75.4
-3.5
Aaronson
RoBERTa
78.9
92.4
96.9
4.5
97.4
5.0
96.3
3.9
Binoculars
78.9
56.5
78.8
-0.1
92.9
14.0
79.5
0.6
Radar
78.9
50.5
78.9
-0.0
82.6
3.7
80.6
1.7
LLh
50.3
71.6
71.9
0.3
76.7
5.1
72.4
0.8
LLR
50.3
59.2
59.2
0.0
63.0
59.2
Kir. (0.5)
RoBERTa
50.3
92.9
91.9
-1.0
91.9
-1.0
91.6
-1.3
Binoculars
50.3
56.4
55.8
-0.6
63.6
7.1
56.9
0.5
Radar
50.3
50.5
50.2
-0.4
51.0
0.5
51.2
0.7
LLh
60.2
67.8
70.0
2.2
85.1
17.3
81.4
13.6
LLR
60.2
56.8
61.5
1.3
78.1
17.9
60.2
-0.0
Kir. (2)
RoBERTa
60.2
92.8
93.5
0.7
93.8
93.9
Binoculars
60.2
55.4
60.5
0.3
80.7
20.5
63.2
3.0
Radar
60.2
50.5
60.3
0.1
63.0
61.6
LLh
77.7
62.0
80.1
2.5
93.9
16.3
89.7
12.0
LLR
77.7
54.0
77.2
-0.4
90.7
13.0
63.5
-14.2
Kir. (3)
RoBERTa
77.7
91.8
96.1
4.3
96.7
4.9
96.2
4.4
Binoculars
77.7
53.6
74.7
-2.9
91.2
13.5
74.2
-3.5
Radar
77.7
50.5
77.7
0.0
80.4
2.8
78.6
0.9
LLh
79.6
71.9
87.0
7.4
95.5
15.9
87.2
7.6
LLR
79.6
59.0
80.8
1.2
92.1
12.5
67.3
-12.3
Bahri
RoBERTa
79.6
92.9
97.3
4.4
97.4
4.5
94.3
1.4
Binoculars
79.6
56.2
79.3
-0.3
93.5
13.9
71.8
-7.7
Radar
79.6
50.5
79.7
0.1
83.0
3.4
76.6
-3.0
LLh
52.1
71.5
71.8
0.3
81.8
10.3
75.6
4.1
LLR
52.1
59.2
59.2
0.0
68.5
9.4
60.3
1.1
Kuditipudi
RoBERTa
52.1
93.4
92.7
-0.7
93.0
-0.4
91.0
-2.5
Binoculars
52.1
56.3
56.2
-0.1
66.5
10.2
58.0
1.7
Radar
52.1
50.5
52.1
-0.0
53.1
52.3
Table 10: Main table of pAUC results (1% max FPR)
when GEMNAA-7B-NSTRUCT is applied to databricks- dolly- 15k and human examples are used negalives at target length of 100 tokens.
27

---

Published in Transactions on Machine Learning Research (01/2026)
WM
Det
MLP
MLP
Tree
Tree +
LLh
93.0
14.1
50.4
-28.5
LLR
77.3
-1.6
50.1
-28.8
Aaronson
RoBERTa
67.8
-24.5
50.7
-41.7
Binoculars
69.3
-9.6
51.9
-27.0
Radar
81.8
2.9
66.1
-12.8
LLh
70.8
-0.8
53.1
-18.5
59.2
54.5
-4.6
Kir. (0.5)
RoBERTa
65.0
-27.9
50.7
-42.2
Binoculars
55.5
-0.9
53.7
-2.8
Radar
51.1
0.6
50.6
0.1
LLh
79.2
11.4
52.3
-15.5
LLR
63.1
2.9
52.5
-7.7
Kir. (2)
RoBERTa
64.2
-28.6
50.7
-42.1
Binoculars
58.7
-1.5
53.3
-6.9
Radar
62.3
52.3
-7.9
LLh
89.1
11.5
49.9
-27.8
LLR
66.0
-11.7
55.6
-22.0
Kir. (3)
RoBERTa
76.0
-15.8
50.7
-41.1
Binoculars
66.9
-10.8
50.0
-27.6
Radar
79.1
1.5
55.8
-21.9
LLh
83.8
4.2
50.6
-9.00
L.R.
63.4
-16.2
50.1
-29.5
Bahri
RoBERTa
77.4
-15.5
50.7
-42.2
Binoculars
64.8
-14.7
57.0
-22.6
Radar
79.9
0.3
59.2
-20.4
LLh
76.7
5.2
49.7
-21.7
LLR
62.2
3.1
51.8
-7.4
Kuditipudi
RoBERTa
59.3
-34.1
50.8
-42.7
Binoculars
56.5
0.2
53.7
-2.6
Radar 52.3
0.2
50.8
-1.3
Table 11: pAUC numbers (1% max FPR) of MLP and Tree methods when GEXMA-7B-INSTRICT is applied to dalabricks-dolly 15k and human examples are used as negatives at target length of 100 tokens.
28

---

Published in Transactions on Machine Learning Research (01/2026)
WM
Det
WM Only
Det Only
18
18 +
28
28
LR
LR
+
LLh
79.3
98.8
99.4
0.6
99.7
1.0
99.9
1.1
LLR
79.3
92.2
97.4
5.1
98.6
6.3
97.8
5.5
Aaronson
RoBERTa
79.3
100.0
99.9
-0.1
99.7
-0.3
99.4
-0.6
Binoculars
79.3
98.8
99.2
0.3
99.3
0.5
99.3
0.5
Radar
79.3
51.7
79.4
0.1
86.3
7.0
82.7
3.4
LLh
50.4
98.9
97.9
-1.0
88.3
-10.7
98.8
-0.2
LLR
50.4
92.1
90.2
-1.9
91.2
91.0
Kir. (0.5)
RoBERTa
50.4
100.0
99.6
-0.3
77.7
-22.2
98.3
-1.7
Binoculars
50.4
98.9
97.7
-1.2
89.5
-9.4
98.6
-0.3
Radar
50.4
51.7
50.4
-1.3
53.0
1.2
51.8
0.1
LLh
59.7
98.4
98.2
-0.1
98.9
0.5
98.8
0.4
LLR
59.7
89.5
90.6
1.1
92.9
3.4
91.9
2.3
Kir. (2)
RoBERTa
59.7
99.9
99.7
-0.2
97.3
Binoculars
59.7
98.4
98.4
-0.0
98.5
0.1
97.7
-0.7
Radar
59.7
51.8
59.9
0.1
67.8
62.3
LLh
78.3
96.8
98.9
2.1
99.3
2.5
99.5
2.7
LLR
78.3
84.1
93.5
9.3
96.4
12.3
96.3
12.1
Kir. (3)
RoBERTa
78.3
99.9
99.9
-0.0
99.5
-0.4
98.9
-1.0
Binoculars
78.3
96.8
98.6
1.9
98.8
2.0
98.3
1.6
Radar
78.3
51.7
78.6
0.3
84.8
6.5
80.5
2.2
LLh
79.6
99.1
99.2
0.1
99.7
0.7
99.4
0.4
LLR
79.6
91.6
97.5
5.9
98.8
7.2
97.5
Bahri
RoBERTa
79.6
100.0
99.9
-0.1
99.5
-0.5
99.3
-0.6
Binoculars
79.6
98.6
99.3
0.7
99.6
99.3
Radar
79.6
51.6
79.8
0.2
85.0
5.4
84.0
4.4
LLh
51.2
98.9
97.4
-1.5
94.2
-4.8
99.1
0.1
LLR
51.2
93.0
90.7
-2.3
93.1
0.1
98.6
0.7
Kuditipudi
RoBERTa
51.2
100.0
99.6
-0.4
79.8
-20.2
99.0
-1.0
Binoculars
51.2
98.9
97.6
-1.2
93.0
-5.8
98.9
0.0
Radar
51.2
51.7
51.1
-0.6
53.9
Table 12: Main table of pAUC results (1% max FPR) when GEMMA-7B-INTRU is applied to the test set of eli5-calegory and human examples are used as negatives at target length of 100 tokens.
29

---

Published in Transactions on Machine Learning Research (01/2026)
WM
Det
MLP
MLP
Tree
Tree +
LLh
99.9
1.1
98.3
-0.5
LLR
99.1
6.9
95.8
3.6
Aaronson
RoBERTa
99.5
-0.5
99.5
-0.5
Binoculars
99.5
0.7
92.3
-6.5
Radar
85.5
6.2
57.1
-22.2
LLh
98.8
-0.1
97.0
-1.9
91.5
-0.6
90.6
-1.5
Kir. (0.5)
RoBERTa
99.5
-0.5
99.9
-0.1
Binoculars
98.7
-0.1
98.5
-0.4
Radar
53.4
1.6
53.0
1.3
LLh
98.7
0.4
96.8
-1.6
LLR
93.3
3.8
88.8
-0.7
Kir. (2)
RoBERTa
98.6
-1.3
99.7
-0.3
Binoculars
98.0
-0.4
93.7
-4.7
Radar
65.3
5.5
57.8
-2.0
LLh
99.6
2.8
97.4
0.6
LLR
97.7
13.5
93.7
9.5
Kir. (3)
RoBERTa
98.9
-1.0
99.2
-0.7
Binoculars
98.7
1.9
95.3
-1.4
Radar
82.0
3.7
74.2
-4.1
LLh
99.5
0.4
97.7
-13
L.R.
97.3
5.7
94.0
Bahri
RoBERTa
99.2
-0.8
99.4
-0.6
Binoculars
99.3
0.7
95.8
-2.8
Radar
83.9
4.3
70.3
-9.3
LLh 99.1
0.1
97.8
-1.1
LLR
93.0
0.0
85.3
-7.7
Kuditipudi
RoBERTa 99.5
-0.5
99.9
-0.1
Binoculars
98.9
0.0
98.6
-0.3
Radar
54.0
2.3
51.7
Table 13: pAUC (1% max FPR) of MLP and Tree methods when GEMMA-7B CEMMA-7B-NNSTRUCT is applied to the test set of eli5-calegory and human examples are used negatives at target length of 100 tokens.
30

---

Published in Transactions on Machine Learning Research (01/2026)
WM
Det
WM Only
Det Only
18
18 +
28
28
LR
LR
+
LLh
96.2
50.8
94.1
-2.1
98.7
2.5
93.3
-2.9
LLR
96.2
50.2
94.2
-2.0
97.1
0.9
86.6
-9.7
Aaronson
RoBERTa
96.2
76.5
97.3
1.1
98.4
2.2
97.1
0.9
Binoculars
96.2
56.2
96.4
0.2
98.5
2.3
94.2
-2.0
Radar
96.2
50.5
96.2
-0.0
96.6
0.4
96.3
0.1
LLh
51.0
50.5
50.8
-0.3
51.4
0.4
51.1
0.1
LLR
51.0
50.3
50.7
-0.3
51.1
50.7
-0.3
Kir. (0.5)
RoBERTa
51.0
76.1
73.9
-2.2
76.4
0.3
53.0
23.11
Binoculars
51.0
55.3
54.2
-1.2
56.8
1.4
56.6
1.3
Radar
51.0
50.5
51.0
-0.1
51.7
0.7
51.7
0.7
LLh
81.6
50.2
79.7
-1.9
85.1
3.5
80.1
-1.5
LLR
81.6
50.1
79.2
-2.4
83.2
1.6
76.7
-4.9
Kir. (2)
RoBERTa
81.6
72.4
87.9
6.3
91.6
10.0
84.4
Binoculars
81.6
52.7
80.2
-1.3
88.6
7.0
88.4
6.8
Radar
81.6
50.5
81.7
0.2
84.9
82.9
LLh
96.4
50.1
95.1
-1.2
97.7
1.3
95.4
-1.0
LLR
96.4
50.1
95.1
-1.2
97.0
0.6
94.6
-1.7
Kir. (3)
RoBERTa
96.4
68.7
97.0
0.6
98.4
2.1
96.2
-0.1
Binoculars
96.4
50.9
95.3
-1.1
98.1
1.8
98.1
1.7
Radar
96.4
50.4
96.4
0.1
97-0
0.7
96.9
0.5
LLh
97.0
50.7
96.1
-0.9
99.2
2.2
91.0
-6.0
LLR
97.0
50.1
96.0
-1.0
97.8
94.5
-2.5
Bahri
RoBERTa
97.0
70.5
97.2
0.2
98.6
1.6
96.8
-0.2
Binoculars
97.0
56.6
97.0
0.1
99.0
91.5
-5.5
Radar
97.0
50.4
96.9
-0.1
97.3
0.3
95.5
-1.5
LLh
77.4
50.5
76.5
-0.9
81.0
3.6
64.7
-12.7
LLR
77.4
50.3
76.3
-1.1
78.2
0.8
54.5
-22.9
Kuditipudi
RoBERTa
77.4
75.6
87.3
9.9
87.8
10.4
78.4
1.0
Binoculars
77.4
55.3
79.5
2.1
84.0
6.6
69.0
-8.4
Radar
77.4
50.5
77.3
-0.1
78.0
75.7
Table 14: Main table of pAUC results (1% max FPR) when MISTRAL-7B-INSTRRO is applied to databricks- dolly- 15k and human examples are used negalives at target length of 100 tokens.
31

---

Published in Transactions on Machine Learning Research (01/2026)
WM
Det
MLP
MLP
Tree
Tree +
LLh
89.3
-6.9
54.4
-41.8
LLR
91.1
-5.1
55.9
-40.3
Aaronson
RoBERTa
97.2
1.0
49.8
-46.4
Binoculars
94.0
-2.2
56.7
-39.5
Radar
96.3
0.1
93.9
-2.3
LLh
50.5
-0.5
50.8
-0.2
50.3
-0.7
50.1
-0.9
Kir. (0.5)
RoBERTa
50.0
-26.1
50.2
-25.8
Binoculars
55.4
0.1
52.1
-3.2
Radar
52.2
1.1
50.9
-0.1
LLh
63.0
-18.6
80.3
-1.3
LLR
76.4
-5.2
79.9
-1.7
Kir. (2)
RoBERTa
84.7
3.1
49.8
-31.8
Binoculars
76.2
-5.4
78.0
-3.6
Radar
84.5
2.9
79.3
-2.3
LLh
92.7
-3.6
81.3
15.1
LLR
93.5
-2.8
85.3
-11.1
Kir. (3)
RoBERTa
94.5
-1.9
50.2
-46.1
Binoculars
97.3
0.9
55.8
-0.5
Radar
97.1
0.7
96.7
0.3
LLh
82.1
-14.8
50.4
-46.5
L.R.
86.5
-10.4
50.1
-46.9
Bahri
RoBERTa
54.4
-42.6
50.4
-46.6
Binoculars
88.2
-8.8
50.9
-46.0
Radar
96.1
-0.9
96.9
-0.0
LLh
55.0
-22.4
74.4
-3.0
LLR
59.7
-17.7
73.7
-3.7
Kuditipudi
RoBERTa
51.4
-26.0
50.2
-27.2
Binoculars
58.2
-19.2
55.3
-22.1
Radar
77.4
0.0
77.2
-0.2
Table 15: PAUC (1% max FPR) of MLP and Tree methods when MISTRAL-TB-NSTRRCTT is applied to databricks-dolly-- 15k and human examples are used as negatives at target length of 100 tokens.
32

---

Published in Transactions on Machine Learning Research (01/2026)
WM
Det
WM Only
Det Only
18
1S +
28
25 +
LR
LR +
LLh
99.8
61.0
100.0
0.2
100.0
0.2
100.0
0.2
LLR
99.8
52.5
99.7
-0.1
99.9
0.1
99.9
0.1
Aaronson
RoBERTa
99.8
98.7
100.0
0.2
100.0
0.2
100.0
0.2
Binoculars
99.8
64.3
100.0
0.2
100.0
0.2
100.0
0.2
Radar
99.8
51.0
99.6
-0.1
99.8
0.0
99.8
0.0
LLh
51.2
57.5
56.2
-1.3
57.6
0.1
53.9
-3.6
52.4
-0.3
52.6
-0.0
51.5
-1.2
Kir. (0.5)
RoBERTa
51.2
98.5
98.0
-0.5
90.5
-8.0
98.1
-0.5
Binoculars
51.2
61.1
60.4
-0.8
60.4
-0.7
59.2
1..9
Radar
51.2
50.9
51.2
-0.0
52.8
1.6
52.1
0.8
LLh
95.1
56.3
96.5
1.3
97.6
2.4
96.5
1.4
LLR
95.1
52.6
94.1
-1.0
96.4
1.2
95.8
0.7
Kir. (2)
RoBERTa
1.5
99.9
1.5
99.8
1.4
Binoculars
95.1
59.3
96.9
1.7
97.8
2.7
97.2
2.0
Radar
95.1
50.9
95.1
-0.0
96.6
1.4
95.8
0.7
LLh
99.7
53.9
99.8
0.1
99.9
0.3
100.0
0.3
LLR
99.7
51.6
99.2
-0.4
99.8
0.1
99.8
0.1
Kir. (3)
RoBERTa
99.7
95.1
100.0
0.3
100.0
0.3
100.0
0.3
Binoculars
99.7
54.6
99.7
0.0
99.9
0.2
99.8
0.2
Radar
99.7
50.8
99.5
-0.1
99.7
0.0
99.7
0.1
LLh
99.7
64.9
100.0
0.3
100.0
0.3
83.8
-15.9
L.R.
99.7
52.0
99.0
-0.7
99.8
0.1
78.6
-21.1
Bahri
RoBERTa
99.7
99.1
100.0
0.3
100.0
0.3
100.0
0.3
Binoculars
99.7
99.8
0.1
100.0
0.3
83.0
-16.7
Radar
99.7
50.9
99.6
-0.2
99.7
-0.0
98.3
-1.4
LLh
95.4
56.5
98.9
3.5
98.9
3.5
63.1
-32.3
LLR
95.4
52.0
96.2
0.8
96.5
1.1
58.8
-36.6
Kuditipudi
RoBERTa
95.4
98.2
99.9
99.2
1.0
99.9
Binoculars
95.4
59.7
99.4
4.0
99.3
3.9
70.5
-24.9
Radar
95.4
95.4
-0.0
95.6
0.2
85.4
-10.0
Table 16: Main table of pAUC results (1% max FPR) when MISTRAL-7B-INSTRUC is applied to the test split of eliō-calegory and human examples are used as negatives at target length of 100 tokens.
33

---

Published in Transactions on Machine Learning Research (01/2026)
WM
Det
MLP
MLP +
Tree
Tree
LLh 100.0
0.2
94.8
-4.9
LLR
99.9
0.1
94.8
-5.0
Aaronson
RoBERTa
99.9
0.2
94.9
-4.9
Binoculars
99.9
0.2
94.9
-4.9
Radar
99.8
0.1
94.8
-5.0
LLh
54.3
-3.2
55.8
-1.7
52.2
-0.5
50.7
-2.0
Kir. (0.5)
RoBERTa
97.1
-1.4
97.1
-1.4
Binoculars
60.1
-1.1
60.2
-1.0
Radar
53.2
2.0
52.4
1.2
LLh
97.2
2.1
83.2
-12.0
LLR
95.6
0.5
83.2
-12.0
Kir. (2)
RoBERTa
99.6
1.2
98.8
0.4
Binoculars
97.7
2.6
83.2
-11.9
Radar
96.5
1.3
83.2
-1..9
LLh
99.9
0.3
92.7
-7.0
LLR
99.8
0.1
92.7
-7.0
Kir. (3)
RoBERTa
99.9
0.2
92.8
-6.9
Binoculars
99.9
0.2
92.7
-6.9
Radar
99.8
0.1
92.7
-6.9
LLh
90.3
-9.4
91.8
-7.9
LLR
97.4
-2.3
91.8
-7.9
Bahri
RoBERTa
100.0
0.3
86.1
-13.6
Binoculars
97.1
-2.6
91.8
-7.9
Radar
97.7
-2.0
87.8
-11.9
LLh 87.5
-7.9
93.3
-2.1
LLR 87.8
-7.6
92.5
-2.8
Kuditipudi
RoBERTa 99.8
1.6
98.3
0.1
Binoculars
94.8
-0.6
93.5
-1.9
Radar 90.3
-5.1
92.6
-2.8
Table 17: pAUC (1% max FPR) of MLP and Tree methods when MISTRAL-7B- is applied to the test set of eli5-calegory and human examples are used negalives at target length of 100 tokens.
34

---

Published in Transactions on Machine Learning Research (01/2026)
WM
Det
WM Only
Det Only
1S
18
28
28
LR
LR +
LLh
50.0
74.4
74.7
0.3
74.7
0.3
74.4
-0.010.
LLR
50.0
71.7
71.7
0.0+a.1
71.7
0.0+o.1
71.7
0.0+0.0
Aaronson
RoBERTa
50.0
71.1
71.3
0.3+0.2
71.3
0.3+0.2
79.1
8,0+1.4
Binoculars
50.0
77.5
77.6
0.1
77.6
0.1
77.5
0.0
Radar
50.0
65.4
65.4
0.0
65.4
0.0
65.1
-0.3
LLh
51.5
73.9
1.2+a.a
75.2
1.3
1.2
LLR
51.5
72.3
72.2
-0. 00.1
72.2
+0.1
74.1
1.8
Kir. (0.5)
RoBERTa
51.5
70.6
70.4
+0.2
70.5
-0.2
79.1
8.4+1.4
Binoculars
51.5
77.1
77.2
0.1
77.3
0.2
77.3
0.2
Radar
51.5
65.9
65.9
0.0
65.9
0.0
65.8
-0.1
LLh
52.5
75.8
73.9
-1.9 -1.920.6
73.9
-1.810.5
74.8
-1.010.3
LLR
52.5
71.7
71.5
-0.2
71.5
-0.210.2
73.5
1.810.8
Kir. (2)
RoBERTa
52.5
71.3
71.3
-0.0
71.3
-0.0
79.7
8.3+1.4
Binoculars
52.5
77.0
77.1
0.1
77.2
0.2+0.4
77.1
0.1
Radar
52.5
65.7
65.7
0.0
65.7
0.0
65.6
-0.1
LLh
54.5
76.1
76.1
0.0
76.2
0.1
76.4
0.2
LLR
54.5
71.4
71.4
0.0
71.4
0.1
73.0
1.620.6
Kir. (3)
RoBERTa
54.5
72.1
72.3
0.2+a.2
72.3
0.3+0.2
79.9
7.9+1.8
Binoculars
54.5
76.6
76.3
-0.3+0.4
76.3
+0.3+a.4
76.6
0.0+o.5
Radar
54.5
65.3
65.3
0.0
65.3
0.0
65.9
0.620.7
LLh
50.4
74.5
74.4
-0.1+0.1
74.4
-0.1+0.1
75.7
1,2+0.5
LLR
50.4
71.8
71.7
-0, 0.1
71.7
-0.1
72.1
0.3+0.2
Bahri
RoBERTs
50.4
71.0
72.2
1.2+0.4
72.2
1.2
79.3
8.3
Binoculars
50.4
76.8
76.8
0.0
76.8
0.0
77.0
0.2
Radar
50.4
65.0
65.0
0.02a.o
65.0
0.0
65.1
0.0ao.0
LLh
52.5
76.6
76.8
0.2+a.2
76.8
0.2+a.2
77.1
0.5+0.4
LLR
52.5
72.0
72.1
0.2+a.2
72.1
0.2+a.2
73.6
1.7+0.6
Kuditipudi
RoBERTa
52.5
71.4
71.5
0.1
71.5
0.110.2
78.9
7.521.4
Binoculars
52.5
78.3
78.4
0.110.1
78.4
0.110.
77.9
-0.540.5
Radar
52.5
65.5
65.5
0.0+a.1
65.1
-0.4+0.6
66.0
0.5+0.7
Table 18: Main table of accuracies under the paraphrasing attack. GEMMA-7B-NNSTRUCH is applied databricks-dolly-E and human examples are used as negatives at 100 tokens. We observe that paraphrasing, attack type known to be, a priori, challenging to defend against, does effectively remove most watermarking signal, as walermark detection is near random. As result, the hybrid approaches rely mostly on the non non-watermark signal and the overall performance improvement is minimal. An intriguing exception is RoBERTa, where both LR and MIPs are able to juice out additional signal: LR under Aaronson and RoBERTa confers an 8% gain.
35

---

Published in Transactions on Machine Learning Research (01/2026)
WM
Det
18 y
28 7
MLP
MLP
Tree
Tree
LLh
0.1
0.140.1
76,5
2.1
77.8
3.4
LLR
0.1
0.1
72.0
0.3
2.4
Aaronson
RoBERTa
0.1
0.1
79.0
8.0=1.
71.0
-0.0
Binoculars
0.1
0.1
77.0
77.0
-0.520.9
Radar
0.1
0.1
65.0
-
65.8
0.4
LLh
0.0+o.0
0.1+0.1
75.0
1.1+a.4
77.6
3.7+o.s
LLR
0.020.0
0.1
73.3
1.020.4
72.3
0.0
Kir. (0.5)
RoBERTa
0.0
0.1
77.7
7.021a
70.4
-0.3
Binoculars
0.0
0.1+0.1
76.6
-0.5
77.3
0.2+0.4
Radar
0.1
0.1+0.1
65.7
-0.2+0.2
65.9
0.0
LLh
0.0
0.1
75.4
00.4
75.8
0.0
LLR
0.0
0.1
72.7
1.0
74.9
3.3
Kir. (2)
RoBERTa
0.0
0.1
79.3
8.0
72.1
0.8
Binoculars
0.0+o.0
0.1+0.1
76.9
-0.1+0.4
77.2
0.2+0.4
Radar
0.0+o.1
0.1+o.1
65.7
-0.0+0.4
66.6
0.9+
LLh
0.0+0.0
0.1+0.1
75.9
-0.3+
77.0
0.9+0.8
LLR
0.0+0.0
0.1+0.1
73.0
1.7+0.c
2.7+0.s
Kir. (3)
RoBERTa
0.0+0.0
0.1+0.1
79.7
7.6=1a
73.0
0.9
Binoculars
0.0
0.1
76.3
-0.320.4
76.6
-0.0
Radar
0.0
1.1
65.4
0.1
64.7
-0.6.o.e
LLh
0.0
0.0+a.0
75.7
1.1+0.a
78.1
3.6
LLR
0.0+0.0
0.0+a.0
71.4
-0.3
75.1
3.4+0.7
Bahri
RoBERTa
0.020.
0.0
78.1
7.1ana
71.3
0.3
Binoculars
0.020.0
0.010.0
76.8
0.02a.4
76.8
0.0
Radar
0.020.e
0.0
65.1
0.1
64.5
-0.520.5
LLh
0.0.0.0
0.1
76.6
0.0
78.2
1.620.a
LLR
0.020.0
0.1
72.4
0.4
75.1
3.411a
Kuditipudi
RoBERTa
0.040.0
0.1
78.7
7.4
71.7
0.340.2
Binoculars
0.040.0
0.1
78.3
-0.0
78.3
0.040.0
Radar
0.1+0.1
2.5+0.4
65.8
0.4+a.s
64.8
-0.6+0.a
Table 19: Cascade hit rates and accuracies of MLP and Tree methods when GEMMA-7B-INSTRICCE is applied
to kalabricks-dolye 15k under the paraphrasing attack (human negatives, 100 tokens). Since watermark
performance is essentially random, the cascades learn to ignore it and rely solely on non-waternark signal. As a result, hit rates are near zero.
36

---

Published in Transactions on Machine Learning Research (01/2026)
WM
Det
WM Only
Det Only
1S
28
28 +
LR
LLh
89.9
83.7
90.4
0.6a0
92.8
2.9
95.1
5.220.6
Aaronson
LLR
89.9
78.5
89.8
-0.1
90.5
0.62o.8
94.1
4.220.7
RoBERTa
89.9
96.0
97.4
1.5+0.3
97.6
1.7+0.3
97.9
1.9+0.4
LLh
59.5
84.7
85.1
0.4
85.1
a.4
85.7
1.0g0.3
Kir. (0.5)
LLR
59.5
78,5
78.7
0.2+o.1
78.7
0.2+
78.9
0.4+0.3
RoBERTa
59.5
96.0
-0.3+
95.7
-0.3+0.1
95.6
-0.4
LLh
81.1
82.8
83.6
0.8
85.3
2.6
91.1
8.420.6
Kir. (2)
LLR
81.1
75.8
81.3
0.1+1.0
83.5
1.3
87.9
6.7+1.0
RoBERTa
81.1
95.5
96.0
0.4
96.0
0.4
96.1
0.6#o.4
LLh
90.3
77.9
88.6
-1.7
92.2
1.9
94.6
4.3
Kir. (3)
LLR
90.3
72.7
83.0
-7.340.s
91.1
0..
91.4
1.1
RoBERTa
90.3
94.2
97.1
3.0+0.4
97.2
3.1+
97.6
3.4+0.5
LLh
89.6
83.9
89.0
-0.610.
91.1
1.5
94.0
4.4g0.7
Bahri
LLR
89.6
76.0
87.6
-2.0
91.6
2.02o.s
92.3
2.720.7
RoBERTa
89.6
95.6
96.9
1.3+0.a
97.5
1.8+0.a
98.2
2.6+0.4
LLh
58.8
83.7
85.0
1.3
85.1
1.3
86.0
2.220.3
Kuditipudi
LLR
58.8
78.5
79.3
0.8+0.2
79.3
0.8
80.7
2.2+0.3
RoBERTa 58.8
96.3
96.0
-0.2+0.2
96.0
-0.2
96.4
0.2+0.3
Table 20: Main table of accuracies when GENMA--B-NNSTRUCC is applied to databricks-dolly- and
MISTRAL--B-INSTRUCE generations are used as negatives at a target length of 100 tokens. The trends here are similar to those for human negatives.
WM
Det
18 Y
2S ?
MLP
MLP
Tree
Tree +
I.Lh
29.4
44.2
94.3
4.510
91.2
1.320.8
Aaronson
LLR
33.1+0.7
49.2+1.0
94.6
4.8+0.7
91.2
1.4-0.8
RoBERTa
25.5
35.7+0.9
98.2
2.2+0.a
97.1
1.1-0.4
LLh
0.020.a
0.1
86.0
1.310
83.9
-0.8203
Kir. (0.5)
LLR
0.0
0.1
78.7
0.240.4
77.0
-1.22..3
RoBERTa
0.0
0.1
96.1
0.1
94.9
-1.120.4
LLh
11.7
18.620.7
89.9
7.140.5
90.0
7.3
Kir. (2)
LLR
18.7
26.7
87.6
6.511.
84.7
3.6
RoBERTa
11.7.0.6
11.8
96.6
1.140.
96.8
1.2
L.Lh
36.21o.7
59.7
94.7
4.440.
91.4
1.1
Kir.
L.R.R
34.7
63.5
92.7
2.410
91.6
1.320.
RoBERTa
32.4+0.7
56.0+1.0
97.9
3.7+0.4
97.3
3.1=0.4
LLh 29.620.7
10.9
94.9
5.310.
92.1
2.520.7
Bahri
LLR
31.5
55.3
93.3
3.7
90.6
1.020.s
RoBERTa
23.1
39.02o.o
98.4
2.740.a
97.5
1.8
LLh
0.5
0.6+a.
86.2
2.5+0.4
84.3
0.5
Kuditipudi
LLR
1.520.a
1.540.2
80.1
1.640.3
82.9
4.5
RoBERTa
0.5ao.1
0.640.
96.1
-0.1
96.2
-0.02o.0
Table 21: Cascade hit rates and accuracies of MLP and Tree methods when GEMMA-7B-INSTRUCH is applicd generations are used as negatives at a larget length of
to dalabricks-dolee dolls-ddll--111 15k and MISTRAL--B-INSTII 100 tokens. The trends here are similar to those for human negatives.
37

---

Published in Transactions on Machine Learning Research (01/2026)
WM
Det
WM Only
Det Only
1S
28
28 +
LR
LLh
90.9
90.8
95.7
4.8
96.1
5.2
97.8
6.92o.5
Aaronson
LLR
90.9
84.5
98.7
2.8
94.4
3.5
96.5
5.620.7
RoBERTa
90.9
97.9
98.8
0.8+o.2
98.8
0.8+o.3
98.9
0.9+o.s
LLh
56.9
90.4
89.9
-0.540.
89.9
-0.5go.1
90.7
0.320.3
Kir. (0.5)
LLR
56.9
83.9
83.9
-0.0+o.a
83,9
-0.0+o.o
84.7
0.7+0.4
RoBERTa
56.9
97.8
97.8
0.0+o.e
97.8
0.0+o.0
97.0
-0.8
LLh
81.6
88.7
91.0
2.3
92.4
3.7
94.8
6.2
Kir. (2)
LLR
81.6
86.8
88.9
2.1
88.5
1.7+0.s
91.6
4.8
RoBERTa
81.6
97.4
97.8
0.3
97.8
0.3
97.4
0.0
LLh
92.0
88.6
94.5
2.5
95.8
3.8
97.7
5.620.6
Kir. (3)
LLR
92.0
79.9
93.3
1.240.7
94.6
2.6
96.4
4.3
RoBERTa
92.0
96.5
98.3
1.8+0.4
98.5
2.0+o.4
98.5
2.0+0.4
LLh
90.2
90.8
95.4
4.620
96.8
6.0
96.6
5.820.5
Bahri
LLR
90.2
84.2
92.9
2. 2.620.s
94.6
4.4
97.2
6.920.7
RoBERIa
90.2
97.4
98.0
0.6+0.a
98.4
1.0+o.a
98.7
1.3+0.3
LLh
59.2
91.2
91.5
0.3
91.6
0.4
88.2
-3.020.4
Kuditipudi
LLR
59.2
85.9
86.7
0.7
86.7
0.7=0.a
86.7
0.8
RoBERTa 59.2
98.2
98.2
0.0
98.2
0.0
97.9
-0.3
Table 22: Main table of accuracies when GrMMA-7B--NSTRUUE is applied to the test set of eliß-category and
ALSTRAL--B-INSTRUCE generations are used as negatives at a target length of 100 tokens. The trends here are similar to those for human negatives.
WM
18
28 y
MLP
MLP
Tree
Tree +
L.L.
35.620.7
58.81o
98.1
7.220.
96.6
5.720.3
Aaronson
LLR
40.5+0.a
58.8+1.
96.6
5.7+0.7
92.3
1.4-0.8
RoBERTa
31.1+
54.2+1.0
99.0
1.1+0.a
99.0
1.0=0.a
LLh
0.020.0
0.0
90.3
-0.020
90.4
0.0
Kir. (0.5)
LLR
0.0
0.0
85.4
1.5+0.4
75.3
-8,6 8.620.6
RoBERTa
0.0
0.0
97.8
-0.0+o
97.8
2.0
LLh
19. 19.8
31.610.8
93.2
4.5
94.5
5.8
Kir. (2)
LLR
7
41.31o
90.8
4.020
92.4
5.620.6
RoBERTa
9.1
14.640
98.2
0.740
96.9
-0.520.
L.h L.R.R.
39.8
67.011.0
97.4
5.410.
97.5
5.520.a
Kir. (3)
42.810.s
71.110.
97.3
5.320
96.4
4.3
RoBERTa
33.2
57.8+1.0
98.7
2.2+0.a
98.3
1.8=0.4
LLh
37.5 54.911.
98.2
7.41os
97.3
6.52p.s
Bahri
LLR
39.240.s
63.1
96.9
6.72a.
78.9
-11.4110
RoBERTa
32.1
42.9
98.7
1.340.a
98.4
1.0mp.a
LLh
0.8
0.8+0.2
91.3
0.1+a.4
88.2
-2.920.s
Kuditipudi
LLR
2.620.3
2.7
85.0
-0.920.6
85.0
-0.920.3
RoBERTa
0.3
0.440.
98.0
-0.1401
98.2
0.0
Table 23: Cascade hit rates and accuracies of MLP and Tree methods when GEMMA-7B-INSTRUCE is applied
to the lest set of eli5-calegory and MISTRALL-T--NSTRUCCT generations are used as negatives al target length of 100 tokens. The trends here are similar to those for human negatives.
38


Reviews:
[{"review_id": "NpqJVeLPMC", "reviewer": "Reviewer_GCTQ", "summary": "", "questions": "", "limitations": "", "rating": null, "confidence": null, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "Yes", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "paper_topic_and_main_contributions": "", "excitement": null, "reproducibility": null, "ethical_concerns": "", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "summary_of_contributions": "This work proposed a hybrid approach that ensembles watermark and non-watermark detectors to handle the low-entropy prompt where the watermark is unreliable. The experiments show that across watermark schemes and detectors on two datasets, hybrid approaches consistently beat either alone. The authors also show the robustness performance under two attacks.", "claims_and_evidence_yn": "Yes", "claims_explanation": "- The experiment is extensive, including a mixture of multiple watermark methods and non-watermark detectors, trying multiple hybrid approaches, and evaluating robustness. The authors also present massive results in the appendix.\n- The summary of experimental findings in Section 4 is insightful and easy to follow.\n- The proposed method is well-motivated.", "audience_yn": "Yes", "audience_explanation": "AI detection is an important topic, and this paper explores a hybrid of watermark and non-watermark detectors which is of interest to many in the TMLR audience", "requested_changes": "- The ensemble method is modest. The author is encouraged to explore this aspect more deeply, such as why *Weaker non-watermark detectors can offer stronger performance in a hybrid setup.*\n- The performance comparison between watermarking methods and detectors on low-entropy prompts could be presented in the introduction to more clearly highlight the motivation.\n- The adversarial attack evaluation is relatively weak. Stronger methods [1,2] should be considered or at least discussed. For example, RoBERTa achieves high performance (85–90% accuracy) on paraphrase attacks.\n- Some important experimental results should be moved to the main paper instead of the appendix, such as overfitting. A concise summary table could be used to present these results clearly.\n\n[1] Paraphrasing evades detectors of AI-generated text, but retrieval is an effective defense. NeurIPS 2023.\n\n[2] Large Language Models can be Guided to Evade AI-generated Text Detection. TMLR 2024.", "broader_impact_concerns": "NA", "additional_comments": "", "extra_scores": {}, "extra_text": {}}, {"review_id": "Hz9u98cS54", "reviewer": "Reviewer_j18Y", "summary": "", "questions": "", "limitations": "", "rating": null, "confidence": null, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "Yes", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "paper_topic_and_main_contributions": "", "excitement": null, "reproducibility": null, "ethical_concerns": "", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "summary_of_contributions": "This paper investigates how combining watermark-based and non-watermark-based detectors can improve the detection of AI-generated text. The authors provide an extensive empirical study covering multiple watermarking schemes, several non-watermark detectors, and three hybrid strategies (two cascades and one learned fusion). The experiments are systematic and the results convincingly show that hybrid detection can significantly boost accuracy, especially in low-entropy conditions where traditional watermark detection is weak.", "claims_and_evidence_yn": "Yes", "claims_explanation": "The experimental results are thorough and well-documented. The paper evaluates a wide variety of watermark and non-watermark detectors under consistent settings, across multiple models and datasets. The evidence clearly supports the claim that hybrid detection improves performance under low-entropy conditions. Statistical comparisons and robustness analyses are convincing.\n\nHowever, while the empirical evidence is strong, the paper lacks a deeper theoretical explanation of why certain hybrid combinations outperform others. The relationship between entropy and detectability remains observational rather than analytically grounded.", "audience_yn": "Yes", "audience_explanation": "The paper addresses a timely and practically relevant topic for the TMLR audience—how to improve provenance and authenticity detection for large language models. Researchers and practitioners in AI safety, watermarking, and responsible AI would find the results valuable. The study also complements recent work on watermark robustness and AI text forensics.\n\nAlthough the conceptual novelty is modest, the empirical insights are important and broadly applicable, making it an interesting contribution for both academic and applied communities.", "requested_changes": "1. While the proposed hybrid detection framework (watermark + non-watermark) is executed carefully and demonstrates empirical value, its conceptual novelty is modest.\n> Suggestions: 1) Introduce adaptive weighting mechanisms that automatically adjust the fusion parameters based on text entropy, token length, or detector confidence. or 2) Provide a mathematical formulation of the decision boundary learned by the logistic regression model to highlight what it learns beyond heuristic thresholding.\n\n2. The study assumes white-box (1P) access to model logits or watermark keys, which limits its generality.\nIn real-world applications such as platform moderation, media provenance, or forensic analysis, detectors typically operate in a third-party (3P) or black-box setting, without access to internal model information.\n> Suggestions: Extend the framework to 3P-compatible detectors that use only surface-level statistics or model-agnostic proxies (e.g., perplexity ratios).\n\n3. The hybrid models (especially logistic regression) are calibrated on a dataset different from the test domain (ELI5 vs. Dolly), implicitly assuming cross-domain consistency. This calibration dependency raises questions about robustness to domain shifts, prompt distribution changes, or stylistic differences in text.\n> Suggestions: 1) Conduct explicit cross-domain generalization experiments (e.g., calibrate on Dolly, test on news or social media text). 2) Quantify the sensitivity of hybrid thresholds to calibration data by measuring AUC variance under dataset perturbations.\n\n4. The paper empirically confirms that detection improves with higher entropy prompts, but provides no formal analysis linking entropy to detection capability. This remains an empirical observation rather than a theoretical understanding.\n> Suggestions: 1) Provide a probabilistic model describing how text entropy bounds the achievable ROC-AUC of watermark detectors. 2) Analyze mutual information between watermark scores and non-watermark features to quantify their complementarity.\n\n5.  Attack evaluation is narrow: paraphrasing uses only Gemini-1.5-Pro with a single paraphrase intensity; random token replacement is synthetically defined and may not reflect realistic obfuscation.\n\n6. Negative samples are limited to “human” or “different-model” text; mixed human–AI co-authored text, which is prevalent in practice, is not tested.\n\n7. Although cascade “hit rate” (γ) is reported as a proxy for computational savings, the paper does not quantify real-world cost metrics such as FLOPs, latency, or energy use. This makes it difficult to assess trade-offs between detection accuracy and efficiency.", "broader_impact_concerns": "I do not see major ethical concerns in the presented work. The paper deals with AI-generated text detection and does not involve human subjects, sensitive data, or downstream decision-making. Its potential societal impact is positive: improving content attribution and AI transparency.", "additional_comments": "I recommend acceptance after major revisions focusing on addressing the above concerns.", "extra_scores": {}, "extra_text": {}}, {"review_id": "4SSfeVR0jc", "reviewer": "Reviewer_FJ8Y", "summary": "", "questions": "", "limitations": "", "rating": null, "confidence": null, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "Yes", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "paper_topic_and_main_contributions": "", "excitement": null, "reproducibility": null, "ethical_concerns": "", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "summary_of_contributions": "In this work, the authors propose to study the complementarity of passive detection methods -- i.e. forensics -- with active ones -- i.e watermarking -- in the case of LLM-generated text. To my knowledge, this hybrid approach has been scantily researched, with good reasons since the practical setting it entails is somewhat strange. I will discuss the significance of the approach later but I believe it does have its place and that such a study is welcomed in the field. \n\nSince the subject matter: \n-  is not aiming at SOTA performance with purely novel methodologies, \n-  entails a peculiar detection setting,\n\nI believe this work has its place in a venue such a TMLR which specifically targets such cases.\nSince the paper is quite a straightforward empirical study of combining watermarking and forensics detector, I don't believe it requires a more lengthy summary. I will just say that the paper claims that an hybrid approach almost always improves watermarking-only methods which is quite surprising (see questions), and thus fully justifies a study.\n\nOverall, the paper is very well-written, with complete and detailed experiments.  The last section summarizes the findings pretty well without overselling the approach. I feel the SOTA of both text watermarking and text forensics have been respected and correctly integrated in the work. I will also command the authors for the effort in providing all the elements needed for replicability.\nIn my opinion, the only main weakness of the paper is its reliance on a high FPR (1%), which I address in the rest of the review.\nThe question of the importance of the findings rests on the usefulness of the hybrid-approach which is not trivial as I will also discuss below.\n\nReferences are provided in the additional comments.", "claims_and_evidence_yn": "Yes", "claims_explanation": "This paper being purely empirical study, I will evaluate its merits based on two main criteria:\n\n- The soundness of the experimental study: do the experiments allow me to validate/invalidate the thesis of the authors ? Is every parameter of importance \n- The replicability of the study\n\n### Soundness of the study \n\n#### Structure\n First and foremost, I will command the authors on the overall structure of the paper. The main thesis of the paper is perfectly delineated: \"It is possible to improve watermark detectors with forensics detector in the case of generated content detection\". Along the thesis, a few quantities of interests are singled out as having an important impact on performance: \n- the entropy of the model output provided the context\n-  the length of the generated text\nAs pointed out by the authors, the entropy in particular is not well-studied in the case of forensics detector, this paper is thus a good reason to provide such a study. I will further highlight that despite the fact that these two quantities have quite a large impact on watermark performance, I still review far too many papers bypassing such study altogether.\n\n#### Choice of attacks\n\nThe choice of attacks for text watermark still looks quite like the wild west in the field to me. I have been convinced by [1, Section 5] that the three main operations of deletion, addition and replacement of tokens provide a universal model of attack for the case of watermarking and that a curve of watermark performance as a function of the percentage of modified tokens is enough to completely characterize its robustness to attacks. The authors provide such a study in Figure 3. which is sufficient to me. The achievability of an attack for a fixed level of quality (provided a metric) is not a problem to solve by the defender, only by the attacker, since it quality should not have any impact on the \"robustness curve\" for the watermark.\n\nNow obviously the question is different for forensics based detector since their performance directly depend on the distribution of the text for their detection. This is opposite to watermarking, which usually only depends on the distribution of the secret. In the former case, I question the use of a paraphrasing attack. Indeed, even though it is currently the most powerful attack against watermarking, that also preserves quality quite well in practice, it does make the text very amenable to forensics detector since it will follow the original, non-watermarked LLM distribution. The authors are aware of this as they (quite rightly) point it out in Section 3.6. To provide a fair comparison, I would have added a more naive attack which does not depend on an LLM, for example random replacement by synonyms which should somewhat preserve the quality of the text while not relying on any LLM.\n\n#### Choice of metrics\n\nThe authors report two main \"one-number\" metrics to report for performance: \n- A partial AUC -- computed under a FPR of interest\n- Accuracy\n\nI appreciate that the authors recognize the uselessness of studying regimes with very high FPR in the case of text detection -- as shown by the use of a pAUC.\nNevertheless, I am disappointed in the use of such a high FPR as 1%. I understand the authors work with a a \"small\" (~5000 samples is actually quite respectable) number of prompts and compute their FPR/TPR empirically on watermarker and detector scores.\n\nTo me, this goes against the philosophy of watermarking which should ensure high power at extremely low FPR ($ << 10^{-6} $ ). In the case of text, this is achieved by constructing a pivotal statistic on a well designed distribution of the watermarking signal (e.g. the red-green list score of Kirchenbauer). A well-designed scheme allows to specify exactly the distribution of the detector score under $H_0$ -- i.e. the non-watermarked case. This, in turn, allows to compute a sound $p$-value for the text under scrutiny -- allowing to reach extremely low **guaranteed** probabilities of false-alarm with usually very high power. A case in point for this approach is [2] -- which derives the correct statistical test for both Kirchenbauer's and Aaronson's schemes. In practice, it is also used with success in Watermax [1] and SynthID [3] (as the so-called \"frequentist\" detector) that the authors cite in this paper. \nThis is the reason why \"active methods\" such as watermarking are quite enticing compared to forensics methods: they allow to reach far better performance than passive methods, with strong  **theoretical guarantees** in terms of $P_{FA}$.  A recent, dramatic illustration of this, though in the different setting of images, is from Figure 3 in the Stable-Signature paper [4].\n\n\nEach metric is computed as a function of: entropy, text length and percentage of corrupted text. These three choices are in my opinion necessary and sufficient for the evaluation of the method.\n\n### Replicability\n\nThe LLMs are open-source.\nAll experimental details are provided with the training and testing set of forensics detector dully specified.\nThe methodology is clear and completely specified.\nError intervals are provided, including the methodology to compute them.\n\nThe only missing piece of information is the choice of hashing function for the Kirchenbauer and Aaronson watermarkers. I will assume that it is the baseline used in the original works (which is not a correct choice -- see [2] and [5] -- but is at least standard).\n\nIn conclusion, this should be a highly replicable paper.", "audience_yn": "Yes", "audience_explanation": "I believe the interest in these findings hinges on the following question: is the setting entailed by the hybrid approach useful in practice ?\n\nThe case for mixing watermarking and forensics is, in the best of worlds, somewhat of a nonsense. A watermark is designed to vastly outperform any forensics methods in exchange for a strong drawback: it needs to be able to modify the content in the first place. Forensics methods, on the other hand, work in the far more difficult setting of knowing **nothing** except the sample under scrutiny. Furthermore, watermarking algorithms, in a multi-bit setting, can also insert a payload in the watermarked content -- though I understand this is not the setting chosen by the authors. My point is that a good watermark design should preclude any hybrid-approach. Yet this paper makes the case that we are not there yet, especially in low-entropy settings.\n\n My counter-claim is two-fold:\n\t1. The low entropy, low text size, setting is generally of no interest to watermarking. Small answers generated by LLMs -- such as answers to quizzes or factual questions -- are not meant to be watermarked as they do not contain the \"added-value\" of the LLM. Even worse, the user can always truncate the answer to what interest them (no point in watermarking a proper name). Texts of interest are usually somewhat long (articles, essays, stories etc...) and provide plenty of room for the watermark signal to be very detectable even at low-entropy. I am ready to discuss this point with the authors since this claim is quite subjective and might reflect a bias in my vision of practical applications.\n\t2. The recent WaterMax scheme [1], given enough compute, should be able to reach an almost arbitrary power at any specified $P_{FA}$ . Granted, this assumes enough entropy such that at least two different tokens can be generated for most of the text. Yet, otherwise, this reinforces my qualms about the hybrid-approach: is it only useful for weak schemes in a very constrained setting which is low-entropy and low text-size?  \n\nAs such, (see requested changes) if the authors can better motivate the use of such an hybrid approach, I believe the results of this paper are very interesting since **they challenge the key assumption that watermarking always provide better performance than forensics detectors**.", "requested_changes": "- **High FPR, bad performance** To follow up on my discussion on $p$-values. I am quite surprised of the bad performance of both Kirchenbauer and Aaronson in this paper. Following Appendix K of [3], text of around 250 tokens should lead to good detectability at FPR around $ 10^{-3} $  under the chosen settings (though the Watermax paper uses $\\gamma =0.5$  which might account for the difference). What I would like to know is: is it because an exact $p$-value is not used (which I would find bizarre), is it because of the low entropy setting? (Aaronson is especially dependent on high entropy). In any case, I would appreciate the authors computing the $p$-value for their scores and reporting them -- this should be very simple if the scores have been saved, by using the formulas in [2]. This is critical to securing my recommendation as this will explain the *apriori* low performance of the SOTA on the author's setting. \n\n- **Hashing function**: Please provide the hashing function used for Kirchenbauer and Aaronson. I expect it to be the original one which has been shown to lead to be insecure and lead to unreliable scores in [2] and [5]. Though not necessary, I believe using a secure hash function that also guarantees the independence of token scores (at the cost of some robustness) would lead to a stronger work. At the very least, the hashing function needs to be spelled out in the appendix.\n\n- **The setting**: I am only expecting the authors to defend their approach by answering my counter-claim. I do not necessarily ask for further experiments -- though a comparison with WaterMax or another scheme able to reach arbitrary performance would be welcome. In this case, the question would be, how much compute do I save by using an hybrid approach instead of WaterMax? This is critical to securing my recommendation, since, to the practitioner, an hybrid approach might seem counter-productive if the watermark is well-designed. \n\n- **Synonym attack**: As said in the discussion, I would appreciate adding one more attack which does not clearly favor forensics detectors. This would strengthen the work in my view.\n\n- **Choice of title**: The current title is very generic and does not convey anything of the paper's content. I would advise the authors to rename their work by making explicit the use of hybrid methods between forensics and watermarking. This would strengthen the work in my view.\n- **Minor nitpicking**: p7 - an attack is always adversarial by definition, otherwise it is an augmentation. Attack is sufficient in this case.", "broader_impact_concerns": "NA", "additional_comments": "References:\n\n- [1] Giboulot, Eva, and Teddy Furon. \"WaterMax: breaking the LLM watermark detectability-robustness-quality trade-off.\" NeurIPS 2024-38th Conference on Neural Information Processing Systems. 2024.\n- [2] Fernandez, Pierre, et al. \"Three bricks to consolidate watermarks for large language models.\" 2023 IEEE international workshop on information forensics and security (WIFS). IEEE, 2023.\n- [3] Dathathri, Sumanth, et al. \"Scalable watermarking for identifying large language model outputs.\" Nature 634.8035 (2024): 818-823.\n- [4] Fernandez, Pierre, et al. \"The stable signature: Rooting watermarks in latent diffusion models.\" Proceedings of the IEEE/CVF International Conference on Computer Vision. 2023.\n- [5] Jovanović, Nikola, Robin Staab, and Martin Vechev. \"Watermark stealing in large language models.\" arXiv preprint arXiv:2402.19361 (2024).", "extra_scores": {}, "extra_text": {}}]

Author rebuttal:

Decision: Accept (poster)