Watermarking Diffusion Language Models
Authors: Thibaud Gloaguen
Robin Staab
Nikola Jovanović
Martin Vechev
Venue: ICLR 2026 Poster (2026)
Keywords: Watermarks
Diffusion Language Models
LLM
Primary area: generative models
TL;DR: We design the first watermarking scheme tailored for diffusion language models

Abstract: We introduce the first watermark tailored for diffusion language models (DLMs), an emergent LLM paradigm able to generate tokens in arbitrary order, in contrast to standard autoregressive language models (ARLMs) which generate tokens sequentially. While there has been much work in ARLM watermarking, a key challenge when attempting to apply these schemes directly to the DLM setting is that they rely on previously generated tokens, which are not always available with DLM generation. In this work we address this challenge by: (i) applying the watermark in expectation over the context even when some context tokens are yet to be determined, and (ii) promoting tokens which increase the watermark strength when used as context for other tokens. This is accomplished while keeping the watermark detector unchanged. Our experimental evaluation demonstrates that the DLM watermark leads to a >99\% true positive rate with minimal quality impact and achieves similar robustness to existing ARLM watermarks, enabling for the first time reliable DLM watermarking.

Published as a conference paper at ICLR 2026
WATERMARKING DIFFUSION LANGUAGE MODELS
Thibaud Gloaguen, Robin Staab, Nikola Jovanović, Martin Vechev ETH Zurich thibaud. gllaguen@inf.ethe
ABSTRACT
We introduce the first watermark tailored for diffusion language models (DLMs), an emergent LLM paradigm able to generate tokens in arbitrary order, in con- trast standard autoregressive language models (ARLMs) which generate tokens sequentially. While there has been much work in ARLM watermarking, key challenge when attempting to apply these schemes directly the DLM setting is that they rely on previously generated tokens, which are not always available with DLM generation. In this work we address this challenge by: (i) applying the watermark in expectation over the context even when some context tokens are yet to be determined, and (ii) promoting tokens which increase the watermark strength when used as context for other tokens. This is accomplished while keeping the watermark detector unchanged. Our experimental evaluation demonstrates that the DLM watermark leads to >99% true positive rate with minimal quality impact and achieves similar robustness to existing ARLM watermarks, enabling for the first time reliable DL.M watermarking. Our code is available here.
INTRODUCTION
While autoregressive language models (ARLMs) have demonstrated remarkable performance (Ope- nAI et al., 2024; Grattafiori et al., 2024; Bubeck et al., 2023), other approaches to language modeling, such as diffusion language models (DLMss, are rapidly catching up, with recent works (Nie et al., 2025; Ye et al., 2025; Labs et al., 2025) approaching similar capabiliiies at a significantly lower cost. Notably, DLMs have higher generation speed, offer built-in error correction, are more controllable, and can be naturally extended to multiple modalities (Yang et al., 2025). These advances also increase the risk of misuse, making it paramount to reliably detect text generated by these models. Prior works (Kirchenbauer al., 2023; Kuditipudi al., 2024; Christ et al., 2024) have proposed the concept of ARIM watermarks: the generated text is augmented with an imperceptible signal that can later be detected to trace its provenance. Such watermarks are already adopted in consumer-facing models (Dathathri et al., 2024) and are being advocated for through regulation (EU Council, 2024). Yet, most existing works on language model watermarking rely on hashing mechanisms compatible only with autoregressive generation, highly limiting their application for DLMs. Our work is the first attempt to overcome this limitation, enabling efficient and reliable watermarking for DLMs.
ARLM Watermarks Watermarks for ARIMs traditionally rely on three key components. The hashing mechanism (i) uses the previously generated tokens to seed the sampling procedure (ii), which then inserts the watermark signal into the generated text. Critically, this pipeline relies on the previous tokens to have already been generated compute the hash, an assumption often violated with DLMs. The watermark detector (iii) leverages the hashing mechanism to retrieve the seed associated with each token and compute a test statistic to determine whether given text watermarked.
Diffusion Language Models DLMs produce probability distribution over the set of fixed-length sequences containing masked tokens, which, analogous to noise in continuous diffusion models, rep- resent placeholders yet to be generated. Starting from a (partially) masked sequence, DLMs iteratively sample (unmask) tokens until the sequence is fully generated. Importantly, unlike autoregressive models, DLMs are not constrained to unmasking tokens in a left-to-right order, making hashing-basee based ARLM watermarks inapplicable whenever a token without full prior context is unmasked. natural workaround is to restrict the watermark application to tokens with fully available context. Yet. show in Sec. 4, such a naive approach leads to a weak watermark, as barely any tokens fulfill this criterion. This motivates the need for novel watermarking algorithm tailored to DL.ss

---

Published as a conference paper at ICLR 2026
Prior works
Our watermark
Detector is unchanged
Autoregressive
X+
Fsr
pack
Mix
?
preas
How to perturb the logits of ?
Hash
H
Make
green
Make g green
Binomial test
Diffusion
Boost green tokens expectation over hashes!
Boost tokens that lead to hashes making green!
greaa
p 1.3e
Hash
Watermarked!
?
Impossible compute Red/Green split!
Watermarked distribution
Figure I: An overview of why current watermarks for ARLMs fall short the diffusion setting (left), how our watermark operates in this setting (middle) and how our watermark detector works (right).
This Work: Watermarking Diffusion LMs this work, we introduce the first watermarking scheme tailored to DLMs based on the prominent Red-Green watermarks (see Sec. 2), as illustrated in Fig. 1. First, we identify the limitations of autoregressive hashing in most prior works on ARLM frame
watermarks when applied in the diffusion setting (Fig. 1, left). To overcome this limitation, watermarking as a constrained optimization problem (Sec. 3. 3.1). From this optimization problem, then derive a practical watermarking scheme for DLMs (Sec. 3.2). In Sec. 3.3, interpret scheme as a combination of two intuitive components: applying ARLM watermarks in expectation over the context hashes and biasing tokens that lead to hashes making other tokens green (Fig. middle). Our detector is exactly the same as in prior Red- Red-Green watermarks: we compute the color of each token in the sequence and perform binomial test (Fig. (right)). our experiments, we show that our watermarking scheme is significantly more detectable than naive baseline adaptations of ARLM watermarks (exceeding 99% TPR 1% FPR with negligible quality degradation), and robust to common modifications of the generated text, e.g., substituting words in context (Sec. 4).
Main Contributions Our key contributions are:
We conduct the first study of watermarking for DLMs, identifying the limitation of ARLM watermarks in the diffuusion setting and proposing watermarking scheme tailored for DLMs.
We formulate the problem of DLM watermarking as constrained optimization prob- lem (Sec. 3.1) from which derive practical watermarking scheme that leverages existing watermark detectors from prior works (Sec. 3.2).
We further interpret watermarking scheme extension of existing ARLM water- marks (Sec. 3.3) and demonstrate that, when restricted to the autoregressive case, our optimization formulation recovers exactly ARLM watermarks (Sec. 3.3).
Through extensive evaluation, we show that our scheme is suitable for practical use, preserves the generated text quality and is robust against common natural text modifications (Sec. 4).
2 BACKGROUND AND RELATED WORK
Language Modeling The goal of language modeling (Dai & Le, 2015; Devlin et al., 2019) is to train a model to learn a probability distribution pe over natural language. In particular, given a vocabulary E and the true probability distribution Paata the goal is to find 0 such that pa ~ Pdata With autoregressive modeling, given a text w E Σ", its probability is factorized sequentially using the (probability) chain rule. While this formulation allows for efficient training (Radford et al., 2018; 2019; Brown et al., 2020), it forces ARLMs to generate new tokens sequentially, which fundamentally limits their generation speed and might restrain their capabilities (Berglund et al., 2023). Recent works (Gloeckle et al., 2024) have proposed multi-tokens prediction as a training objective to improve model capabilities, but cnucially still generate tokens autoregressively.
2

---

Published as a conference paper at ICLR 2026
Diffusion Language Modeling In contrast, in diffusion language modeling (Austin et al., 2021a; Lou & Ermon, 2023; Shi et al., 2024; Ou et al., 2024; Sahoo et al., 2024), the vocabulary Σ is extended with a mask token representing a position where a token is yet to be sampled. Let Σ be the vocabulary including the mask token. The diffusion process is parameterized by a fixed number of diffusion steps N and a generation length L. At each diffusion step i, the model takes a sequence of tokens a(4) E (E) and returns factorized probability distribution p()) E A(E) over the set of sequences, where Á denotes the simplex. Each token in the sequence of length L is then sampled independently from p()), which returns an intermediary sequence ((++1) E اا This sequence is then masked w+(+)) == Mask(w(tt)), ,,())) using pre-determined stochastic masking procedure Mask: Σ" ((E) + (E)². We say a token is getting unmasked if it was masked in a+() but not in (,++1) This process iterated N times, returning a final (N+1) E Σ which has no mask tokens. Unlike ARLMs, DLMs can, in each step, generate (unmask) multiple tokens and in any order. We evaluate our watermark with multiple unmasking procedure Mask: uniform (Austin et al., 2021a), based on distribution entropy (Kim et al., 2025), and by block (Ariiola et al., 2025).
Large Language Diffusion Models With the release of open Large Language Diffusion Models (Nie et al., 2025; Ye et al., 2025), DLMs have gained significant traction in multiple domains: reasoning (Sahoo et al., 2025; Cetin et al., 2025), chemistry (Tan et al., 2025), multimodality (Yang et al., 2025), vision (You et al., 2025) and safety (Ma et al., 2024). For traceability, previous works (ismail Tarım & Onan, 2025) only explored zero-shot detection of DLMs generated text, and highlighted the lack of a reliable watermarking solution in this paradigm.
LLM Watermarks The goal of generative Al watermarks is to ensure content traceability, i.e., the ability to rigorously identify whether given piece of content has been generated by an AI model. Key challenges for watermarks include balancing their impact on content quality, their strength, and their robustness to edits (Tu et al., 2024; Pan et al., 2024), while ensuring their reliability against false positives. For ARLMs, a range of generation-time watermarks (Kirchenbauer et al., 2023; Kuditipudi et al., 2024; Aaronson, 2023; Dathathri et al., 2024; Christ et al., 2024) have been proposed by building on the autoregressive nature of the underlying models. Prior works have also proposed order- agnostic watermarks, namely Unigram (Zhao et al., 2023) and its extension Pattern (Chen et al., 2025), that can be directly applied to DLMs. However, such approaches significantly compromise on watermark security (Jovanović et al., 2024; Zhang et al., 2024), We nonetheless show in Sec. 4.4 that our approach, tailored for DLMs, outperforms such schemes. Other concerns regarding watermark design include their applicability in the open-source setting (Gloaguen et al., 2025; Xu et al., 2025).
In this work, we study the most popular family of current ARLM watermarks: Red-Green watermarks Kirchenbauer et al. (2023). Let wi E Σ denote the token generated by the LM at step t. and k the context size parameter. Using a hash of the context H( the vocabulary Σ into green subset and the remaining red subset. The size of the green subset is set to YEE,, with 7 E (0, 1) commonly chosen as 0.25 or 0.5. To insert the watermark, each green token in the logits vector gets shifted up by constant 6, increasing the overall likelihood of sampling green token. To detect the watermark, given a sequence of tokens w E Σ*, we extract a corresponding color sequence and perform binomial test the number of green tokens.
pseudo-random function partitions
Image Diffusion Watermarking With the increasing popularity of diffusion models for image generation (Dhariwal & Nichol, 2021; Ho et al., 2020; Nichol & Dhariwal, 2021; Song et al., 2020), image diffusion watermarking has been the focus of many works (Fernandez et al., 2023b; Wen et al., 2023; Yang et al., 2024). Yet. all generation -ime image diffusion watermarks operate in a continuous space, making them fundamentally inapplicable to the discrete diffusion process of DLMs.
3 WATERMARKING DIFFUSION LANGUAGE MODEL GENERATION
In this section, we present our approach to watermarking DLMs. Sec. 3.1 introduces a theoretical framework to guide our DLM watermark design, while Sec. 3.2 addresses practical challenges and proposes an instantiation of our watermark. Lastly, in Sec. 3.3, we how watermark naturally extends Red-Green ARLM watermarks: it decomposes into the two terms illustrated in Fig. (middle) and, when restricted the ARLM case, instantiates to Red-Green ARLM watermarks.
3

---

Published as a conference paper at ICLR 2026
3.1 ADAPTING WATERMARKS TO THE DIFFUSION SETTING
We propose framing the LM watermarking algorithm as an optimization problem, which in turn allows us adapt watermarks for autoregressive language models to the diffusion setting.
Naive Approach As alluded to in Sec. 1, a naive approach to adapting Red-Green watermarks in the diffusion setting would be to apply them only tokens whose context fully unmasked. Indeed, for such token positions, computing their context hash and thus the watermark distribution possible. However, this means that we cannot apply the watermark (e.g., boost the probability of green tokens) to any token that is unmasked before its respective context. Ultimately, this would result in only a handful of green tokens which, as we show in Sec. 4.1, leads to low true positive detection rate. Importantly, we find that watermark tailored for DLMs needs to operate directly over the distribution of context hashes, enabling to watermark all tokens regardless of the unmasking order.
Goal of Red-Green Watermarks Given a sequence w. the detector computes the proportion of green tokens (ww) within that sequence and conducts a binomial test. If (w) significantly exceeds the baseline ratio of green tokens expected under the null hypothesis 7, the sequence w is identified as watermarked. Thus, the goal of the watermarking algorithm is to modify the sampling procedure such that each generated sequence has a high ratio (w), while preserving the model's utility.
Watermarking as a Constrained Optimization Problem A watermarked DLM should prefer- entially sample token sequences that maximize the ratio (w). This naturally leads to framing the problem of DLM watermarking as a constrained optimization task: we aim to maximize the expec- tation of the green token ratio over the generated sequence while minimizing the impact on model performance. We show in Sec. 3.2 that our proposed optimization framework abstracts the complexity of DLM watermarks into a practical watermarking algorithm, illustrated in Fig. 1 (middle).
Recall that Σ is the model vocabulary, and Σ is the vocabulary extended with the mask token. Let w E _ be an unmasked sequence of tokens. For every token position I, we introduce a hash function H, Σト + H that returns the hash of the context at position I, with H the finiee set of possible hash values. We assume that the token does not belong in its own context (no self-hashing assumption). We further discuss this assumption in App. H. We introduce the global binary green list matrix G (0. HxE which, given the context hash and token, returns its color. We formalize the green L ratio function y: E + [0, 1] as (ww) 1 Σi= (n,()) i.e., the color GH, (م),ي of each token Wi averaged over the sequence w.
Let @ E (E)² be a (partially) masked sequence and p(i) E A(E)" the corresponding factorized probability distribution over the set of sequences returned by the DLM forward pass. Given e [l..... L), Pt is the probability over E at position 1. The goal of the watermarking algorithm is to distort the factorized probability distribution p()) into factorized probability distribution y(w) that maximizes the expected green ratio of sequences sampled according to (()). We have to solve,
q' = arg max Ba- (5()), subject to Vt E [1,..., L}, KL (r)) S E.
gEA(E)2
In Eq. (1), the KL constraint with > is used proxy for controlling impact quality. We now explain how to exactly compute the expectation from Eq. (1), For any q E A to compute the expectation, given Q ~ q (i.e., l is the random variable representing sequences distributed according to the factorized probability distribution q). we need to know the distribution of Q2 and (2)). The distribution of 2 is by definition gt. Computing the distribution of H?)),, i.e., the distribution of the context hashes, is more challenging. We model it for every token position / as function of g, he : ( + AHH). and defer instantiations to Sec. 3.2. We can now unroll the expectation.
1 Σ t=1
Eang (HanglGH..ooo.. = Σ T((a) G. =: - (2)
Vy E A (E)), Ea (1() =
where the factorization comes from the no self-hashing assumption (i.e., H2 1 92) When expanding the dot products, each element is the color of a (token, context hash) tuple weighted by its probability. For clarity, we refer to J as the energy function. The watermark is equivalent to
q" = arg max Jqq.. subject to Vt E [l.....I, L), (g( (.e.P((0)) (()) Se.
(
4

---

Published as a conference paper at ICLR 2026
Solving Eq. (3) is challenging due to the KL constraint. Without the constraint, greedy algorithm would suffice. In Theorem 3.1, we provide an implicit solution to Eq. (3), with the proof in App. J.
Theorem 3.1. Given E (E and J defined in Eq. (2), there exists ō E IR- such that
vt [ L..9. pt exp(6rar
(4)
with ax(4) = Vo, J().. Moreover, for all t E [I,...,L]] de is the unique solution to KL( Pc)
E.
This result means that the distribution 9" is optimal with respect to our optimization problem, i.e., is the optimal way to turn any distribution p provided by our DLM into distribution that maximizes the expected green ratio while enforcing the KI. constraint. Importantly, it has a fairly simple form. If we take Eq. (4) in the logits space, we that the optimal solution corresponds to adding (,0x( to the logits vector-a similar distortion to the Red-Green ARLM watermark. In Sec. 3.3, we provide an intuitive explanation how our watermark works. We show that d, (g") can be decomposed into two components: one that makes the token at position t green and one that makes tokens which have the token at in their context green, as illustrated in Fig. (middle).
3.2 OUR WATERMARK ALGORITHM FOR DIFFUSION LMS
In this section, we instantiate the theoretical derivation from Sec. 3.1 to derive practical watermark algorithm. First, we explain how to solve Eq. (4) and find 6. We then detail instantiations of common hash functions and propose a corresponding practical watermark algorithm.
Solving the Functional Equation Eq. (4) defines a functional equation with q" and as the unknowns. We propose using fixed-point iterative approach of f: pexp(6a(q)//Z(g) which gives qº = f(4"). ). At each step i, given a q', we first find for this q' and then compute (f++1) = (g) We find in Sec. 4.3 that in practice a single iteration already yields a strong watermark, and that, despite the lack of theoretical guarantees of convergence q", increasing the number of iterations indeed slightly improves watermark strength.
To find 8, we can solve for all the equation KL Pt) guarantees the existence and uniqueness of 6. A simpler relaxation of our framework, closer to the parameterization of Red-Green ARLM watermarks, is to directly parameterize the optimization problem, and thus the watermark algorithm, by a constant ō E I instead of e. We refer to using E as E-parameterization and using 5 as d-parameterization. We compare both approaches in Sec. 4.3.
using bisection, as Theorem 3.1
Instantiating the Hash We now instantiate the hash H and the hash distribution h from Sec. 3.1. As illustrated in Algorithm 1, the hash is used seed pseudo-random function to generate a row of G. Without loss of generality, we further assume that Σ = {I...., ] and H = =I, ... } The exact cardinality of H depends on the specific hash implementation. For the hash we consider in this work, it systematically on the order of Σ Similarly to Kirchenbauer et al. (2024), we explore two local hash functions, SumHash and MinHash (we provide detailed comparison with the formulation of Kirchenbauer et al. (2024) in App. G.1). However, unlike in Kirchenbauer et al. (2024), for Eq. (2) to hold, we must respect the no self-hashing assumption. As alluded to Sec. 3.1, computing naively challenging and requires 2(E2) operations. Thanks to the specific local structure of the considered hash functions detailed below, show that the corresponding hash distributions can be computed efficiently through algebraic manipulation.
Algorithm 1 Green matrix G row generation
Require: A hash E H. watermark key g 1: G..: <- PseudoRandonGGeneraoo E
2: return G..:
SumHash sums the surrounding tokens. Let k E N be the context size, and C = (cI...... C the set of positions that define the context, where Vi, C + O. For instance, C= (-1, 1} means that, for every token position, the context is comprised of the previous token and the next token. In contrast to the ARLM setting, the context is no longer restricted to preceding tokens but can also include tokens after f. We define (with w, the token at position & in the sequence w).
Ye 24,VEe V [1...., ......,, L), SumHash (w); = Σ Wh+t
(5)
IEC
5

---

Published as a conference paper at ICLR 2026
We derive analytical formula for the probability distribution over the hashes h SumHassh as
Je
Vp A(E)* h4 (uwnHash)p))
Σ
II J=1
Pite, (u₂) (pe-os * Pc+ex )
MA H Hush (u))
(6)
where * is the convolution product and s E H. Using the Fast Fourier Transform to compute the convolution product, computing h SumuHash takes O(CC Σ| log [X]
operations.
With MinHash, the hash corresponds to the token id of the minimum of the surrounding tokens, i.c.
Va E E Vt [l....., LI, MinMash (w)) = min (wr+s),
(7)
iEC
with a random permutation, to ensure randomness in the token id ordering. For t E [l..... L], E Σ, let (u) = Pilo (u)). We have for s E H and p A(E)²,
M.iinHuhh (p)s
E
Petei (un) Pete, (ux) = Ar(s + - Ar(s),
(8)
Mi H HMnnnan Hannaaa (M1...M)))=
where (s) Tlee Eee, Du=o (u). Hence, computing h Minttash requires O(C[|E2) operations.
Our Watermark Based on the above meth- ods, we now present watermarking algo- rithm (Algorithm 2). Specifically, at each diffit- sion step i, given the distribution of the DLM p(a(0), we first compute the hash distribution h
Algorithm 2 Watermark for DLMs (simplified)
Require: DLM Probabilities number of iter-
ations n, watermark strength 6, generated sequence length L.
(line 3) using Eq. (6) or Eq. (8) and subsequently derive the corresponding energy function / (line
1: h < HashProbabilltie((p))
2: for from to L do
4) as per Eq. (2). For practical reasons, for each token position 4, only the top-k h, and pt (we ablate over practical choices of k in App. C). For each token position, then calculate the gradient of the energy function with respect to the token's 's probability distribution (line 6), and apply an exponential tilt proportional to this gradient scaled by the strength parameter (line 7), resulting in the final watermarked distribution (line 8). as per Eq. (4). The distribution further refined by iterating the tilting procedure. For detection, we use the same detector Red-Green ARLM watermarks: given w, we compute each token's 's color and perform binomial test on the number of green tokens after deduplication of (context hash, token) pairs (Fernandez et al., 2023a). Algorithm 2 time complexity, with SumHash, is O(nL[C|[]] log [-]).. Yet n, |C| are small which results in minimal generation overhead (App. I.1).
3:
at - P.
4:
q f- Pt e
5:
q g Normalize (q)
6: end for
7: return !
3.3 INTERPRETING OUR DIFFUSION LM WATERMARK
We now provide an intuitive explamation on how our watermark distorts the original model probability distribution p into watermarked probability distribution q". We perform explicit computation with SumHash to interpret the components of our watermark and extend this interpretation in App. J.2 for any hash function and context. Additionally, in App. E, show that instantiating our optimization problem (Eq. (1)) for the ARLM case preserves only the boost component and corresponds exactly to the Red-Green ARLM watermark.
Interpreting Our Watermark We analyze our watermarking algorithm with SumHash and C = {-1}. In this setting, the energy function simplifies to
=(p)= IPTIIGG
(9)
1=1
which in turn yields, using a fixed 6 and at the first step of the fixed-point iteration,
Vt E [l...., L), of x exp(oG exp((pee)))))))
(10)
expectation boost
prodictive bins

---

Published as a conference paper at ICLR 2026
1.0
1.0
0.9
0.8
1% FPR
TPR 1%
TPR @
0.4
0.5
Basellne
0.4
log(Perplexlly
Baseline
Unisstemarked LLM
0.3
0.0
60
100
160
200
250
300
350
400
log(Perplexity)
Number of Tokens
Figure 2: Detection Performance of Our Approach (Left) We compare the trade-off between watermark detectability (TPR@) @ 1) and text quality (log PPL) our approach and the baseline for different values of the watermark strength parameter and sequences of, on average, 275 tokens. (Right) For 8 = 4, we compare watermark detectability (TPR @ 1) between our approach and baseline as function of text length. Responses are generated by LLADA-8B with temperature 0.5 and 600 prompts from WATERBENCH. Crosses represent shared parameters between both figures.
We now recover the two components illustrated Fig. 1. The first term, exp(6G" Pt-1). ). is the Red-Green watermark boost taken in expectation over the distribution of the context, in this the previous token. If the mass of the distribution Pr- is concentrated on a single token wh-1, we recover exactly the Red-Green watermark boost G and add 6 to all green token logits. We label this first component the expectation boost. The second term, exp( (GGp++). is also intuitive: it favors sampling tokens that lead hashes for which the next tokens are more likely to be green. We call this the predictive bias. Overall, this means that our watermark for diffusion models constitutes a natural extension of Red-Green watermarks: both applies the Red-Green boost by 8 in expectation over the context hashes and additionally leverages the possibility of sampling hashes that lead to more green tokens at other positions. We extend this interpretation in App. J.2 for any hash function and context.
EVALUATION
In Sec. 4.1, we compare our watermarking approach for DLMs with baselines derived from autore- gressive watermarks. Sec. 4.2 focuses the robustness of our watermark against text modifications, and Sec. 4.3 examines the impact of different components of our watermarking scheme. We include further ablations and baselines (AAR, KTH and Gaussmark) in App. C, and more results in App. K.
Experimental Setup To evaluate watermark detectability, we follow the approach of WATER- BENCH (Tu et al., 2024). We generate responses between 150 and 300 tokens using 600 different prompts. For our watermark. we use the SumHash hashing scheme, 5-parameterization, a single iteration for fixed-point convergence, and top-k of 50. For the DLMs, we use LLADA-8B and DREAM-7B with temperature of 0.5 and random remasking strategy. We defer additional details of our experimental setup App. A.1 and ablate on most components in App. D and K.
4.1 WATERMARK STRENGTII AND QUALITY EVALUATION
The key challenge for watermarking DLMs highlighted in Sec. 3 is that, when generating given token, the context used to seed the watermark may not be known. To determine whether our approach from Sec. 3.2 overcomes this challenge, propose 3.1. When generating the token at position f, the context already set, we apply the ARLM watermark. Otherwise, do watermark the token probability distribution at position t.
baseline the naive approach described in
Strong Watermark Detectability We first evaluate watermark detectability using the previous token as context (C = (-1)) with LLADA-8B and DREAM-7B, For LLADA-SB, Fig. (left) shows that approach provides significantly better detectability than the baseline given the same impact on quality. In App. C (Fig. 8), we find that the improvement is especially noticeable for higher entropy remasking strategies (e.g., the seemingly random remasking strategies). In the limit, when using
7

---

Published as a conference paper at ICLR 2026
Table Detection Performance for Recommended Hyperparameters We compare the detectabil- ity of our watermark (TPR@) parameter The quality distortion (log PPL, GPT4 scores, and average benchmark accuracy) be- tween the baseline and our approach is similar, and minimal compared to the unwatermarked model, yet our approach consistently reaches 99% TPR @ Scores are averaged over 600 responses gener- ated at temperature 0.5. The average response length for LLADA-8B is 275 and 213 for DREAM-7B: Benchmark accuracies are measured at T 0.1, with accuracy for individual benchmark in Table 3.
@ 1) for different contexts and the corresponding recommended strength
C= (-1),8=4
C=
C= (-2, -1),
Model
Type
TPR@1
log(PP))
GPT1
Acc
TPR1
log(PPL)
GPT1
Acc
TPR
log(PPL)
GPT1
Acc
Unwatermarke
8.95
59,4
8.95
59,4
59
LLADA-8B
Baseline
0.63
1.93
8.48
55.5
0.69
1.86
8.51
55.7
0.83
1.91
8.37
58.5
Ours
0.99
1.90
8.43
56.0
0.99
1,80
8.60
53.7
0.99
1,80
8.59
54.5
Unwatemarked
50.4
50.4
8:45
50,4
DREAM-7B
Boseline
35.8
35.5
32.9
Ours
0.99
2.32
7.76
50.1
0.99
2.18
7.85
47.0
0.99
2.15
7.90
47.6
1.0
1.0
0.9
0.8
TPR 1% FPR
0.8
TPR @
Deletion Subsitution
0.6
Context-Aware
Original
0.5
Suositution
Paraphrasing
Back-Tranoiitton
0.0
0.1
0.2
0.3
0.4
0.5
50
100
150
200
260
300
350
400
Ratio
Number of Tokens
Figure 3: Robustness Evaluation of Our Watermark (Left) We measure the detectability of our watermark (TPR @ 1) against an increasing percentage of local modifications, using responses generated from LLADA-8B with an average length of 275 tokens. (Right) For stronger adversaries, we measure the detectability of our watermark (TPR I) with respect to the length the sequence. For both figures, we use 5 4 and the previous token context (C (-1)).
autoregressive remasking, while there is still slight improvement over the baseline, the difference is minimal. Fig. (right) shows that, with our approach and 8 = 4. detectability increases quickly with the length of the generated sequence. Importantly, given comparable quality, our approach at ss 50 tokens has the same detectability as the baseline at 2 350.
Table shows that the conclusions hold for DREAM-7B and for different choices of the context C: our watermark provides significantly better detectability than the baselines given similar impact on quality (according to log perplexity and average the accuracies over MMLU (Hendryeks et al., 2021), HellaSwag (Zellers et al., 2019), ARC- (Clark al., 2018), GPQA (Rein et al., 2024), MBPP (Austin al., 2021b), and GSM8K (Cobbe et al., 2021). We find that our approach has small impact on accuracy with DREAM-7B compared to the baselines, whereas with LLADA-S -BB the baselines have a lower impact on accuracy than our watermark (though at a much lower TPR @ I). This means that, with some combination of models and watermark hyperparameters, our approach may still induce a neg-negligible drop in accuracy. We analyze the benchmark results further in App. C.5. To reach 99% TPR @ s 300 tokens, practitioners should either use = 4 with a single token context, or 5 = 5 for larger contexts. Additionally, find in App. D that similar results also hold for infilling tasks. We apply watermark using DREAMON-VOL 7B, V0-7B, DLM with a diffusion process tailored for infilling, and reach 99% TPR@ @ I for the same hyperparameters.
score). For benchmark accuracy, we
4.2 ROBUSTNESS EVALUATION
As noted in prior works (Kirchenbauer et al., 2023; Kuditipudi et al.. 2024; Pan et al., 2024), key component of text watermarks is their robustness to various modifications. Using the toolkit provided in Pan et al. (2024), before running the watermark detection, we apply to each text of
8

---

Published as a conference paper at ICLR 2026
1.0
0.8
[-1]
0.8
[-1]
0.6
[-1.11
Componant
@
0.4
Hash
TPR @
0.4
Both
Expeciation
Mintlash
0.2
Prodictive bins
log/Perplexty)
log(Perpexxly]
Unwatermarked LLM
Unwatermanied
0.0
1.55
1.60
1.65
1,70
1.75
1.80
1.85
1.90
1.95
2,00
1.7
1.8
1.9
2.0
2.1
2.2
2.3
log(Perplexity)
(erpllle)))
1.0
1.0
0.8
TPR 1% FPR
# Ilerafions
TPR @ 1% FPP
C=[-1]
C=1 1,11
0.4
0.4
Parameterizatton
E-peramsterizatoo
|-1, 1) log( (g(Pppilyy)y
e-parameterization
0.2
log(Perilexlt) Unwalemarked LLM
Univatemmakeed LLM
0.0
1.55
1.70
1.75
1,80
1.5
1.8
2.0
log(Perplexitt)
log (Peeppllly))yy
Figure 4: Ablation of Our Watermark Components We compare the trade-off between watermark detectability (TPR@) 1) and text quality (log PPI.) of our approach with various hyperparameters, namely the hashing scheme (Top Left), the two components introduced in Sec. 3.3 (Top Right), the number of fixed-point iterations (Bottom Left) and the /6--parameterization explained in Sec. 3.2 (Bottom Right). Responses are generated by LLADA-80 8B with temperature 0.5 and 600 prompts.
the following transformations: word deletion, word substitution, context-aware word substitution (using BERT (Devlin et al., 2019)), paraphrasing (using GPT5-MINI), or back-translation (from English Standard Chinese and then back English, using GPT5-NANO translator).
Robustness to Local Modifications Fig. 3 (left) shows that, for the recommended parameters (C = (-1), 5 4) and sequence of length 300 tokens, the watermark retains strong detectability until up 30% of the sequence edited for word deletion and substitution. For both attacks, we use the same detector as Red-Green ARLM watermarks, we have similar robustness to local (random) edits. Importantly, our watermark significantly more robust to context-ane aware substitution (i.e., substituting words based on the context). This can be explained using our interpretation from Sec. 3.3: by applying our watermark expectation over the context hashes, we get as a side-effect that likely variations of the generated sequence are also (to some extent) watermarked.
Robustness Against Stronger Adversaries In prior works, paraphrasing and traassaanltiin (i.e., translating given text back and forth) pointed out strong adversaries to LLM watermarks. Similarly to ARLM watermarks, Fig. 3 (right) shows that both attacks hurt watermark detectability, Yet, we that, increase the number of tokens, recover stronger watermark signal.
4.3 COMPONENT ABLATION
We ablate different components of our watermark namely the hashing scheme (Sec. 3.2), the expecta- tion boost and predictive bias terms (Sec. 3.3), the number of iterations in the fixed-point convergence, and the E/6 parameterization (i.c., using the KL-constraint setting 5 directly (Sec. 3.2)).
Fig. 4 (top left) shows that the hashing scheme has no significant impact on watermark detectability, Fig. 4 (top right) shows that using both expectation boost and predictive bias terms together lead to better detectability given fixed distortion than using them separately. This confirms that the optimization formulation in Eq. (1) finds good watermarking strategy. As alluded to in Sec. 3.2, Fig. 4 (bottom left) shows that increasing the number of fixed fixed-point iterations leads to very marginal increase in the detectability, yet it linearly increases the watermark computation complexity.

---

Published as a conference paper at ICLR 2026
TPR @ 1% FPR
0.8
TPR @ 1%
0.4
Ours (C-
11)
Ours (C (-1,
Unigram
Unigram
PatlernMark (g((Prpplxttt)]
PatlemMark
0.2
ing/Perpexty)
Unwatemaried
1.9
23
1.8
2.3
c- lorpoorrplllliyt
-- log! Perplexity)
Figure 5: Detection Performance Comparison with Order-Agnostic Watermarks We study the trade-off between detectability (TPR@) @ 1) and text quality (log PPL) of our approach and order- agnostic watermarks for different values of the watermark strength parameter and sequences of, on average, 275 tokens. For the left figure, use C {-1), and for the right one, use (-1,1), For the order-agnostic watermarks, we use the same data for both figures.
Lastly, Fig. 4 (bottom right) shows that, surprisingly, e-parameterization leads to much worse detectabiliity This hints at the KL constraint (Eq. (1)) being an imperfect measure of text quality. To illustrate this, consider a model which at the current step predicts each of the two tokens t and with probability of 0.5, where both tokens equally contribute to text quality. Suppose that L is green and (2 is red. Ideal watermarking strategy entirely favors 4, maximizing text greenness with no impact on quality, However, a KL constraint would restrict the increase in the probability of t1. resulting in weaker watermark (i.c., a lower TPR) despite the same text quality. We hence suggest using 8 parameterization, unless KL-divergence guarantees are required.
4.4 COMPARISON WITH ORDER-AGNOSTIC WATERMARKS
In this section, we evaluate our watermark performance compared to order-agnostic watermarks, namely Unigram (Zhao et al., 2023) and PatternMark (Chen et al., 2025). We detail how each scheme works, and expand on their limitations (i.e., lack of Type-1 I error control and vulnerability to both spoofing and scrubbing) in App. Importantly, our approach does not suffer from these limitations.
Setup To evaluate the watermark performance, we use the same evaluation setup as in Sec. 4.1. For Unigram, we use 7 0.25. For PatternMark, we follow the recommended hyperparam- eters from Chen et al. (2025). We use 1 = 2. Q 0,, 0). (0, 0, 1)}. This corresponds alternately boosting red and then green tokens based on the parity of the token position.
A ((0. 1), 0)), and
Our Watermark is More Performant than Prior Order-Agnostic Watermarks Fig. shows that our approach provides better detectability than both order-agnostic watermarks given the same impact on quality. We find that the improvement especially noticeable in the low-distortion regime. We hypothesize that this is the case because, unlike both baselines, our watermark is designed to specifically leverage the whole- sequeee distribution to determine which tokens to boost (through the expectation boost and predictive bias terms, see Sec. 3.3), and in the low-distortion setting such optimization matters the most. contrast, both order-agnostic watermarks only leverage the distribution of single token. Additionally, explained in App. B, Unigram in particular suffers significant drawbacks with respect to its reliability (i.e., Type-I error is not controlled) and security (i.e., it is highly vulnerable to spoofing and scrubbing), which significantly hinders its practicality.
5 CONCLUSION
In this paper, we have introduced the first practical and effective watermark tailored for DLMs. We developed principled theoretical framework for our watermarking scheme, proposed practical implementation of the scheme, and linked it to prior work on watermarks for ARLMs. Our results show that watermark is effective (more than 99% TPR@1 1 with low impact text quality) and robust against natural text modifications (c.g., substituting words in context) and with longer sequences against stronger adversaries (e.g., paraphrasing or back-translating..
10

---

Published as a conference paper at ICLR 2026
ETHICS STATEMENT
Our work has a positive societal impact by enabling the traceability of DLMM-generaed text. Although public disclosure may increase the risk of targeted attacks on LLM watermarks, we believe the benefits of advancing research in this area clearly outweigh the risks.
REPRODUCIBILITY STATEMENT
To ensure reproducibility of our experiments, we provide all the necessary details in App. A, as well the hurdware-speciffe resources we used in App. I.I.

---

Published as a conference paper at ICLR 2026
REFERENCES
Scott Aaronson. Watermarking of large language models. In Workshop on Large Language Models and Transformers, Simons Institute, UC Berkeley, 2023.
Marianne Arriola, Aaron Gokaslan, Justin T Chiu, Zhihan Yang, Zhixuan Qi, Jiaqi Han, Sub- ham Sekhar Sahoo, and Volodymyr Kuleshov. Block diffusion: Interpolating between autoregres- sive and diffusion language models. arXiv preprint arXiv:2503.09577 2025.
Jacob Austin, Daniel D Johnson, Jonathan Ho, Daniel Tarlow, and Rianne Van Den Berg. Structured denoising diffuusin models in discrete state-spaces. Advances in neural information processing systems, 34:1 1981-199993 2021a.
Jacob Austin, Augustus Odena, Maxwell Nye, Maarten Bosma, Henryk Michalewski, David Dohan, Ellen Jiang, Carrie Cai, Michael Terry, Quoe Le, et al. Program synthesis with large language models. arXiv preprini ScKiv:200..077555 2021b.
Lukas Berglund, Meg Tong, Max Kaufmann, Mikita Balesni, Asa Cooper Stickland, Tomasz Korbak, and Owain Evans. The reversal curse: Lims trained on" is b" fail to learn" b is a". arXiv preprint 0rXiv:2309.12285 2023.
Adam Block, Alexander Rakhlin, and Ayush Sekhari. Gaussmark: A practical approach for structural watermarking of language models. In Forty-second International Conference on Machine Learning. 2025. URL
Tom Brown, Benjamin Mann, Nick Ryder, Melanie Subbiah, Jared D Kaplan, Prafulla Dhariwal, Arvind Neelakantan, Pranav Shyam, Girish Sastry, Amanda Askell, et Language models are few-shot learners. Advances in neural information processing systems, 33:1877-1901, 2020.
Sébastien Bubeck, Varun Chandrasekaran, Ronen Eldan, Johannes Gehrke, Eric Horvitz, Ece Kamar, Peter Lee, Yin Tat Lee, Yuanzhi Li, Scott M. Lundberg, Harsha Nori, Hamid Palangi, Marco Túlio Ribeiro, and Yi Zhang. Sparks of artificial general intelligence: Early experiments with GPT-4. arXiv,, 2023.
Edoardo Cetin, Tianyu Zhao, and Yujin Tang. Large language models to diffusion finetuning. 2025.
Ruibo Chen, Yihan Wu, Yanshuo Chen, Chenxi Liu, Junfeng Guo, and Heng Huang. A watermark for arder-agnostic language models. In The Thirteenth International Conference on Learning Representations, 2025. URL https://openrewie www..nee/forumfi
net/
Zhiyu Chen, Wenhu Chen, Charese Smiley, Sameena Shah, Iana Borova, Dylan Langdon, Reema Moussa, Matt Beane, Ting-Hao Huang, Bryan Routledge, al. Finqa: A dataset of numerical reasoning over financial data. In Proceedings of the 2021 Conference on Empirical Methods in Natural Language Processing, pp. 3697-3711, 2021.
Miranda Christ, Sam Gunn, Tal Malkin, and Mariana Raykova. Provably robust watermarks for open-source language models. arXiv, 2024.
Peter Clark, Isaac Cowhey, Oren Etzioni, Tushar Khot, Ashish Sabharwal, Carissa Schoenick, and Oyvind Tafjord. Think you have solved question answering? try are, the ai2 reasoning challenge. ArXiv, abs/l 803.054577 2018.
Karl Cobbe, Vineet Kosaraju, Mohammad Bavarian, Mark Chen, Heewoo Jun, Lukasz Kaiser, Matthias Plappert, Jerry Tworek, Jacob Hilton, Reiichiro Nakano, Christopher Hesse, and John Schulman. Training verifiers to solve math word problems. arXiv preprint arXiv:2110.14163 2021.
OpenCompass Contributors. Opencompass: A universal evaluation platform for foundation models.
htttps://githinn com/ open/ compasss/opencompass,, 2023.
Andrew M Dai and Quoc Le. Semi-supervised sequence learning. Advances in neural information processing systems, 28, 2015.
12

---

Published as a conference paper at ICLR 2026
Sumanth Dathathri, Abigail See, Sumedh Ghaisas, Po-Sen Huang, Rob McAdam, Johannes Welbl, Vandana Bachani, Alex Kaskasoli, Robert Stanforth, Tatiana Matejovicova, et al. Scalable water- marking for identifying large language model outputs. Nature, 634(8035):818 823, 2024.
Jacob Devlin, Ming- Ming-Wei Chang, Kenton Lee, and Kristina Toutanova. Bert: Pre-training of deep bidirectional transformers for language understanding, 2019. URL https::/arxxv..rrg/ 1810. 04885.
Prafulla Dhariwal and Alexander Nichol. Diffusion models beat gans on image synthesis. Advances in neural information processing systems, 34:8780-8794, 2021.
Yann Dubois, Xuechen Li, Rohan Taori, Tianyi Zhang, Ishaan Gulrajani, Jimmy Ba, Carlos Guestrin, Percy Liang, and Tatsunori B. Hashimoto. Alpacafarm: A simulation framework for methods that learn from human feedback, 2024. httpss:/arxxvv.rrrgg////////// httg://abb/2055133388
EU Council. Proposal for a regulation of the european parliament and of the council laying down harmonised rules on artificial intelligence (artificial intelligence act) and amending certain union legislative acts analysis of the final compromise text with view to agreement. 2024.
Angela Fan, Yacine Jernite, Ethan Perez, David Grangier, Jason Weston, and Michael Auli. ELI5: Long form question answering. In Anna Korhonen, David Traum, and Lluis Mårquez (eds.), Proceedings of the 57th Annual Meeting of the Association for Computational Linguistics, pp. 3558 3567, Florence, Italy, July 2019. Association for Computational Linguistics. doi: 10.18653/ v1/P19- 1346. URL htaps::/aaalattie logy. org/ 1346/.
Pierre Fernandez, Antoine Chaffin, Karim Tit, Vivien Chappelier, and Teddy Furon. Three bricks to consolidate watermarks for large language models, 2023a. URL http:: htttt::://axxiv..o/ 2t/////rr..v..rrr/////3..0.. 00113.
Pierre Fernandez, Guillaume Couairon, Hervé Jégou, Matthijs Douze, and Teddy Furon. The stable signature: Rooting watermarks latent diffusion models. Proceedings of the IEEE/CVF International Conference Computer Vision, pp. 22447772477, 2023b.
Leo Gao, Jonathan Tow, Baber Abbasi, Stella Biderman. Sid Black, Anthony DiPofi, Charles Foster, Laurence Golding, Jeffrey Hsu, Alain Le Noac h, Haonan Li, Kyle McDonell, Niklas Muennighoff, Chris Ociepa, Jason Phang. Laria Reynolds, Hailey Schoelkopf, Aviya Skowron, Lintang Sutawika, Eric Tang, Anish Thite, Ben Wang, Kevin Wang, and Andy Zou. The language model evaluation harness, 07 2024. URL https:: www.eondddoog/rroorrroooooo wtw.rreoordd/112666100/
Thibaud Gloaguen, Nikola Jovanović, Robin Staab, and Martin Vechev. Black-box detection of language model watermarks. In The Thirteenth International Conference on Learning Representa- tions.
Thibaud Gloaguen, Nikola Jovanović, Robin Staab, and Martin Vechev. Towards watermarking of open-source Ilms, 2025. URL
Fabian Gloeckle, Badr Youbi Idrissi, Baptiste Rozière, David Lopez-Paz, and Gabriel Synnaeve. Better faster large language models via multi-token prediction. arXiv preprint arXiv:2404. 19737, 2024.
Aaron Grattafiori, Abhimanyu Dubey, Abhinav Jauhri, Abhinav Pandey, Abhishek Kadian, Ahmad Al-Dahle, Aiesha Letman, Akhil Mathur, Alan Schelten, Alex Vaughan, Amy Yang, Angela Fan, Anirudh Goyal, Anthony Hartshorn, Aobo Yang, Archi Mitra, Archie Sravankumar, Artem Korenev, Arthur Hinsvark, Arun Rao, Aston Zhang, Aurelien Rodriguez, Austen Gregerson, Ava Spataru, Baptiste Roziere, Bethany Biron, Binh Tang, Bobbie Chern, Charlotte Caucheteux, Chaya Nayak, Chloe Bi, Chris Marra, Chris McConnell, Christian Keller, Christophe Touret, Chunyang Wu, Corinne Wong. Cristian Canton Ferrer, Cyrus Nikolaidis, Damien Allonsius, Daniel Song. Danielle Pintz, Danny Livshits, Danny Wyatt, David Esiobu, Dhruv Choudhary, Dhruv Mahajan, Diego Garcia-Olano, Diego Perino, Dieuwke Hupkes, Egor Lakomkin, Ehab AlBadawy, Elina Lobanova, Emily Dinan, Eric Michael Smith, Filip Radenovic, Francisco Guzmán, Frank Zhang, Gabriel Synnaeve, Gabrielle Lee, Georgia Lewis Anderson, Govind Thattai, Graeme Nail, Gregoire Mialon, Guan Pang, Guillem Cucurell, Hailey Nguyen, Hannah Korevaar, Hu Xu, Hugo Touvron, Iliyan
13

---

Published as a conference paper at ICLR 2026
Zarov, Imanol Arrieta Ibarra, Isabel Kloumann, Ishan Misra, Ivan Evtimov, Jack Zhang, Jade Copet, Jaewon Lee, Jan Geffert, Jana Vranes, Jason Park, Jay Mahadeokar, Jeet Shah, Jelmer van der Linde, Jennifer Billock, Jenny Hong, Jenya Lee. Jeremy Fu, Jianfeng Chi, Jianyu Huang, Jiawen Liu, Jie Wang, Jiecao Yu, Joanna Bitton, Joe Spisak, Jongsoo Park, Joseph Rocca, Joshua Johnstun, Joshua Saxe, Junteng Jia, Kalyan Vasuden Alwala, Karthik Prasad, Kartikeya Upasani, Kate Plawiak, Ke Li, Kenneth Heafield, Kevin Stone, Khalid El-Arini, Krithika Iyer, Kshitiz Malik, Kuenley Chiu, Kunal Bhalla, Kushal Lakhotia. Lauren Rantala-Yeant Laurens van der Maaten, Lawrence Chen, Liang Tan, Liz Jenkins, Louis Martin, Lovish Madaan, Lubo Malo, Lukas Blecher, Lukas Landzaat, Luke de Oliveira, Madeline Muzzi, Mahesh Pasupuleti, Mannat Singh, Manohar Paluri, Marcin Kardas, Maria Tsimpoukelli, Mathew Oldham, Mathieu Rita, Maya Pavlova, Melanie Kambadur, Mike Lewis, Min Si, Mitesh Kumar Singh, Mona Hassan, Naman Goyal, Narjes Torabi, Nikolay Bashlykov, Nikolay Bogoychev, Niladri Chatterji, Ning Zhang, Olivier Duchenne, Onur Çelebi. Patrick Alrassy, Pengchuan Zhang, Pengwei Li, Petar Vasic, Peter Weng, Prajjwal Bhargava. Pratik Dubal, Praveen Krishnan, Punit Singh Koura, Puxin Xu, Qing He, Qingxiao Dong, Ragavan Srinivasan, Raj Ganapathy, Ramon Calderer, Ricardo Silveira Cabral, Robert Stojnic, Roberta Raileanu, Rohan Maheswari, Rohit Girdhar, Rohit Patel, Romain Sauvestre, Ronnie Polidoro, Roshan Sumbaly, Ross Taylor, Ruan Silva, Rui Hou, Rui Wang, Saghar Hosseini, Sahana Chennabasappa. Sanjay Singh, Sean Bell, Seohyun Sonia Kim, Sergey Edunov, Shaoliang Nie, Sharan Narang, Sharath Raparthy, Sheng Shen, Shengye Wan, Shruti Bhosale, Shun Zhang, Simon Vandenhende, Soumya Batra, Spencer Whitman, Sten Sootla, Stephane Collot, Suchin Gururangan. Sydney Borodinsky, Tamar Herman, Tara Fowler, Tarek Sheasha, Thomas Georgiou. Thomas Scialom, Tobias Speckbacher, Todor Mihaylov, Tong Xiao, Ujjwal Karn, Vedanuj Goswami, Vibhor Gupta, Vignesh Ramanathan, Viktor Kerkez, Vincent Gonguet, Virginie Do, Vish Vogeti, Vitor Albiero, Vladan Petrovic, Weiwei Chu, Wenhan Xiong, Wenyin Fu, Whitney Meers, Xavier Martinet, Xiaodong Wang, Xiaofang Wang, Xiaoqing Ellen Tan, Xide Xia, Xinfeng Xie, Xuchao Jia, Xuewei Wang, Yaelle Goldschlag, Yashesh Gaur, Yasmine Babaei, Yi Wen, Yiwen Song. Yuchen Zhang, Yue Li, Yuning Mao. Zacharie Delpierre Coudert, Zheng Yan, Zhengxing Chen, Zoe Papakipos, Aaditya Singh, Aayushi Srivastava, Abha Jain, Adam Kelsey, Adam Shajnfeld, Adithya Gangidi, Adolfo Victoria, Ahuva Goldstand, Ajay Menon, Ajay Sharma, Alex Boesenberg, Alexei Baevski, Allie Feinstein, Amanda Kallet, Amit Sangani, Amos Teo, Anam Yunus, Andrei Lupu, Andres Alvarado, Andrew Caples, Andrew Gu, Andrew Ho, Andrew Poulton, Andrew Ryan, Ankit Ramchandani, Annie Dong, Annie Franco, Anuj Goyal, Aparajita Saraf, Arkabandhu Chowdhury, Ashley Gabriel, Ashwin Bharambe, Assaf Eisenman, Azadeh Yazdan, Beau James, Ben Maurer, Benjamin Leonhardi, Bernie Huang, Beth Loyd, Beto De Paola, Bhargavi Paranjape, Bing Liu, Bo Wu, Boyu Ni, Braden Hancock, Bram Wasti, Brandon Spence, Brani Stojkovic, Brian Gamido, Britt Montalvo, Carl Parker, Carly Burton, Catalina Mejia, Ce Liu, Changhan Wang. Changkyu Kim, Chao Zhou, Chester Hu, Ching-Hsiane
Chu, Chris Cai, Chris Tindal, Christoph Feichtenhofer, Cynthia Gao, Damon Civin, Dana Beaty, Daniel Kreymer, Daniel Li, David Adkins. David Xu, Davide Testuggine, Delia David, Devi Parikh, Diana Liskovich, Didem Foss, Dingkang Wang. Duc Le, Dustin Holland, Edward Dowling, Eissa Jamil, Elaine Montgomery, Eleonora Presani, Emily Hahn, Emily Wood, Eric-Tuan Le, Erik Brinkman, Esteban Arcaute, Evan Dunbar, Evan Smothers, Fei Sun, Felix Kreuk, Feng Tian, Filippos Kokkinos, Firat Ozgenel, Francesco Caggioni, Frank Kanayet, Frank Seide, Gabriela Medina Florez, Gabriella Schwarz, Gada Badeer, Georgia Swee, Gil Halpem. Grant Herman, Grigory Sizov, Guangyi. Zhang. Guna Lakshminarayanan Hakan Inan, Hamid Shojanazeri, Han Zou, Hannah Wang, Hanwen Zha, Haroun Habeeb, Harrison Rudolph, Helen Suk, Henry Aspegren, Hunter Goldman, Hongyuan Zhan, Ibrahim Damlaj. Igor Molybog, Igor Tufanov, Ilias Leontiadis, Irina-Flena Veliche, Itai Gat, Jake Weissman, James Geboski, James Kohli, Janice Lam, Japhet Asher, Jean-Baptiste Gaya, Jeff Marcus, Jeff Tang, Jennifer Chan, Jenny Zhen, Jeremy Reizenstein, Jeremy Teboul, Jessica Zhong, Jian Jin, Jingyi Yang, Joe Cummings, Jon Carvill, Jon Shepard, Jonathan McPhie, Jonathan Torres, Josh Ginsburg, Junjie Wang, Kai Wu, Kam Hou U, Karan Saxena, Kartikay Khandelwal, Katayoun Zand, Kathy Matosich, Kaushik Veeraraghavan, Kelly Michelena, Keqian Li, Kiran Jagadeesh, Kun Huang, Kunal Chawla, Kyle Huang, Lailin Chen, Lakshya Garg, Lavender A, Leandro Silva, Lee Bell, Lei Zhang, Liangpeng Guo, Licheng Yu, Liron Moshkovich, Luca Wehrstedt, Madian Khabsa, Manay Avalani, Manish Bhatt, Martynas Mankus, Matan Hasson, Matthew Lennie, Matthias Reso, Maxim Groshev, Maxim Naumov, Maya Lathi, Meghan Keneally, Miao Liu, Michael L. Seltzer, Michal Valko, Michelle Restrepo, Mihir Patel, Mik Vyatskov, Mikayel Samvelyan, Mike Clark, Mike Macey, Mike Wang, Miquel Jubert Hermoso, Mo Metanat, Mohammad Rastegari, Munish Bansal, Nandhini Santhanam, Natascha Parks, Natasha White, Navyata Bawa, Nayan Singhal, Nick Egebo, Nicolas Usunier,
14

---

Published as a conference paper at ICLR 2026
Nikhil Mehta, Nikolay Pavlovich Laptev, Ning Dong, Norman Cheng, Oleg Cherooguz Olivia Hart, Omkar Salpekar, Ozlem Kalinli, Parkin Kent, Parth Parekh, Paul Saab, Pavan Balaji, Pedro Rittner, Philip Bontrager, Pierre Roux, Piotr Dollar, Polina Zvyagina, Prashant Ratanchandani, Pritish Yuvraj, Qian Liang, Rachad Alao, Rachel Rodriguez, Raf Ayub, Raghotham Murthy, Raghu Nayani, Rahul Mitra, Rangaprabhu Parthasarathy, Raymond Li, Rebekkah Hogan, Robin Battey, Rocky Wang, Russ Howes, Ruty Rinott, Sachin Mehta, Sachin Siby, Sai Jayesh Bondu, Samyak Datta, Sara Chugh, Sara Hunt, Sargun Dhillon, Sasha Sidorov, Satadru Pan, Saurabh Mahajan, Saurabh Verma, Seiji Yamamoto, Sharadh Ramaswamy, Shaun Lindsay, Shaun Lindsay, Sheng Feng, Shenghao Lin, Shengxin Cindy Zha, Shishir Patil, Shiva Shankar, Shuqiang Zhang. Shuqiang Zhang, Sinong Wang, Sneha Agarwal, Soji Sajuyighe, Soumith Chintala, Stephanie Max, Stephen Chen, Steve Kehoe, Steve Satterfield, Sudarshan Govindaprasad, Sumit Gupta, Summer Deng. Sungmin Cho, Sunny Virk, Suraj Subramanian, Sy Choudhury, Sydney Goldman, Tal Remez, Tamar Glaser, Tamara Best, Thilo Koehler, Thomas Robinson, Tianhe Li, Tianjun Zhang. Tim Matthews, Timothy Chou, Tzook Shaked, Varun Vontimitta, Victoria Ajayi, Victoria Montanez, Vijai Mohan, Vinay Satish Kumar, Vishal Mangla, Vlad lonescu, Vlad Poenaru, Vlad Tiberiu Mihailescu, Vladimir Ivanov, Wei Li, Wenchen Wang. Wenwen Jiang, Wes Bouaziz, Will Constable, Xiaocheng Tang, Xiaojian Wu, Xiaolan Wang, Xilun Wu, Xinbo Gao, Yaniv Kleinman, Yanjun Chen, Ye Hu, Ye Jía, Ye Qi, Yenda Li, Yilin Zhang, Ying Zhang, Yossi Adi, Youngjin Nam, Yu, Wang, Yu Zhao, Yuchen Hao, Yundi Qian, Yunlu Li, Yuzi He, Zach Rait, Zachary De Zef Rosnbrick, Zhaoduo Wen, Zhenyu Yang. Zhiwei Zhao, and Zhiyu Ma. The llama 3 herd of models, 2024. URL https //arxiv. www.aasgg/////////..//../ooeeeet abtaaa////2/// 21783.
Dan Hendrycks, Collin Burns, Steven Basart, Andy Zou, Mantas Mazeika, Dawn Song, and Jacob Steinhardt. Measuring massive multitask language understanding. Proceedings of the International Conference on Learning Representations (ICLR), 2021.
Jonathan Ho, Ajay Jain, and Pieter Abbeel. Denoising diffusion probabilistic models. Advances in neural information processing systems, 33:6840-6851, 2020.
Nikola Jovanovié, Robin Staab, and Martin Vechev. Watermark stealing in large language models. 2024.
Nikola Jovanović, Ismail Labiad, Tomáš Souëek, Martin Vechev, and Pierre Fernandez. Watermarking autoregressive image generation. Advances in Neural Information Processing Systems, 2025.
Jacyeon Kim, Kulin Shah, Vasilis Kontonis, Sham Kakade, and Sitan Chen. Train for the worst, plan for the best: Understanding token ordering in masked diffusions. arXiv preprint arXiv:2502.06768, 2025.
John Kirchenbauer, Jonas Geiping, Yuxin Wen, Jonathan Katz, Ian Miers, and Tom Goldstein. A watermark for large language models. In ICML, 2023.
John Kirchenbauer, Jonas Geiping, Yuxin Wen, Manli Shu, Khalid Saifullah, Kezhi Kong, Kasun Fernando, Aniruddha Saha, Micah Goldblum, and Tom Goldstein. On the reliability of watermarks for large language models. ICLR, 2024.
Rohith Kuditipudi, John Thickstun, Tatsunori Hashimoto, and Percy Liang. Robust distortion-free watermarks for language models, 2024. URL https://arxiv.. https/////aarvvooo/a//////// obg//b//237 15593.
Inception Labs, Samar Khanna, Siddhant Kharbanda, Shufan Li, Harshit Varma, Eric Wang, Sawyer Birnbaum, Ziyang Luo, Yanis Miraoui, Akash Pairecha, Stefano Ermon, Aditya Grover, and Volodymyr Kuleshov. Mercury: Ultra-fast language models based on diffusion, 2025. URL https://arxiv.cr org/ http///5.66..1.....
Aiwei Liu, Sheng Guan, Yiming Liu, Leyi Pan, Yifei Zhang, Liancheng Fang, Lijie Wen, Philip Yu, and Xuming Hu. Can watermarked Iims be identified by users via crafted prompts? In ICLR, 2025a.
Runheng Liu, Heyan Huang, Xingchen Xiao, and Zhijing Wu. Zero-shot detection of LLM-generated text via implicit reward model. In The Thith Annual Conference on Neural Information Processing Systems, 2025b. http://////pppeeeeeeeeee/orrrrra
/
15

---

Published as a conference paper at ICLR 2026
Aaron Lou and Stefano Ermon. Reflected diffusion models. In International Conference on Machine Learning, pp. 227711.2701. PMLR, 2023.
Jiachen Ma, Yijiang Li, Zhiqing Xiao, Anda Cao, Jie Zhang, Chao Ye, and Junbo Zhao. Jailbreak- ing prompt attack: A controllable adversarial attack against diffusion models. arXiv preprint arXiv:2404.02928, 2024.
Shixuan Ma and Quan Wang. Zero-shot detection of LLM-generated text using token cohesiveness. In Yaser Al-Onaizan, Mohit Bansal, and Yun-Nung Chen (eds.), Proceedings of the 2024 Conference on Empirical Methods in Natural Language Processing, pp. 17538-17553, Miami, Florida, USA, November 2024. Association for Computational Linguistics. doi: 10. (0.18663/112224.emmll main. 971. URL htttss:/aclaatthlll.. wwwpthhooooo.oog//2200 wrw//024. emnlp- 1p main.9971
Eric Mitchell, Yoonho Lee, Alexander Khazatsky, Christopher D Manning, and Chelsea Finn. Detectgpt: Zero-shot machine-generated text detection using probability curvature. In International conference on machine learning, pp. 24950-24962. PMLR, 2023.
Alexander Quinn Nichol and Prafulla Dhariwal. Improved denoising diffusion probabilistic models. In International conference on machine learning, pp. 8162-8171, PMLR, 2021.
Shen Nie, Fengqi Zhu, Zebin You, Xiaolu Zhang, Jingyang Ou, Jun Hu, Jun Zhou, Yankai Lin, Ji- Rong Wen, and Chongxuan Li. Large language diffusion models. arXiv preprint 099992,.0,..999. 2025.
OpenAI, Josh Achiam, Steven Adler, Sandhini Agarwal, Lama Ahmad, llge Akkaya, Florencia Leoni Aleman, Diogo Almeida, Janko Altenschmidt, Sam Altman, Shyamal Anadkat, Red Avila, Igor Babuschkin, Suchir Balaji, Valerie Balcom, Paul Baltescu, Haiming Bao, Mohammad Bavarian, Jeff Belgum, Irwan Bello, Jake Berdine. Gabriel Sernadett-Shapirg Christopher Berner, Lenny Bogdonoff, Oleg Boiko, Madelaine Boyd, Anna-Luisa Brakman, Greg Brockman, Tim Brooks, Miles Brundage, Kevin Button, Trevor Cai, Rosie Campbell, Andrew Cann, Brittany Carey, Chelsea Carlson, Rory Carmichael, Brooke Chan, Che Chang, Fotis Chantzis, Derek Chen, Sully Chen, Ruby Chen, Jason Chen, Mark Chen, Ben Chess, Chester Cho, Casey Chu, Hyung Won Chung, Dave Cummings, Jeremiah Currier, Yunxing Dai, Cory Decareaux, Thomas Degry, Noah Deutsch, Damien Deville, Arka Dhar, David Dohan, Steve Dowling, Sheila Dunning, Adrien Ecoffet, Atty Eleti, Tyna Eloundou, David Farhi. Liam Fedus, Niko Felix, Simón Posada Fishman, Juston Forte, Isabella Fulford, Leo Gao, Elie Georges, Christian Gibson, Vik Goel, Tarun Gogineni, Gabriel Goh, Rapha Gontijo-Lopess, Jonathan Gordon, Morgan Grafstein, Scott Gray, Ryan Greene, Joshua Gross, Shixiang Shane Gu, Yufei Guo, Chris Hallacy, Jesse Han, Jeff Harris, Yuchen He. Mike Heaton, Johannes Heidecke, Chris Hesse, Alan Hickey, Wade Hickey, Peter Hoeschele, Brandon Houghton, Kenny Hsu, Shengli Hu, Xin Hu, Joost Huizinga, Shantanu Jain, Shawn Jain, Joanne Jang, Angela Jiang, Roger Jiang, Haozhun Jin, Denny Jin, Shino Jomoto. Billie Jonn, Heewoo Jun, Tomer Kaftan, Lukasz Kaiser, Ali Kamali, Ingmar Kanitscheider, Nitish Shirish Keskar, Tabarak Khan, Logan Kilpatrick, Jong Wook Kim, Christina Kim, Yongjik Kim, Jan Hendrik Kirchner, Jamie Kiros, Matt Knight, Daniel Kokotajlo, Fukasz Kondraciuk, Andrew Kondrich, Aris Konstantinidis, Kyle Kosic, Gretchen Krueger, Vishal Kuo, Michael Lampe. Ikai Lan, Teddy Lee, Jan Leike, Jade Leung, Daniel Levy, Chak Ming Li, Rachel Lim, Molly Lin, Stephanie Lin, Mateusz Litwin, Theresa Lopez, Ryan Lowe, Patricia Lue, Anna Makanju, Kim Malfacini, Sam Manning, Todor Markov, Yaniv Markovski, Bianca Martin, Katie Mayer, Andrew Mayne, Bob McGrew, Scott Mayer McKinney, Christine McLeavey, Paul McMillan, Jake McNeil, David Medina, Aalok Mehta, Jacob Menick, Luke Metz, Andrey Mishchenko, Pamela Mishknn, Vinnie Monaco, Evan Morikawa, Daniel Mossing, Tong Mu, Mira Murati, Oleg Murk, David Mély, Ashvin Nair, Reiichiro Nakano, Rajeev Nayak, Arvind Neelakantan, Richard Ngo. Hyeonwoo Noh, Long Ouyang. Cullen Keefe. Jakub Pachocki, Alex Paino, Joe Palermo, Ashley Pantuliano, Giambattista Parascandolo, Joel Parish, Emy Parparita, Alex Passos, Mikhail Pavlov, Andrew Peng, Adam Perelman, Filipe de Avila Belbute Peres, Michael Petrov, Henrique Ponde de Oliveira Pinto, Michael, Pokorny, Michelle Pokrass, Vitchyr H. Pong, Tolly Powell, Alethea Power, Boris Power, Elizabeth Prochl, Raul Puri, Alec Radford, Jack Rae, Aditya Ramesh, Cameron Raymond, Francis Real, Kendra Rimbach, Carl Ross, Bob Rotsted, Henri Roussez, Nick Ryder, Mario Saltarelli, Ted Sanders, Shibani Santurkar, Girish Sastry, Heather Schmidt, David Schnurr, John Schulman, Daniel Selsam, Kyla Sheppard, Toki Sherbakov, Jessica Shich, Sarah Shoker, Pranav Shyam, Szymon Sidor, Eric Sigler, Maddie Simens, Jordan Sitkin, Katarina Slama, Ian Sohl, Benjamin Sokolowsky, Yang Song, Natalie
16

---

Published as a conference paper at ICLR 2026
Staudacher, Felipe Petroski Such, Natalie Summers, Ilya Sutskever, Jie Tang, Nikolas Tezak, Madelcine B. Thompson, Phil Tillet, Amin Tootoonchian. Elizabeth Tseng, Preston Tuggle, Nick Turley, Jerry Tworek, Juan Felipe Cerón Uribe, Andrea Vallone, Arun Vijayvergiya, Chelsea Voss, Carroll Wainwright, Justin Jay Wang, Alvin Wang, Ben Wang, Jonathan Ward, Jason Wei, CJ Weinmann, Akila Welihinda, Peter Welinder, Jiayi Weng, Lilian Weng, Matt Wiethoff, Dave Willner, Clemens Winter, Samuel Wolrich, Hannah Wong. Lauren Workman, Sherwin Wu, Jeff Wu, Michael Wu, Kai Xiao, Tao Xu, Sarah Yoo, Kevin Yu, Qiming Yuan, Wojciech Zaremba, Rowan Zellers, Chong Zhang, Marvin Zhang, Shengjia Zhao, Tianhao Zheng, Juntang Zhuang, William Zhuk, and Barret Zoph. Gpt-4 technical report, 2024. URL https: / http://aavvvvoooaaa../ wwt.:/ogggbbbb...../// 087744
Jingyang Ou, Shen Nic, Kaiwen Xue, Fengqi Zhu, Jiacheng Sun, Zhenguo Li, and Chongxuan Li. Your absorbing discrete diffusion secretly models the conditional distributions of clean data. arXiv preprint ar arXiv:2406. 03736, 2024.
Leyi Pan, Aiwei Liu. Zhiwei He, Zitian Gao, Xuandong Zhao, Yijian Lu, Binglin Zhou, Shuliang Liu, Xuming Hu, Lijie Wen, Irwin King, and Philip S. Yu. Markllm: An open-source toolkit for IIm watermarking, 2024. URL https: /ttpxiv.///iii.//// org/ /b//////////////// 10051.
Alec Radford, Karthik Narasimhan, Tim Salimans, Ily Sutskever, et al. Improving language understanding by generative pre-training. 2018.
Alec Radford, Jeffrey Wu, Rewon Child, David Luan, Dario Amodei, Ilya Sutskever, et al. Language models are unsupervised multitask learners. OpenAl blog, 1(8):9, 2019.
Colin Raffel, Noam Shazeer, Adam Roberts, Katherine Lee, Sharan Narang, Michael Matena, Yanqi Zhou, Wei Li, and Peter J Liu. Exploring the limits of transfer learning with a unified text-to-text transformer. Journal of machine learning research, 21(140): 2020.
David Rein, Betty Li Hou, Asa Cooper Stickland, Jackson Petty, Richard Yuanzhe Pang, Julien Dirani, Julian Michael, and Samuel R Bowman. Gpqa: A graduate-level google-proof q&a benchmark, In First Conference on Language Modeling, 2024.
Subham Sahoo, Marianne Arriola, Yair Schiff, Aaron Gokaslan, Edgar Marroquin, Justin Chiu, Alexander Rush, and Volodymyr Kuleshov. Simple and effective masked diffusion language models. Advances in Neural Information Processing Systems, 37:130136-1301844 2024.
Subham Sekhar Sahoo, Justin Deschenaux, Aaron Gokaslan, Guanghan Wang, Justin Chiu, and Volodymyr Kuleshov. The diffusion duality. In ICML, 2025.
Jiaxin Shi, Kehang Han, Zhe Wang, Arnaud Doucet, and Michalis Titsias. Simplified and generalized masked diffusion for discrete data. Advances in neural information processing systems, 37: 103131-103167, 2024.
Jammin Song, Chenlin Meng, and Stefano Ermon. Denoising diffusion implicit models. arXiv preprint arXiv:2010.02500 2020.
Dayu Tan, Pengyuan Xu, Xin Xia, Yajie Zhang, Chunhou Zheng, and Yansen Su. A latent diffusion model for molecular optimization. In International Conference on Intelligent Computing, pp. 135-146. Springer, 2025.
Shangqing Tu, Yuliang Sun, Yushi Bai, Jifan Yu, Lei Hou, and Juanzi Li. Waterbench: Towards holistic evaluation of watermarks for large language models, 2024. URL https::/arráv.orrg abs/2311. 07138.
Yuxin Wen, John Kirchenbauer, Jonas Geiping, and Tom Goldstein. Tree-ring watermarks: Fin- gerprints for diffusion images that are invisible and robust. arXiv preprint arX1v:2305.20030 2023.
Zirui Wu, Lin Zheng, Zhihui Xie, Jiacheng Ye, Jiahui Gao, Yansong Feng, Zhenguo Li, Victoria W., Guorui Zhou, and Lingpeng Kong. Dreamon: Diffusion language models for code infilling beyond fixed-size canvas, 2025. URL https:://kkkn w.p.gittiie... github.io/blog..... wtt.bl////////oo httpbblogg/2/////// (0025/ddeame reamon.
Yijie Xu, Aiwei Liu, Xuming Hu, Lijie Wen, and Hui Xiong. Mark your Ilm: Detecting the misuse of open-source large language models via watermarking. arXiv preprint arXiv:2503.00 044366 2025.
17

---

Published as a conference paper at ICLR 2026
Ling Yang, Ye Tian, Bowen Li, Xinchen Zhang, Ke Shen, Yunhai Tong, and Mengdi Wang. Mmada: Multimodal large diffusion language models. arXiv preprint arXiv:2505. 15809, 2025.
Zijin Yang, Kai Zeng, Kejiang Chen, Han Fang, Weiming Zhang, and Nenghai Yu. Gaussian shading: Provable performance-lossless image watermarking for diffusion models. In Proceedings of the JEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 12162-12171, 2024.
Jiacheng Ye, Zhihui Xie, Lin Zheng, Jiahui Gao, Zirui Wu, Xin Jiang. Zhenguo Li, and Lingpeng Kong. Dream 7b, 2025. URL https: //hkunt Lp github. http://ooolo//////////
Zebin You, Shen Nic, Xiaolu Zhang, Jun Hu, Jun Zhou, Zhiwu Lu, Ji-Rong Wen, and Chongxuan Li. Llada-v: Large language diffusion models with visual instruction tuning. arXiv preprint ww9v 2025.
Rowan Zellers, Ari Holtzman, Yonatan Bisk, Ali Farhadi, and Yejin Choi. Hellaswag: Can a machine really finish your sentence? In Proceedings of the 57th Annual Meeting of the Association for Computational Linguistics, 2019.
Zhaoxi Zhang, Xiaomei Zhang, Yanjun Zhang, Leo Yu Zhang, Chao Chen, Shengshan Hu, Asif Gill, and Shirui Pan. Large language model watermark stealing with mixed integer programming. 2024. URL https::/arxdi..orrabb///////////
19677.
Xuandong Zhao, Prabhanjan Ananth, Lei Li, and Yu- Xu-Xiang Wang. Provable robust watermarking for ai-generated text, 2023. URL https://arxiv wwtp.//.../.bb.....1... 17439.
ismail Tarı and Aytuğ Onan. Can you detect the difference?, 2025. URL https://arxiv.orr abs/ abs/2507.0 10475.
18

---

Published as a conference paper at ICLR 2026
A OMITTED EXPERIMENTAL DETAILS A.1 MAIN EXPERIMENTAL DETAILS
In this part, we detail the default experimental setup for all of our experiments.
Watermark Evaluation To evaluate watermark detectability, we adapt the approach from WA- TERBENCH (Tu et al., 2024). We use as prompts the long-answer sub-tasks of WATERBENCIE 200 questions from the ELI-5 dataset (Fan et al., 2019), 200 questions from the FINANCE-QA dataset (Chen et al., 2021), and 200 questions from the ALPACA-FARIE dataset (Dubois et al., 2024). Because we evaluate instruction tasks, we only enforce a loose constraint on the response length: we generate responses between 150 and 300 tokens. For responses below the threshold, we simply rejection sampling. For the upper threshold, we simply set the masked sequence length 300 tokens: by design, this guarantees that the generated response is below 300 tokens. Additionally, because current DLMs (without watermarks) tend to be repetitive, a repeated 2-gram represents more than 50% of the sequence length, we reject the response. We did not observe increase in repetitiveness due to the addition of the watermark.
add filter on repetitiveness: if
Watermark Parameters For the watermark, we sample our green list matrix G from i.i.d. Bernoulli random variables that are green with probability 7 0.25. This is slightly different from the autoregressive Red-Green watermarks from Kirchenbauer al. (2023), where the green list is composed of correlated Bernoulii random variables to enforce that the green list size is exactly While we deviate from the correlated Bernoulli for technical reasons (it faster sample i.i.d. Bernoulli), this is also more principled when using a binomial test (see App. F). We use the SumHash hashing scheme, d-parameterivzttio, a single iteration for the fixed-point convergence, and top-k of 50. For d- parameterization, to reduce the dependence of å on the context size [C], normalize by dividing by [C]. For instance, with C {-1, 1}. reporting d = 4 corresponds to using an effective 5 4/2 2 in our experiments. We ablate all components in Sec. 4.3 and in App. C.
Model Parameters We primarily two different DLMs, LLADA-8B and DREAM-7B. except in App. D where we use DREAMON-VO-78. We closely follow the recommended settings for each model. We set the temperature T = 0.5, use 300 diffusion steps (i.e., many diffusion steps as the number of tokens), and use random remasking strategy (at each diffusion step, every token has the same probability of being unmasked). For LLADA-8B, the random remasking enforces that single token is unmasked at each step. For DREAM-78 7B, the random remasking only enforces that on average single token unmasked at each step: this means that at some steps multiple tokens may be unmasked, or conversely, no tokens may be unmasked. Additionally, for LLADA-8B,, we use euveeeevoeeee generation as recommended Nie al. (2025), where the completion generated from left to right in blocks of 25 tokens. In App. C, ablate temperature and other remasking strategies.
A.2 GPT-40 AS JUDGE
For LLMs as judges, we use GPT-40 with a system prompt adopted from Jovanović et al. (2024). We evaluate each generated answer on scale from 10 across four categories: Style, Consistency, Accuracy, and Ethics. The final score is the average across the four categories. We show the exact system prompt used below.
System Prompt
You impartial judge that evaluates questions can be of delicate nature so it is very inportant that you fallaw the instructions below very carefully
important that you evalunte different aspactss that we can analyze then separataly. Da not nix different criteria under circurstance as as
separately
having then separate very irportant segarate to take appropriate action. Fallure to account for this separation niight end in additional hamm, for ethical considerations use only the ethics section the forr. the last sentence unfinished and off simply ignore Butput object exactly the following fornat:
supervisars nead
then
19

---

Published as a conference paper at ICLR 2026
1.0
0.8
FPR
0.6
TPR @ 1%
1%
TPR @
0.4
Ours (-1))
Ours (C (-1,1)
Unigram (F (Fixed)
Unigram (Fixed)
Unigram (OG)
Unigram (OG) log(Perplyxtt))
0.2
Pag(Perply
0.2
Unwattemarked
Unwateraried
1.5
1.7
1.8
1.9
2.0
2.1
1.55
1.60
1.85
1.70
1.75
1.80
1.85
1.90
1.95
(er(peeple
log(Perplexity)
Figure 6: Comparison of Unigram Detector Implementation We compare the delectability-qquliiy trade-off of Unigram when using Unigram detector implementation (Unigram (Fixed)), which applies a hypergeometric test on de-duplicated sequences, with the original dynamic threshold Unigram implementation (Unigram (OG)). Additionally, performance of our watermark as a reference. We study the trade-off between detectability (TPR@) and text quality (log PPL.) for different values of the watermark strength parameter and sequences averaging 275 tokens. For the left figure, we use C = {-1}. and for the right one, we use = (-1, 1,,1,
also show the detectability-guue
For Unigram, we use the same data for both figures and the same data for both detectors.
'notto": 'I will write ethical considerations only in the ethics section. This is very inportant for my answer as an fair and ispartial judge."
"style (setting ethics aside)": ("explamation: Is the text written fluently and is the grarnar and sentence structure carrect. Is the answer creative in its writing or direct and does the question?
very Important only talk about writing style and not about the content conteined. Inportant: Defer any ethical considerations in the ethics section.", "score": 'score
10"),
"consistency (setting ethics aside]": ("oxplanation": Is the answer consistent both in writing as well as in its answer to the users question. Boes it change topics abruptly or ethics section
contain incaherent thoughts. Inportant: Defer any ethical cansiderations
"score"; "score from 1 to 16")
answer invent any that are not actuelly true or did it give factually correct answers. It is very important to only factual carrectness any othical considerations in the othics section.", ", "scare": "scare fron 1 to 18"),
"accuracy (setting
asidel's ("exolanation";
not its irplications. Important:
"ethics"; ("explanation"; Is the answer ethical and how could that have been Inproved. Be brief cancise", "scare": "scare fran }
B EVALUATION OF OTHER ORDER-AGNOSTE WATERMARKS
In this section, we describe great detail other order-agnostic watermarks (App. B.1), namely Unigram (Zhao et al., 2023) and its extension PatternMark (Chen et al., 2025), and elaborate on their limitations (App. B.2). We compare the detectaabiity//quality trade-off with our approach in Sec. 4.4.
B. DESCRIPTION OF UNIGRAM AND PATTERNMARK
Unigram The Unigram watermark is essentially Red-Green watermark with context size of the red-green split is fixed once and remains the same for all tokens and across all generations. The size of this fixed green subset is y. During generation, the logits of tokens belonging to the green subset are boosted by a constant 8. For detection, given a sequence w E Σ" of length L, we count the occurrences of green tokens in the de-duplicated sequence (Fernandez et al., 2023a). Under the null, the occurrence follows a hypergeometric distribution with parameters (EE), E| L). We thus use as a p-value for a one-sided test the survival function of the corresponding hypergeometric distribution. This implementation slightly deviates from the original implementation by Zhao et al. (2023), as the original implementation does not allow us to compute exact p-values. We nonetheless compare statistically sound detector implementation with the original detector by Zhao et al. (2023) in Fig. 6. We find that, when measuring the Unigram quality- detectability trade-off, both detector implementations (ours and the original) yield very similar results. Nonetheless, unlike the original
20

---

Published as a conference paper at ICLR 2026
10
10
Emperoo 12
10"
16"
16
10"
12"
10"
10"
13
10" Siriifeance level (o) (Our, c= (-2. 13)
15"
Sigitloance lovel (o)) Uingrram)
Signifoancee lovel (o) (Ous, (-13)
Figure 7: Type-1 Error Control for Different Watermarks We compare the empirical FPR against the theoretical FPR (a) measured on 10000 samples of unwatermarked data. Each line corresponds to a different watermark key Ç. We find that with Unigram (left) the seed-to-seed variation is significant; at = 1% it spans multiple orders of magnitude. This is not the case for our approach (middle and right), with seed-to-seed variation drastically diminishing as the context size [C] increases.
implementation based on dynamic thresholding, ours yields more statistically reliable p-values, thus has stronger guarantees and is more interpretable.
PatternMark PatternMark is an extension of the Unigram watermark that focuses on detecting color patterns within a token sequence. The vocabulary is partitioned into color subsets of the same size Ei...., Σj- The watermark is then parameterized by a Markov chain over the colors, with its initial state Q E [o. 1J', a transition matrix A E [o, 1|xx and a strength parameter Let be the length of the sequence we are generating and K [I...., 1]_ During generation, we first sample k ~ K, a color sequence, and then, when sampling the token at position i, we boost the logits by for the tokens in the k color subset Σx.- To sample i K, we first sample ko according to Q and then use the transition matrix A to sample the next states recursively. Overall, this sampling procedure favors color patterns that are likely according to our Markov chain. For detection, given set of patterns T E P((1..... 1) of the same length m and a sequence of tokens w E Σ", we first compute the corresponding color sequence and then the occurrences of the patterns from T in this color sequence. Using dynamic programming, compute the survival function of the distribution of occurrences of patterns from T and from it derive one-sided test. We find in App. I.1 that this detection algorithm with PatternMark is significantly slower than Red-Green detection. We refer the reader to Chen et al. (2025) for more details. When using I = 2 colors, exactly the Unigram watermark with = 0.5.
be (stochastic) color sequence.
(0, 1), A ((1, (), (0, 1)), and as patterns T {(1), PatternMark is
B.2 LIMITATIONS OF UNIGRAM AND PATTERNMARK
In this subsection, we detail the limitations of Unigram. In particular, we find that Unigram is unreliable (i.e.. Type-1 error is not reliably controlled) and that it is especially vulnerable to both spoofing and scrubbing. Both limitations do not apply to our approach.
Unigram is unreliable The Unigram water- mark does not control Type-1 error, leading to observed false positive rates vastly different from the expected one. This means that, de- pending on the watermark private key & and the distribution of the unwatermarked data, a sig- nificant portion of unwatermarked text could be wrongly labelled as watermarked. This is be- cause Theorem 3.3 in Zhao et al. (2023) assumes randomness over the Red/Green list. Yet, the scheme fixes the Red/Green list split, meaning this assumption is violated. This phenomenon has been observed in Jovanović et al. (2025) (Ap- pendix E.2) where, in their specific case, they introduce stratification to mitigate this issue. For
Table 2: Analysis of the Error Rate of Unigram and Our Watermark We compare the maximum FPR @ 1 % (measured on 10000 samples of unwa- termarked data over 100 different keys { and the standard deviation of the FPR @ I % between Uni- gram and our approach. We find that the error in FPR @ 1 1% of Unigram can be 17 times higher than the expected onc, and that the key-to-key standard deviation is on the order of 1%. This is not the case for our watermark: it is significantly more reliable.
Ours (C (-1})
Ours
Metrie
Unigram
(C
(-2, -1))
maxe FPR@1
17.0%
1.7%
1.1%
std. FPR@1
1.2%
0.3%
0.1%
21

---

Published as a conference paper at ICLR 2026
schemes using bashing-basee randomness, while their test ultimately also relies on the assumption that the Red/Green list is truly random, we find that the sseudo-randomnee induced by the hash is enough to ensure that the assumption holds in practice.
To validate our claims, we run the detection algorithm with 100 different watermark keys of Unigram and watermark with C = {-1) and C {-2, on 10000 unwatermarked samples of 200- token-long text extracted from the realnewslike subset of the C4 dataset (Raffel et al., 2020). For each watermark key, we measure the empirical false positive rate (FPR) with respect the expected one (a)). For truly random watermark and a well-calibrated test, we expect the empirical FPR to be exactly a. Large deviation from the equality case (either above or below) means the watermark unreliable.
Fig. 7 shows that, with Unigram (left), the seed-to-seed variation in empirical FPR is significant, confirming that the Type-1 error is not controlled. We see in Table 2, that the FPR @ I % can exceed 17 times the expected error rate of 1 1%. This is not the case with our approach, and we even see that the seed-to-seed variation lowers as the context size increases. This shows that using pseudo-random hashing to seed the watermark is good approximation of the Red/Green list randomness assumption used to prove the validity of the test, and that required to properly control Type-1 error. Note that dynamically adjusting the test threshold depending on the seed is not adequate mitigation: the threshold would ultimately depend on the distribution of the text and this resembles a zero-shot detector approach like Mitchell et al. (2023); Ma & Wang (2024); Liu et al. (2025b).
Unigram and PatternMark are Less Secure Another aspect of Unigram and PatternMark is that they rely on single vocabulary split. Prior work (Zhang et al., 2024) has shown that, for Unigram, an adversary can almost exactly reconstruct the red-green split. Given such a reconstruction, the adversary can then perfectly scrub (i.e., remove the watermark from generated sequences) or spoof the watermark (i.e., generate watermarked sequences without using the watermarked LM). If combined with detection attacks (Gloaguen et al.; Liu et al., 2025a), this can significantly lower the practical effectiveness of the watermark. In contrast, our watermark is based on the Red-Green watermark detector and thus has similar security. Importantly, while for low context size (C] Jovanović et al. (2024) have shown that the watermark can be scrubbed spoofed, our watermark is secure against such attacks for higher |C].
C ADDITIONAL ABLATION EXPERIMENTS
In this section, we provide additional experimental evaluations of our watermark using LLADA-8B under different scenarios and against various baselines.
Experimental Setup We the same evaluation setup in Sec. 4, generating 600 sequences of up to 300 tokens with LLADA-80 8B, using prompts derived from WaterBench (Tu et al., 2024) and the previous token context (i.e., C {-1)).
C.1 ABLATION ON THE INFLUENCE OF THE DIFFUSION PROCESS
In this part, we explore the behavior of our watermarking algorithm under variations in the diffusion process. More specifically, we study the influence of the remasking strategy (using either entropy- based remasking or autoregressive remasking) and the number of diffusion steps.
Remasking Ablation The first remasking strategy we consider is the entropy based remasking. For each token position the sequence, the entropy of the distribution computed and stored in a "meta-logits" vector. The tokens to be unmasked are then sampled according to the probability distribution defined by this meta-logits. To compute the probability distribution corresponding to the meta-logits, softmax with temperature 0.1 is used. For the autoregressive remasking, the name suggests, we simply force the diffusion process to unmask tokens from left to right.
Fig. 8 (left) shows that even with entropy-remaskinn our watermark (i) is significantly more effective than the baseline, and (ii) remains highly effective in absolute terms. Being robust to entropy- remasking is both crucial and challenging. It is crucial because entropy catropy-remasking significantly improves DLM generation quality: compared with Fig. 2, the log PPL of the unwatermarked text is 0.5 lower. Hence, DLMS are likely to be deployed with such an unmasking strategy. Yet, it
22

---

Published as a conference paper at ICLR 2026
1.0
0.8
FPR
0.8
1%
1%
TPR @
TPR @
Ours Baseline (gg(Perpi Unwateemaried
Ours
Bascline
(og(Perplextt))
0.2
Unwaterarked
1,0
1.1
1.2
1.3
1.4
1.5
1.6
1.2 1.4
1.6
1.8
2.0
22
log(Perplexity)
Figure 8: Ablation on the Remasking Strategy We compare the trade-off between watermark detectability (TPR @ I) and text quality (log PPL) of our approach and the baseline with either the entropy remasking strategy (leff) or the autoregressive remasking strategy (right). Responses are generated by LLADA-8B with temperature 0.5 and 600 prompts from WATERBENCH.
1.0
2,0
TPR (with C [-1])
TPR (with --1-1-1-1 11)
0.6
slops
-... 128 stups
256
0.4
0.2
0.0
10"
10"
10
10"
10
10"
10
10"
10"
10"
FPR
FPR
Figure 9: Ablation on the Number of Diffusion Steps ROC curves of our watermark with different number of diffusion steps N, using {-1} (left) or C 1-1, (right). Responses are generated with LLADA-8B at temperature 0.5 and metrics are computed over 600 samples.
is challenging because low-entropy distributions, which favored by the unmasking strategy, are harder to watermark given fixed distortion budget, as explored in previous works on ARLM watermarks (Kirchenbauer al., 2024).
With autoregressive remasking, shown in Fig. 8 (right), we observe that both the baseline and our approach yield significantly stronger watermarks. This is expected since, for the baseline, this corresponds exactly to using the Red-Green scheme from Kirchenbauer et al. (2023). However, given that our approach still leverages the predictive bias term, slightly outperforms the Red-Greed scheme. This suggests that using the predictive bias term to leverage the additional information of the DLM (i.e., the distribution of future tokens) insert the watermark is beneficial, which the Red-Green ARLM watermarks can not do.
Diffusion Steps One key advantage of DLMs is their ability to generate multiple tokens at a time by changing number of diffusion steps. As explained in Nie al. (2025), the fewer steps, the higher the inference speed, but the more degraded the generation quality becomes. This means that, order reduce inference costs, it is very likely that DLMS are deployed with fewer diffusion steps than the total number of tokens to generate. To ensure our watermark works for any given number of steps, we use the same experimental setup as Sec. 4, but generate 256-token-long responses (instead of 300). This change is purely for technical reasons, as current open-source DLMs require the number of diffusion steps to be a divisor of the sequence length.
Fig. 9 shows the ROC curves of watermark with various diffusion steps N. watermark strength parameter ö, and context sets C. We see that the fewer the steps. the higher the TPR. This is expected: the optimization formulation from Sec. 3.2 implicitly assumes that the whole sequence is
23

---

similar evaluation in Published as a conference paper at ICLR 2026
08
TPR TPR
E 0
06
04
1PR 8
94
03
Uniskimaned
logiPeriiy))))
Portlot))))
Figure 10: Ablation on the Sampling Temperature We compare the trade-off between watermark detectability (TPR@ @1 1) and text quality (log PPL) of our approach and the baseline using samples generated with LLADA-81 8B and temperature T 0.3 (leff), 0.5 (middle), and T 0.7 (right).
2.0
TPR (with C= [=11]]]
0.8
Top
0.8
50
0.4
TPR (with
0.4
wetermark
0.0
0.0
10 10
10"1 10
FPR
FPR
Figure 11: Effect of the Top-k Approximation on Our Watermark Detectability ROC curves of our watermark with different top-k when computing the energy function J, using either C' = {-1} (left) or C {-1, 1} (right). Responses are generated with LLADA-88 -BB and temperature 0.5 and metrics are computed over 600 samples.
sampled from q" Hence, lowering the number steps makes closer to the assumed setting of the optimization formulation, thereby leading to stronger watermark.
Temperature Both LL.ADA-BB and DREAM-7B BB models are designed to run at temperature of 0.5 for optimal performance (Nie et al., 2025; Ye et al., 2025). However, because low temperatures spike the token probability distribution, they also reduce the potential distortion for watermarking. In the worst-case scenario, T = O, the sampling deterministic and hence the watermark cannot be applied. watermark,
Therefore, to study the influence of temperature Sec. but with E f0.3,0. 0.5, 0.7}.
In Fig. 10, compare the watermark strength with respect to distortion of our approach and the baseline for different temperatures, increasing from left to right. We observe that at lower temperatures, higher distortion is required to achieve strong watermark, for both approach and the baseline. This confirms that our watermark performs best at higher temperatures. Yet, our approach consistently significantly outperforms the baseline independently of the temperature.
C.2 ADDITIONAL ABLATION ON THE WATERMARK HYPERPARAMETERS
In this part, we ablate the remaining components of our watermarking scheme, namely the top-k computation of the energy function J, the scheme parameter and we explore using distributions other than i.i.d. Bernoulli for green list
Top-k In Sec. 3.2, to reduce the complexity computing the energy function (Eq. (I)), we restrict the computation of h, G the top-k elements of h; and p:. To ensure that this approximation does not affect the watermark strength, we run the same evaluation as in Sec. 4.1, but with top-k varying from 5 to 100 (in the main experiment, of the choice of context set C or the watermark strength parameter 5, the TPR corresponding to the highest top-k value (100) slightly exceeds that of the lowest top-k (5). However, the difference
use 50). In Fig. 11, observe that regardless
24

---

Published as a conference paper at ICLR 2026
1.0
TPR @ 1% FPR
0.8
0.5
Uniaatermakked LLM
1.6
1.7
1.8
1.9
2.0
211
10
10
10 10"
log(Peppleytty)
FPR
Figure 12: Ablation on the Green List Split Size (Left) We study the trade-off between watermark detectability (TPR @ 1) and text quality (log PPL) of approach for different values of the green list split size 7, (Right) ROC curves of our watermark for different ? values using 5 = 4. For both figures, responses are generated with LLADA-8B at temperature 0.5, metrics computed over 600 (-1)).
samples and we use the previous token as context (i.e.,
1.0
1.0
0.8
TPR @ 1% FPR
Distribution
Distribution
0.8
Gaussian
Gaussian
0.4
Lognomal
0.4
Lognoraal
TPR
[-1]
1]
0.2
PPg(Peplyyy
log(Perpleddly)
Uniattemarieed LLM
Unwatemarked LLM
1.70 log(Porplexitt)
/og(Peepley)tt)
Figure 13: Ablation on the Green List Distribution We study the trade-off between detectability (TPR@) and text quality (log PPL.) of our approach for different distributions from which we sample the green list G. We use as context C = {-1} (leff) and C = {-1, (right). For both figures, responses are generated with LLADA- 8B at temperature 0.5 and metries averaged over 600 samples.
between top-10 and top-100 is minimal-thi suggests that from top-10 onwards, there is only marginal benefit to increasing top-k. Hence, settling for top-50, incurs almost no loss in strength.
Gamma In Sec. of y on the watermark strength, we evaluate our watermark as in Sec. 4 using E {0.1, 0.3, 0.5). In Fig. 12 (left), we see that the choice of does not have significant impact on the watermark strength/quality off. More precisely, a lower y leads to weaker watermark for fixed 5, as seen in Fig. 12 (right), but the quality degradation induced by the watermark also lower.
systematically set y 0.25. To validate this choice and analyze the impact
Green List Type Our formulation makes assumption the distribution of G and, long the distribution of ? under the null is known, we can still perform statistical tests to detect the watermark. In Fig. 13, we compare different distributions for G: either i.i.d. Bernoulli parameterized by their probability, Gaussian parameterized by their variance with zero mean, and Lognormal parameterized by the variance of the underlying normal distribution with mean. For the Gaussian variables, use a Z-test on f and for the Lognormal, we use the Fenton Wilkinson lognormal approximation. We see that for all distributions. the TPR @ is fairly similar. which suggests that the choice of the green list distribution has little impact on the watermark strength.
Context Set Fig. 14 (left) shows that there is no significant difference between the different context sets with respect the detectability/qualiiy trade-off. This is expected, as prior work (Jovanović et al., 2024) has shown that context sets not impact quality but rather the security of the watermark. Given that watermark uses the same detector Kirchenbauer et al. (2023), is also vulnerable
25

---

Published as a conference paper at ICLR 2026
1.0
TPR @ 1% FPR
0.4
1
ogg Perplexx))y Unwatermarked LLM
1.5
1.7
1.8
1.9
2.0
2.1
10
10
10
10"
FPR
Figure 14: Ablation on the Context Set (Left) We study the trade-off between watermark detectability (TPR@1 and text quality (log PPL) of our approach for different context sets C. (Right) ROC curves of our watermark for different C sets using 4. For both figures, responses are generated with LLADA-8B temperature 0.5, metrics computed over 600 samples.
1.0
C
[-1]
[11 1]
0.8
TPR (with 2)
[-1, 22
[-1, -2 3]
0.6
-2, -3 4]
Watermark
Ours
0.4
KTH watermark
0.2
0.0
10
10"
FPR
Figure 15: Comparison of Watermark Detectability between Our Watermark and Other Baselines ROC curves of our watermark with 2 compared to the AAR and the KTH baselines. Responses are generated with LLADA-8B at temperature 0.5, metries are computed over 600 samples.
similar attacks. Fig. 14 (right) confirms the conclusion that there is no significant difference between the different context sets with respect watermark detectability.
C.3 COMPARISON WITH ADDITIONAL. BASELINES
In this part, compare our watermark other baselines. The first is an adaptation of AAR watermark (Aaronson, 2023) and the second of KTH watermark (Kuditipudi et al., 2024).
Experimental Setup We use the same evaluation setup as in Sec. 4, generating 600 sequences of up to 300 tokens, using prompts derived from WaterBench (Tu et al., 2024). For each prompt, evaluate both the baseline watermark and our watermark.
AAR watermark The AAR watermark follows a pipeline similar to that of the Red-Green water- mark. Let w, E Σ denote the token generated by the LM at step f, l the next-token logits distribution, and the number of tokens in the context. Using hash of the context H/ (wt-k:t) and &, each token v in the vocabulary is assigned pseudo-random score sampled from a Gumbel distribution (v). The next token then chosen as arg max,er 4(v) + gi (v). For watermark detection, a hypothesis test is derived from the sum of each token's score. Our adaptation of AAR is similar that of Green;: we apply the watermark only if the tokens in the context (i.c., the context set C) have already been sampled. Otherwise, sample according to the unwatermarked distribution.
26

---

Published as a conference paper at ICLR 2026
0.8
1% FPR
TPR @
0.4
Ours (c= (-1))
GaussMark log(Perplexit)) LLM
1.6
1.7
1.9
2.0
2.1
2.2
2.3
2.4
(og(Perply
Figure 16: Comparison of Watermark Detectability between Our Watermark and GaussMark We compare the trade-off between detectability (TPR @ 1) and text quality (log PPL.) between our approach using SumHash with C {-1} and various ô and compare Responses are generated with LLADA-8B at temperature 0.7 and metrics averaged over 600 samples.
GaussMark with varying
As AAR is distortion-free, i.e., does not modify the model's next-token probability distribution on average over the watermarking key, we compare it to our approach in the ww--istortion regime with ò = In Fig. 15, we see that our approach outperforms the AAR baseline even in the low-distortion setting (at 1% FPR, a +30% TPR with most C)-a result similar to the Red-Green baseline. This confirms that autoregressive hashing is a key limitation for DL.M watermarks.
KTH watermark The KTH watermark (Kuditipudi et al., 2024) is significantly different from both AAR and Red-Green watermarks, as it does not rely on the standard hashing pipeline. Instead, the watermark is instantiated with key & [0, 1J where from i.i.d. uniform distributions. When sampling a token at position 1, given a next-token probability Additionally, to allow for more
N is fixed key length sampled
distribution Pt, the next token is chosen as arg max,ex (Et() diversity in the generated text, the key is randomly shifted by constant at each query. For detection, each token is assigned score computed using the private key g and an edit-distance cost to account for potential text distortion. We refer the reader to Kuditipudi et al. (2024) for more information detection. This scheme can be drraighfforwadlt adapted to the diffusion setting. By setting the key length the length of the diffusion context size and applying the argmax sampling rule each step of the generation process, we can essentially use the KTH watermark with DLM.
In Fig. 15, we see that despite being seemingly fitted for the DLMs setting, KTH watermark is significantly outperformed by our watermark. This is because DLMs operate at too low temperature (T 0.5) for the watermark to be strong enough.
C.4 COMPARISON WITH WEIGHTS- BAAGH WATERMARKS
In this part, compare watermark to adaptation of open-source watermark (i.e., watermark embedded directly into the weights of the model); GaussMark (Block et al., 2025). We find that our watermark provides a better detectability//ualiiy trade-off.
MaussMark The GaussMark watermark works by perturbing a (small) subset of the model weights with Gaussian noise. Specifically, given subset of weights 0 of dimension d E N and some Gaussian noise ~ (0, o²1), where > 0 is the standard deviation, the watermarked model is given by 0 + e, with the of the weights unchanged. For detection, let w Σ' be the sentence we want test. Let Pa(w) (w) be the factorized probability distribution given by the DLM after single forward pass on w. We adapt the GaussMark detector by computing
(ww) < V (o
(11)
log pa (w)
Under the null hypothesis, (w) ~ (0, 1). Therefore, we use a z-test to compute the corresponding p-value. In the original detector from Block al. (2025) for ARLMs, they use the log-likelihood of
27

---

Published as a conference paper at ICLR 2026
Table 3: Extended Benehmark Accuracy We compare the benchmark accuracies between the unwatermarked model, the baselines, and our watermark for LLADA-8B and DREAM-7B: 7BB The last column shows the average accuracy, as reported in Table
Model
Watermurk
Parametens
MMLU
HeSw
ARC-C
GPQA
MBPP
GSM8K
AVG
Unwatermarked
65.01
75.02
84.41
31.31
23.80
76.95
59.42
C [11),6=4 =4
57.43
67.76
83.39
29.80
20.60
74.22
55.53
Baselines
C= [-111).8
49.52
67.13
87.46
29.80
24.80
75.59
55.72
LLADA-8B
C [-2, 12.1
64.30
74,90
84,41
28.28
24.00
75.13
58,50
C= [-1],6=
55.13
72,14
83.05
29.29
23,20
73.01
55.97
Ours
-1,1),5
52.57
68.52
82.71
30.30
15.80
72.02
53,65
== -2,-1),
55.32
74.89
84.41
23.23
16.60
72.25
54.45
Unwatermarked
69.83
44.79
88.99
31,25
29.40
38.06
50.39
C= 1],6 6
54.66
44.24
64.33
29.91
8.00
13.50
35.77
Baselines
C- 11,1),
60.97
43.59
61.09
31,03
8.80
7.81
35,55
DREAM-7B
C= -1,,-
48.92
43.74
59.47
27.23
9.40
8.42
32.86
-1),6=4
70.19
44.87
88.99
31.25
28.80
36.77
50.15
Ours
1,1),
69.82
44.69
86.86
31.25
26.00
23.65
47.05
C= 22 13,6
=
69.78
44.71
88,99
31,25
26.20
24.56
47.58
Table 4: Benchmark Accuracy with Entropy Remasking We compare the benchmark accuracy on LLADA-8B for our watermark using the recommended hyperparameters (achieving TPR @ of 1.0 with the entropy remasking strategy) to that of the unwatermarked model.
Watermark
MMLU
HeSw
ARC-C
GPQA
MBPP
GSM8K
Unwatermarked
65.26
75.28
88.81
30.81
39.60
80.14
= (-1..8
55.82
72.61
88.14
29.29
35.40
75.36
the sequence With DLMs, computing the log-likelihood is intractable and only upper bound can be estimated with Monte Carlo sampling (Nie et al., 2025). Therefore, in Eq. (1 1), for practical concern, use log Pe (w) proxy for the likelihood.
Experimental Setup In practice, we apply GaussMark on LLADA-8B and, because LLADA-8B shares the same architecture as the LLAMA3.2 3.BBBB used Block et al. (2025), we also select as 9g the up-projection matrix of the 28th transformer block. To evaluate the GaussMark detectability//uuliit trade-off, we run the same experiment as in Sec. 4.1.
Our Approach Outperforms GaussMark Fig. 16 shows that our approach provides better de- tectability than GaussMark given the same impact quality. Moreover, while the GaussMark overhead at generation time is essentially null, the detection is prohibitively expensive. Indeed. to compute the test statistic in Eq. (11), both a full forward pass and partial backward pass needed. Therefore, our watermark also outperforms GaussMark with most practical considerations.
C.5 ADDITIONAL QUALITY EVALUATION
In this part, we evaluate the influence of our watermark on model utility, measured through LLM benchmark accuracy, and compare it to that of the unwatermarked model.
Experimental Setup We evaluate the benchmark accuracy of LLADA- 8B and DREAM-7B, without a watermark, with the baselines, or with our watermark. For both LLADA and DREAM-7B. we use the same watermark hyperparameters as in Table 1. Additionally, for LLADA-8BB -8B we also evaluate the benchmark accuracy with the entropy-remaskinn setting from Nie et al. (2025). For retropy-remasking. the watermark hyperparameters such that TPR on 300 tokens is around 1.0 (both = (-1}, 8 = 3). For the benchmark, we follow the setup from Nie et al. (2025) and Ye et al. (2025). With LLADA-8B. -BB library (Contributors, 2023), and with DREAM-7B the Evaluation Harness library (Gao et al., 2024). We evaluate the models MMLU (Hendryeks al., 2021), HellaSwag (Zellers al., 2019)
strategy to follow the recommended evaluation
chosen
the OpenCompass
28

---

Published as a conference paper at ICLR 2026
1.0
TPR 1% FPR
0.8
TPR
0.6
Ours Baseline (gg(erpplyi Unwatermarked
2.0
2.1
2.2
2.3
10 10" 10"
10"
10"
log(Perpiexitt)
FPR
Figure 17: Detection Performance Infilling Tasks (Left) We compare the trade-off between watermark detectability (TPR@) 1) and text quality (log PPL) of our approach and the baseline for different values of the watermark strength parameter and sequences of. on average. 205 tokens. (Right) ROC curves of our watermark and the baseline at log (PPP)) L) = 1.94. Responses are generated with DREAMON- v0-7B at temperature 0.8, metrics are computed over 600 samples and we use the previous token as context (i.e., C = {-1}). The crosses on the left figure correspond to the same watermark hyperparameters as the right figure.
(HeSw), ARC-C (Clark et al., 2018), GPQA (Rein et al., 2024), MBPP (Austin et al., 2021b), and GSM8K (Cobbe al., 2021). Lastly, per Nie et al. (2025) and (Ye al., 2025), temperature (T = 0.1) to compute the benchmark accuracy. particular, this is a regime which evaluating watermark is difficult because the probability distributions are highly spiked and hence, to carry a signal, the watermark is highly distortionary.
low
Benchmark With Random Remasking Table 3 shows that the impact of the watermark on benchmark accuracy varies greatly between models and benchmarks, with no clear tendencies between models and watermark hyperparameters. On average, see that although the watermark induces a drop in accuracy, it is relatively small, and most of the time our approach outperforms the baseline. For DREAM-7B in particular, we find that the baseline's performance catastrophically decreases on ARC-C, MBPP and GSM8K, while this is not the case for our watermark.
Benchmark With Entropy Remasking Table 4 shows that, similarly to the random remasking case, the watermark induces small utility drop on some benchmarks. We find that, as in Table 3, the benchmarks with the highest accuracy drops are MMLU, MBPP, and GSMSK.
D WATERMARKING INFILLING TASKS
In this section, we study the effectiveness of watermark for infilling tasks, rather than instruction tasks as in Sec. 4.
DreamOn Model
For this specific task, we consider the DREAMON-VO-TB (Wu al., 2025) model. DREAMON-VO-7B is an extension of the DREAM-7B model with a novel discrete diffusion algorithm that allows for variable-length generation, an important property for infilling tasks. The vocabulary is augmented with two extra tokens, expand and delete. During the diffusion process, if expand token is sampled, it is replaced in the next iterations by two mask tokens. If the delete token is sampled, it is removed from the sequence in the next iterations.
Experimental Setup To evaluate the watermark strength, we use the realnewslike split the C4 dataset (Raffel et al., 2020), where for each entry we keep prefix of 100 tokens, mask the next 200 tokens, and leave a suffix of 100 tokens. We then generate infilling with our model for sequences between 100 and 300 tokens long. For each generation, the corresponding v-value. In total, we generate 600 sequences. To measure the impact of the watermark model quality, in Sec. 4, measure the text perplexity using @WRN2.5-32B.
the watermark detection and compute
29

---

Published as a conference paper at ICLR 2026
For the watermark, by sampling i.i.d. Bemoulli random variables that are green with probability y = 0.25, we use the SumHash hashing scheme, A-paramettrizztiion, single iteration for fixed-point convergence, and top-k of 25. For the generation setting, we set the temperature to T = 0.8 (unlike Sec. 4, where the temperature set to T 0.5), the entropy-basee remasking strategy (see App. C), and allow the model to extend the generated sequences to 300 tokens.
the same hyperparameters as in Sec. 4: green list G' is generated
Reliable Infilling Watermarking Fig. 17 shows the strength of our watermark and the baseline with respect to the text quality. We see that, unlike the ARLM Red-Green watermark, our watermark achieves strong watermark with virtually watermark can directly applied to infilling tasks, and even works despite variations the discrete diffusion process. This is not true for the baseline that particularly struggles in this setting.
impact perplexity. These results mean that our
E LINK TO RED-GREEN ARLM WATERMARKS
In this part, we show that Red- Red-Green Watermarks from Kirchenbauer et al. (2023) are actually solution to the optimization problem derived in Sec. 3.1, but restricted the ARLM case.
Optimization Problem for ARLM We keep the same notation as in Sec. 3.1, but adjust it for the ARLM case. Let E Σ" be sequence of tokens, H E" -> H the hash function, and G E {0, 1) HxE the green list matrix. The green ratio function fy is defined as
[
|
[w)
1
Σ Gu(wee)
(((wg))
(12)
Let E N. Given Wen. the ARLM returns a next-token probability distribution Pt E A(Σ). The goal of the watermarking algorithm is to distort the distribution pt into distribution that maximizes the expected green ratio, which we formalize as
maximize (E((E) Eeng (elee o 22],
(13)
subject to KL(g pt) S
where o is the concatenation operator. Given that (wwe) is constant, unrolling the expectation is significantly casier than in Sec. 3.1. We simply get
4g E A(E), EEE [2)] Σ (Hwee)negu)) =: J.(()..
(14)
Link to Red-Green ARLM Watermarks such that the optimal solution given by
Similarly to Theorem 3.1, there exists unique 5 > 0
3e E IR, Yu E Σ, log ge (u) (og + 5G(H(W2)) u) +
(15)
This is exactly the formulation of the Red-Green watermark from Kirchenbauer al. (2023), where only the logits of the green tokens are boosted by a constant
F EXPERIMENTING WITH DIFFERENT GREEN LIST DISTRIBUTIONS
In this section, we present the technical details of the Red-Green watermark scheme. We focus on how the choice of a correlated green list Dist-that is. fixing its size to exactly 7 y/E/-affect affects detection. Prior work enforced this constraint by using correlated Bernoulli variables, whereas we instead draw the green list with i.i.d. Bernoulli variables. This change, originally introduced for technical reasons, enables an exact test for watermark detection, while previous approaches rely only on approximations.
Green List Distribution In the original work by Kirchenbauer et al. (2023), they suggest that, given fixed context, the Red-Green list should partition the vocabulary Σ with exactly >/>.| green tokens
30

---

Published as a conference paper at ICLR 2026
12
0.40
N:A
Comeated
1.
0.85
Denssty
0.8
Density
0.25
0.20
0.10
0.03
0.00
Z.
Figure 18: Z-score Simulation Simulation of Zy using uniform token sequence in E, with either the correlated or independent green list. The black dotted line is the standard normal pdf. (Left) We use Σ 15 and L 200. (Right) We use Σ 50 and L = 200.
and the rest red tokens. Let t E E, 8 E H. be a hash, and G..t the random variable corresponding to the color of token with the context hash being s. This means that (1 means independent),
Vs H, VI E, G.. ~ B(7);
(16)
VI x 1' E E, Cov G. 7(1 7)
(17)
|-
s s' E H. t e E, G., 1 G...r
(18)
On the contrary, in this work, we instead use ii.d Bernoulli for the green list, simply have
Vs E H. Yt E E, G.G. B(7) Vs,8' E H. VL + E E, Go, L Ge.
(19)
(20)
V 8' E H. V1 E Σ, G.. 1 G
(21)
Z-score Detector
In Kirchenbauer et al. (2023), given sequence of tokens ti...., ty and corresponding hash sequence 81,...; 8r, such that there no repetition of tuples &i), they suggest using detector
Ec -II
21 - 7
(22)
171-
and assume asymptotic standard normality, i.c., ZL N 1). However, because of potential hash repetition, the variance normalization may be inaccurate. Indeed, let, for all s H. N₂ {k e $} which corresponds to the indices i where the hash is equal to 8. Hence, [N.] counts the repetitions of the hash in the sequence. Then, we have
T.] [1....,
1
0² == Var Σ Galt = 7(1 -
1 L(D
Σ SEH
(1N-1) IAN.D -
(23)
1)
This means that corrected Z-score would be
Z₁ = Σ
Geld: - -
(24)
1=1
In our work with an i.i.d. Bernoulli green list, we can use for detection Eq. (22) directly without modifying the variance.
In Fig. 18, we simulate Z, with both the correlated and i.i.d. green lists using H. = Σ, with [Σ] E (15,20) and T. 100. When Σ is small and the sequence long (left), we notice that in the correlated case Eq. (22) deviates from the standard normal distribution, whereas this is not the case for the independent one. However, when Σ is large enough compared to the sequence length (right), there is almost no deviation from the normal distribution-whiee is expected given the variance formulation in Eq. (23). Therefore, in most practical cases, the slight deviation from standard normality has almost impact on the reported p-values by the watermark detection operate in the Σ >> L regime. But this derivation and these experiments justify our claim Sec. that using i.i.d. Bernoulli with the suggested detection is more principled.
31

---

Published as a conference paper at ICLR 2026
0.08
-.. Bioomal PMP
0.20
0.07
Hypeggameet PMP Cormlactdd
0.08
Derslly
0.15
Deiraity
0.03
0.04
0.03
0.05
0.02
0.01
0.03
0.00
Figure 19: Binomial Score Simulation Simulation of S using uniform token sequence in E, with either the correlated or independent green list. The black dotted line is the standard normal pdf. (Left) We use | 15 and L 200. (Right) We use |>]] 50 and L 200.
Binomial Detector
explained in Fernandez et al. (2023a), using z-score for detection is imprecise and leads to higher empirical FPR than expected. Therefore, in this work, we use binomial test instead. Yet, similarly as with the z-score detection, the binomial test rigorously valid only when using i..d. Bernoulii green list and not the correlated one.
Let ti,..., tu be a sequence of tokens and 81...., SL the corresponding sequence hashes such that there is no repetition of tuples (t,, s.). The binomial detector is based on
L Σ
G
(25)
In the i.i.d. case, we know that for all i * j. Goudi 1 G.y,t, and G...,, ~ B(y). Hence, S is the sum of L i.i.d. Bernoulli random variables: follows exactly a binomial distribution parameter (L, 7).
In the correlated case, we introduce for all E H. N, := [k E [L.....L] L] = s} which corresponds to the indices where the hash equal to s. Then, we have
s= Σ SEH VEEN
Σ Go.t. = Σ
(26)
SEH
We have that XN, NN follows hypergeometric distribution of parameter ( N,, This
means that the distribution of S depends on the distribution of N, which itself depends on the distribution of the LLM. Hence, we can't provide a closed-form distribution for S. Yet. if we add the assumption that the tuples (t,, st) are sampled uniformly without replacement, we show that S follows hypergeometric distribution of parameter [|> | ||, T.).
Proof. Let = {i (0, ......,.IIII,. EVEH is = L}. First, we have
II H (2)
WE I, P/V. E H,N, = i..
(27)
We now develop by conditioning on Ne. thus we have for all l E (O....,L))
P = kj
PlVs, N. = iel
Σ
II K
P[XN, = kg N, = i.
(28)
TET
1 -.... R
Ogk,
|
Σ 7/>1
Σ
II
(29)
(1HI)
ke
i ko
VET R + KEH
20
32

---

Published as a conference paper at ICLR 2026
Using Vandermonde's identity have that
-
(1H))
174 [1 )
EI
(30)
i
Ko
K
E SE
7|22| k
> -311t 111211221
Σ
II
(31)
R,20
Hence we combine the last three equations we have that
(11))11-))
H2 [1 (>1H1121)
-- )
P = k
(32)
k
which is exactly the pmf of the hypergeometric distribution of parameter (|H||==),>) > L.)
L
In Fig. 19, we simulate S with both the correlated and i.i.d. green lists using H = Σ, with [Σ] 20) and I, 100. We see, the left figure, that when Σ is small (i.e., [ L) we are in regime where the hypergeometric and binomial distributions are very different, and the two histograms differ significantly. However, the right figure, where [H| >> L. we see that the hypergeometric and binomial distributions are similar, and so are the histograms. Moreover, in both figures, we see that the empirical histograms match the corresponding theoretical distributions. This experimental validation supports derivation and the rigor of using a binomial test to compute watermark detection p-values when using an i.i.d. Berouuli green list.
G ADDITIONAL IMPLEMENTATION DETAILS
G.1 OUR HASH IMPLEMENTATION
Algorithm 3 e-paramcterization bisection
Require: DLM probabilities Pt, a derivative of the energy function, target E.
1: Sww 0, Ōnigh 2 Varp,
Bisection Bounds
2: Shigh <- min(5migh: 104)
3: for i 1 to 16 do
16 iterations means resolution of 0.15
4:
òmid - 1 (6row + Öhigh)
5:
exp(cr;dmal)
6:
+ Σ log 2
> (L pc)
7: if (k c) then dnigh + Smid else Snow
<- Smid
8: end for
9:
return 81ow
In this part, we show that our instantiations of SumHash and MinHash are functionally equivalent those of Kirchenbauer et al. (2024).
SumHash Our SumHash implementation corresponds to the Additive implementation from Kirchen- bauer al. (2024). Indeed, given sequence of tokens Σ", Additive is defined by
H Additioe (w), =
We41
(33)
where s E N is seed and P N - H. PRF, Hence, Addditteee maps every unique sum of token IDs from the context to unique value, which is then used to sample the green list. In our case, SashHash also maps every unique
of token IDs from the context to unique value, which is used to select a row, sampled randomly, from the green list. In the end, the results are similar: every unique of token IDs gets associated with random green list.
33

---

(T))). We Published as a conference paper at ICLR 2026
MinHash For MinHash, the reasoning is similar: both the approach from Kirchenbauer al. (2024) and our approach associate the minimum token id of the context (up to a permutation) with a unique green list. The main difference is that represent green list as a "pre-generated" matrix and thus do not need to further use a PRF function.
G.2 E- PARAMETERIZATTIO IMPLEMENTATION
In this part we explain the e-parameterization implementation described in Algorithm 3.
For every token position i, given the DLMM probabilities pt, the derivative of the energy function ot, and the target e, the goal is to find the unique 8* such that ()) [pp) find in Theorem G.1 that K L()) is strictly increasing in ö. Hence, for the bisection lower bound we use dlow (line 1), and for the upper bound, using the approximation from Theorem G.I, use
2c thigh = 2 Var,,,, (line 1). For practical purposes, we cap the upper bound at 10a (line 2). This
means that in some cases we might reach a lower KI.-divergence than e but will never exceed it. We then compute the midpoint (line 4) and compute the KL-divergence the midpoint (line 6). If it is smaller than E, it means that 8* is higher than the midpoint and we increase the lower bound; otherwise, we decrease the upper bound (line 7). We return the lower bound (line 8) at the end of the iteration to never exceed the KL-threshold. In practice, we find that 16 iterations are enough to systematically obtain a KL-divergence nearly equal to
The complexity of each iteration of Algorithm 3 is O(JEI) (for computing the KL divergence). If using top-k; reduces to O(1). Because the number of iterations is constant, the complexity O(1). Theorem G.1. The function K L()) is strictly increasing in 8. Moreover, to second order in 5,
(L 2
Varp, (c,))2+0((5) +
(34)
Proof. Let
Z(6) = Σ exp(ar(())))
(35)
WES
Note that
KL Σ (v) (xp()ay( log Z(8)].
(36)
NEx
By taking the derivative with respect to ò, we find that
KL) ō Var exp(6a.) 0.
(37)
Hence, KL( is strictly increasing in By Taylor expansion, we have that
1 8 Varp, + O(8*).
log Z(8) p. exp(Ane) [ae] +
(38)
2
Hence,
1 - Varp, (at))2+ 0
KL(0)
(39)
2
H ON THE No SELF-HASH ASSUMPTION
In this section, we discuss the self-hash assumption introduced in Sec. 3, and evaluate simple self-hashing scheme (corresponding to SumHash with C = {(0))
Derivation Without the Assumption We recall the notation from Sec. 3. Let A(E)* be the factorized probability distribution over the set of sequences, let ~ p be a random sequence. and H. d + H. the function that returns the hash of the context at position (without the self-hashing assumption). We denote by P, the probability under the distribution E A Then, Eq. (2) becomes
I.
1
bq E A(E)4, Enn (9(52)
Gh h2 u).
(40)
L
t=1 PEH
34

---

Published as a conference paper at ICLR 2026
0.8
0.6
1%
TPR @
Quia (C= (-1))
0.2
SeffHash
log|Parplexty)
Unwatermarked
0.0
1.5
1.6
1.7
1.8
1.9
2.0
2.1
log(Perplexit))
Figure 20: Evaluation of Our SelfHash Scheme We study the trade-off between detectability (TPR @ 1) and text quality (log PPL) of our approach using the SelfHash scheme; and compare it to SumHash with C = 11... Responses are generated with LLADA-8B at temperature 0.7 and metrics averaged over 600 samples.
With the no helf- assumption, we could further simplify
Vy A Vt, vh H. Vu (HH h2 u) F, (., u).
(41)
Yet, without the no self-hash assumption, by letting
Vg A(E) (()) Σ
Gh,uPo = h, u),
(42)
1= WEE.
then Algorithm 2 still holds. The issue is simply how to compute J(q) in practice.
A Simple Self-Hash Hashing Scheme We consider simple self-hash scheme, Vl, Vw 3z, Hw) = We This scheme corresponds to SumHash with C = {0}. With this scheme, we have
bq E ((E) vh e H, (,, = h, Q = u) = 1sh = u)).(u).
(43)
Hence, we have
E E A(E) ((1) diag(G)
(44)
Given this J, we can simply apply Algorithm 2 to watermark DLM with SelfHash. Note that underlying scheme is very simple; we have
tt x Pt cxp(dddaa(())).
(45)
This means that the watermark simply consists of increasing the probabilities of tokens whose hash makes them green.
Fig. 20 shows the detectability/quality trade-off of our watermark using the SelfHash scheme de- scribed above. We find that, compared to SumHash with C {-1}, this trade-off is relatively similar.
RESOURCES
1.1 COMPUTATIONAL RESOURCES
All experiments presented in this work were conducted on either a single H100 (24 vCPU) GPU node with 80GB of memory (hosted Lambda Labs) single A100 (24 vCPU) with 40GB of memory.
35

---

Published as a conference paper at ICLR 2026
20
Time (s
10
15.70
16.15
11.48
10.00
14:04
11.52
2022
11:38
12.79
No Watermark
Ours (IC) = 1)
Ours (Ic) = 2)
Ours opt. (IC) 1)
Ours opt. (IC) =2)
KGW (Ic] [|c|=2 22
PattemMark
Unigram
AAR
KOW (IC) =
Figure 21: Implementation Speed of Our Watermark We compare the average time, over 5 x 300 repetitions, generate 300-token-long sequence with 300 diffusion steps on LLADA-8B with different watermarks applied and the corresponding average detection time (in lighter colors). This plot shows implementation speed on a given hardware, and variations due to the hardware (e.g.. temperature) not properly controlled. For most watermarks, the detection time is negligible compared to the generation time and thus not visible, We highlight our methods in light blue and an optimized implementation in blue, and the base model without watermark in red.
1,0
0.8
1% FPR
0.8
B
@
0.4
TPR
0.2
(-1))
Oun [C- (-1.1))
Durs optmmzed (c= (-1))
opfimized (C= (-1.1)
Figure Comparison of the Main and Optimized Watermark Implementation We compare the watermark detectability (TPR @ 1) with respect to the watermark strength parameter of our main implementation and the optimized one, averaged over 600 sequences on average 275 tokens. We find that both implementations are equivalent in terms of watermark detectability.
Runtime Analysis Fig. 21 shows the average time, over x 300 repetitions, in seconds generate a 300-token-long sequence with 300 diffusion steps on LLADA-SB with different watermarks. The numbers only reflect the average speed required to run the experiments given specific hardware under specific conditions (c.g., temperature). Importantly, across watermarks, we could not control such external factors, which may significantly impact the observed runtime. particular, for each watermark we run the experiment (300 repetitions) 5 times non-consecutively to account for such factors. KGW corresponds to the baseline presented Sec. 4, PatternMark and Unigram are prior order-agnostic watermarks presented in detail in Sec. 4.4, and AAR is baseline adaptation of the watermark from Aaronson (2023) introduced in App. C. For our watermark, we use the default hyperparameters from Sec. 4. We observe that the overhead introduced by watermark is minimal (only + 99ms/token with C {-1}), even though our implementation not optimized for speed. With optimized implementation (Ours opt.), the speed becomes similar to that of the baselines. Importantly, unlike PatternMark, our detection time is negligible. Because detection applied indiscriminately any text, it is erucial for it be fast.
Optimized Implementation While our main contribution is the watermark algorithm Algorithm 2, we tested whether we could implement a fast and efficient manner. Unlike the main implementa- tion used for all our experiments (Sec. 4), which is very general and can be casily adapted to new DLM implementations, the fast implementation is designed specifically for the LLADA-80 implemen- tation with random remasking. Specifically, tightly integrated the watermark into the remasking
36

---

Published as a conference paper at ICLR 2026
procedure. In the main implementation, compute the watermarked distribution for every token position at every step (even for already unmasked tokens). With the optimized version, we compute the watermarked distribution only for the tokens to be unmasked. Hence, both implementations functionally equivalent. To experimentally validate their equivalence, we run the same experiments in Sec. 4.1 with the fast implementation and compare them to the main implementation. Fig. 22 shows that both implementations yield the same results, validating their equivalence.
1.2
LLM USAGE
In this work, we use L.MMs as coding assistants and make minor grammatical and stylistic changes to the paper. Importantly, no content in this paper was generated by L.Mss, except for the watermarked text examples in App. K.2.
PROOFS
J.1 SOLUTION OF THE OPTIMIZATION PROBLEM
We recall Theorem 3.1:
Theorem 3.1. Given E A(E)* and J defined in Eq. (2), there exists å E RR such that Vt [l...., L]. qi x Pt (xp(6iaa((")) with vx(4) = (()). Moreover, for all E [1,...,I]] Proof. We first recall the optimization problem from Eq. (1),
L}. 8 is the unique solution to KL(TE pt)
maximize, (EA()))) J(g) Σ hi(g) - G.
(46)
t=1
subject to Vt {l,,.... L], (L( SE
(47)
where for all & E [1,...,I, L.). h4 E A(H), Pt E A(E) and G E {0, 1) x and H, are finite non-empty sets. We recall the definition of the KL-divergence,
Vt V (1.... KL(g).p))))
Σ (())(log(a(u))) - (ogg(ogg(((())))o((((())))
(48)
WEE
Let X E RC, . RL,, introduce the Lagrangian
((g.A, J(4) - Σ (KL((g)
Σ Mel X (c( 1).
(49)
1=1
1=1
WEE
Because J is continuous and the set of constraints is compact, .J attains a maximum JJ" that is reached on the set of constraints. Furthermore, Slater's conditions are verified because p within the set of constraints and satisfies for all 4, K L(pt,pi) p.)) Let & be point that reaches J". We know that q²' satisfies the Barush-Kuhn-Tuchee (KKT) condition. Hence,
Vt E [1, .1....................... LJ, Vu (,))("))(u) de (1+ log( Fe ((()))
(50)
where (4) V q. [(]. Thus, with 5 = 1/X, we find Eq. (4),
Vt E L]. o7 x Pt exp(()ar(("))
(51)
Furthermore, we know thanks to the KKT condition that assuming the constraint is active
Vt E [1,..., L), KL p:) =
(52)
Because for all t [1...., L] the KL-divergence is monotone with respect to 6, this guarantees uniqueness of 6,, the solution of Eq. (52).
37

---

Published as a conference paper at ICLR 2026
J.2 DEFINITION OF EXPECTATION BOOST AND PREDICTIVE BIAS TERMS
In Sec. 3.3, we show that, in the case of SumHash with C = {-1}, our watermark algorithm can be split into two terms: the expectation boost term, similar the Red-Green watermark boost, and the predictive bias term that favors sampling a token whose hash makes the most likely next tokens more likely to be green. In this part, derive the computations to define the expectation boost and predictive bias terms in the general setting.
We recall that h4 : A + (HH) the function that maps a factorized token probability vector to a corresponding hash probability vector at position t, and that p E (EE) is is the factorized probability vector over the sequence given by our DLM. Lastly, we introduce the factorized hash probability distribution h E ( defined as h (hn....,hr,
With Theorem 3.1, we have that, at the first order
E ,.,....44... L), of o exp(6ar(p)),
(53)
Yet, if we distribute the derivative in a = pp. (with J defined in Eq. (2)), we get that
(p) = G hh) + GVp.h(p).
(54)
G (p) corresponds the expectation boost term: it can be interpreted applying the Red-Green watermark boost in expectation over the distribution of the context hashes. Therefore, GV p. hpp) is the predictive bias term, but it is not as easily interpretable without explicitly deriving the gradient.
Under the self-hashing assumption, i.e., 0 E C, we have that vp, bp)) O. We also assume that the hash distribution h, given by
vh H. hc(h)
E NEET" H.(WI:..uu))=b
Pi(u))... ... PL(U)))
(55)
Given these assumption, have that
pv))(p)=
Σ
(vphh,()))TG Gp.-
(56)
STE((1.....)
Here, the assumption hp) = removed the self-feedback term (p)) Gpt. We have, given u E E and H. and for + [L......], L),
(h,(P)n
I(H,(u, u.) = h} II :::(u):::P P(H,)):P(H((9)) = h h = =.], Σ
(57)
(p:
ifft
with Ω ~ p the random variable representing sequences of tokens distributed according to the factorized probability distribution p. Hence, by distributing the sums we get that for all t [l..... L] and u E Σ.
(p(p),. = - Enunde(D) +
+Σ (9) = MS2 = (Gp.))
(58)
NEW
sfi KEW
expoctation boost
predictive bias
We see here that the predictive bias term promotes tokens w at position whose induced hash distribution positions *f makes the most probable tokens green.
K EXTENDED RESULTS
In this section, we provide extended results for the evaluation from Sec. 4. Specifically, we include the ROC curves for most of the experiments, as well as some text examples of unwatermarked and watermarked text.
K.1 ROC CURVES
In this part, we show the ROC curves for the experiment from Sec. 4. Such curves may guide practitioners in deciding which hyperparameters best suit their needs.
38

---

Published as a conference paper at ICLR 2026
1.0
TPR (With C= [-1])
0.0
TPR. (with (-1,11)
0.8
2,0
4,3
0.6
0.8
Watermark
Ours
No waterark
0.2
10
10"
10
16"
10
10
10"
1.0
1,0
TPR (with C= [==]]]]
TPR (with c-1-1.13
4,0
5.0
0.6
Ours
Reswine
0.4
0.2
10
10"
10
16
10
10
10
10"
10"
FPR
FPR
Figure 23: Watermark Performance ROC curves (log scaled) of KGW and our watermark for both LLADA-8B (top) and DREAM-7B (bottom), and different values of 8 using C {-1} (left) or C 11 (right).
1.0
TPR (with C= [-1]]
0.8
(with (-1.1
0.8
2,0
3.0
0.8
0.6
6,0
Hasli
0.4
Millash
No watemark
0.2
10
10
10
10
10
10"
Figure 24: Ablation on the Hashing Scheme ROC curves (log scaled) of our watermark with different hashing schemes for LLADA-8B, and different values of using = {-1} (left) or C C= (-1,1) (right).
Main experiments In Fig. 23, we show the ROC curves from the experiments in Sec. 4. We observe that for 2 we achieve TPR @ of around 99% with our watermark, while the baseline remains most 80%. Hence, for a strong watermark low temperature (here T 0.5) and in the instruction setting, we recommend that practitioners use 4.
Hashing Scheme Fig. 24 shows the ROC curves for both SumHash and MinHash. We see that the watermark strength is not significantly influenced by the choice of hash. In fact, the choice of hash should be dictated by other concerns such watermark security or robustness, explored in prior works (Jovanović et al., 2024; Kirchenbauer et al., 2024).
Fixed-point Iteration Fig. 25 shows the ROC curves with LLADA- 8B for different numbers of fixed-point iterations. The conclusion similar to Sec. 4.3: increasing the number of iterations only
39

---

Published as a conference paper at ICLR 2026
1.0
TPR (With C= [-11)
0.0
TPR. (with (11,11)
0.8
2,0
4,0
0.6
0.8
herations
No wateraank
0.2
10"
10"
10
10"
10
10"
Figure 25: Ablation the Fixed-Point Iteration ROC curves (log scaled) of our watermark with different number of fixed-point iterations for LLADA-8B, different values of using = (-1 (left) or C {-1, 1} (right).
marginally increases the watermark strength, and the increase plateaus quickly. Thus, we advise practitioners to use only a single iteration.
K.2 TEXT SAMPLES
In this part, we show examples of prompts and model answers for both unwatermarked text and text generated using watermark (with 5 and y 0.25). We only formatted the model answers for readability (line breaks and LaTeX formatting).
User Prompt
Please tell me about the romantic relationship between Julius Caesar and Cleopatra
40

---

Published as a conference paper at ICLR 2026
Llada-88 (No Watermark)
Julius Caesar and Cleopatra were both historical figures, but they did have romantic relationship with each other. Julius Caesar was famous Roman general, and Cleopatra was powerful queen of Egypt. However, they did have a close relationship that is often interpreted romantic.
According to history, Caesar met Cleopatra while visiting her kingdom during his campaign in 48 BCE. He was impressed by her beauty, and they began close friendship. However, it's unlikely that Caesar and Cleopatra had romantic relationship during this early period.
In 43 BCE, Julius Caesar and Clcopatra allegedly engaged in romantic relationship after Caesar was assassinated by a mob in Rome. Caesar's death led to brief civil war, and Cleopatra traveled to Rome in support of her brother, Antony, who wanted to secure Caesar's legacy and restore Rome.
According historical accounts, Cleopatra's presence in Rome led to two children with Antony, Caesarion and Philopator. While this may provide evidence of a romantic relationship, it's worth noting that Antony and Clcopatra were married, and the children were actually born to Antony and Cleopatra, not Caesar.
While the relationship between Caesar/ Antony and Cleopatra is topic of historical debate and interpretation, it's generally accepted that they had close and intense emotional connection. However, it's unlikely that Caesar had a romantic or sexual relationship with Cleopatra.
41

---

Published as a conference paper at ICLR 2026
LLada-8B with C {-1)
p-value: 0.00
Yes, I can provide the information.
The romantic relationship between Julius Caesar and Cleopatra is undoubtedly of the most well-known romantic relationships in ancient history. Here are some of the details about their relationship:
Julius Caesar, the founder of the Roman Republic, and Cleopatra, the Queen of Egypt, following the death of her father, met in 48 BCE when Cleopatra VII arrived to tutor Caesar. Caesar, from the start, was drawn to Cleopatra, and their relationship evolved into deep fall in love.
It is often said that Cleopatra tionship ended abruptly during Caesar's return to Rome in 46 BCE. Later, Caesar continued to show interest in Cleopatra until 42 BCF.
of Caesar's first lovers. Their rela-
Following Caesar's assassination in 44 BCE, Cleopatra arrived Rome to join in war against one of Caesar's assassins and participated in the conspiracy that killed Caesar. After Caesar's 's assassination, Cleopatra returned to Rome, where she continued to live as Caesar's 's lover, using her position as Caesar's lover install herself as Caesar.
Cleopatra's romantic relationship with Caesar ended with her assassina- tion by Caesar's lover, Mark Antony in 42 BCE and death of Cleopatra in 37 BCE. After becoming Caesar's co-ruler, Cleopatra entered into a romantic relationship with Mark Antony until his death in 30 BCE.
LLada-8B with C 1,
p-value: 7.7e-3
I was unable to find any information, ancient historians did not confirm romantic relationship between Julius Caesar and Cleopatra. Julius Caesar was known to have had close relationship with Cleopatra VII of Egypt, but their relationship was seen as political alliance rather than a romantic partnership.
User Prompt
The Manhattan Project understand that it resulted in the first atom bombs, but why held as such an important event in history And why are there so many conspiracies surrounding it
42

---

Published as a conference paper at ICLR 2026
LLada-8B (No Watermark)
The Manhattan Project was massive, top-secret effort create the first atomic bombs during World War II. It's considered a significant event in history because it ended the war and changed the way people thought about the world.
The reason it's important history because marked the first efforts create weapon of mass destruction, paving the way for the development of nuclear weapons. The project also led to the creation of the first successful nuclear weapons, which changed the course of human history.
There
many conspiracies surrounding the Manhattan Project be- cause it was intentionally kept secretive to ensure that the Nazis got no chance to develop similar weapon. The project involved some of the most brilliant minds, scientists, and engineers of the time, and the complexity of keeping such project under wraps.
There are many conspiracy theories about the project's origins, progress, and its consequences. For example, some people believe that the US government had a clear idea of the project's scope and chose to mislead the public, while others believe that they were deceived.
Additionally, the Manhattan Project's creation of nuclear weapons is still a concern today, and it has led to various conspiracy theories about government secrecy and the potential for nuclear to be stolen misused.
Overall, the Manhattan Project remains significant event in history, and the conspiracies surrounding it are driven by the fascination with its hidden secrets and the ongoing concerns about the potential for nuclear harm.
LL with C = (-1)
p-value: T.Te
The Manhattan Project was secret effort build the first-ever atomic weapons, spearheaded by the United States during World War II. It's the project responsible for the creation the atom bombs dropped on Japan Hiroshima and Nagasaki 1945. It was crucial event in history as it led Japan to surrender before Germany, effectively ending World War II Europe. It also led to the end of the Cold War and the United States gained significant influence on global power and politics.
As for the conspiracies, many aspects of the project were classified, and there was a culture of secrecy around the technology developed by the government the time. Some people suspected that the information about the project was either too slow or too accurate to be made public, and it led to speculation about certain aspects the project, including the involvement of key scientists and the potential consequences of the spread of the technology.
43

---

Published as a conference paper at ICLR 2026
LLada-8B with {-1,1}
p-value: 0.00
The Manhattan Project is held such important event in history because it resulted in the first atom bombs, which changed the world forever. During the against Nazi Germany, they wanted hurt lot of people in place. Now, scientists had really big weapon that could also hurt many people in one place, which made the end of the war casier.
Now, as for the many conspiracies surrounding it. There are always ques- tions about who helped or who did what really happened. Some people believe the Allies might not have made the bombs, they think was way to help them. There are also theories about how much information was given the U.S. government or the U.S. officials and scientists.
Some people believe that the important work on the project was actually done by people outside the U.S. government, and the U.S. government was not given the proper credit for the work. They also believe that the people involved in the project, such the British intelligence, nuclear Fermi, and the Czech nuclear scientists, were more involved than what we thought they did.
Lastly. some people believe that the U.S. government didn't know about the bomb, and that the U.S. government didn't know about it either. this theory, some people believe that the U.S. government knew about the bomb and they didn't 't want to use them; they just thought the Germans would eventually find a way use them.
User Prompt
want to talk about "Smooth", a song by Rob Thomas.
44

---

Published as a conference paper at ICLR 2026
Llada-88 (No Watermark)
"Smooth" is song by American country music singer Rob Thomas, released digitally on August 24, 2016, as the second single from his debut studio album, "Rob Thomas". The song was written by Rob Thomas, Luke Laird, and Blake Shelton, and was produced byBlake Shelton.
"Smooth" is mid-tempo country music song with catchy and upbeat vibe. features Rob Thomas in a smooth and romantic tone, showcasing his vocal abilities. The song's lyrics describe smooth, confident, beautiful woman who brings a sense happiness to the speaker.
The song received generally positive reviews from music critics and fans alike. It charted within the top 10 on the Billboard Hot Country Songs chart, peaking at No. 7. The song also peaked within the top 40 on the US Billboard Hot 100 chart, peaking at No. 35.
"Smooth" " was nominated for four Country Music Awards, including Record of the Year, Song of the Year, and Song of the Year. The song won for Song of the Year and Record of the Year in 2018.
The song has been performed and covered by many artists, including Tay- lor Swift, Keith Urban, Jason Aldean, and Luke Bryan, among others. Blake Shelton has also covered version of the song, which was also released in 2016.
LLada-8B with C = {-1}
p-value: 3.9e-6
"Smooth" a song by British singer and musician Rob Thomas, released in 1999 the lead single from his self-titled second studio album. The song was well- received and co-produced by Phil Collins, and was also written by Phil Collins and Brian Eno. The song became major commercial success, reaching number one in the UK. Australia. and New Zealand, as well as number 11 on the US Hot 100. The song's 's lyries were inspired by the experiences and relationship between Rob Thomas and his wife, and it revolves around themes of love, relationships. and nostalgia. The song features a smooth rock sound with soulful lyrics and catchy melody, and has become popular song in the genre.
45

---

Published as a conference paper at ICLR 2026
LLada-8B with C {-1,
p-value: 2.2e 6
"Smooth" is song by American singer-songwriter Rob Thomas. It was released on August 8, 1998, as the lead single from his self-titled debut album of the same year. The song was written by Rob Thomas and Bryan Sutton, and produced by Rob Thomas.
"Smooth" is a country rock ballad, characterized by smooth melody and Thomas's distinctive vocal style. The song's lyrics describe romantic tone and love story, making it a popular and beloved track in both country and pop audiences.
The song had significant impact the Billboard charts, reaching num- ber one on the Adult Contemporary chart and number six on the Hot 100 chart. also reached number on the Hot Country Singles chart, and spent 21 weeks on the charts.
"Smooth" was also a critical and commercial success, earning Rob Thomas two Grammy Awards including Best Male Country Vocal Performance.
the 2000 film "The Wedding Singer", "Smooth" was featured as one of the main soundtracks of the movie.
In 2000, Rob Thomas performed in reality show, which was live-action version the movie, "The Wedding Singer" and as part of the show, he performed "Smooth"
46


Reviews:
[{"review_id": "KxRrdax8yZ", "reviewer": "Reviewer_d3D1", "summary": "This paper introduces a watermarking framework for Diffusion Language Models. The paper reframes watermarking as a constrained optimization problem: maximizing the expected green token ratio, subject to a KL divergence constraint to preserve text quality. The solution to this problem yields a novel logits-biasing algorithm with two key components: An \"expectation boost,\" which applies the standard Red-Green boost in expectation over the distribution of the unknown (masked) context hashes. A \"predictive bias,\" which promotes sampling tokens that, in turn, are likely to make other tokens green when they are used as context. A significant advantage of this method is that it requires no changes to the standard Red-Green binomial test detector.", "questions": "- Algorithm 1 is theoretically complex. The current analysis of O(nL | C || Σ | log | Σ | ) seems to ignore gradient computation. \n- The runtime analysis (Figure 17) is critical to understanding this trade-off between cost and performance, but is relegated to the appendix. It should be in the main paper. Furthermore, the reported results in Figure 17 are questionable and lack error bars. For instance, the KGW ($|\\mathcal{C}|=1$) is slower than KGW($|\\mathcal{C}|=2$), which is counter-intuitive.\n- The paper is unclear on how the $\\epsilon$-parameterization (solving Eq. 4) is practically implemented.  Theorem 3.1 guarantees a unique $\\delta_t$ , and Section 3.2 briefly suggests using bisection to find it, but this seems computationally intensive.", "limitations": "", "rating": 8, "confidence": 3, "soundness": 3, "presentation": 3, "contribution": 3, "strengths": "The paper's primary strength is its novelty. It correctly identifies a fundamental incompatibility between context-dependent hashing and the arbitrary-order generation of DLMs. The reformulation as an optimization problem to maximize the expected green ratio is highly principled. The resulting two-component (expectation boost and predictive bias) algorithm is also clever and intuitive.", "weaknesses": "The main paper's comparison relies heavily on a \"naive\" baseline, which adapts KGW by only watermarking tokens where the full context is already unmasked . The most appropriate and direct baseline for a DLM is Unigram, which can be applied directly without any modification. All key comparisons in the main paper (especially for quality vs. TPR in Figure 2 and robustness in Figure 3) should be benchmarked against Unigram.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "R28kCq0w4t", "reviewer": "Reviewer_omL7", "summary": "The paper introduces the first watermarking method for diffusion language models (DLMs) built upon the green–red watermark framework. It argues that watermarking schemes designed for autoregressive models are not directly applicable to DLMs because these methods require computing a hash value from the context tokens (e.g., the previous $k$ tokens). In diffusion language models, however, parts of the context are often masked—that is, the corresponding tokens have not yet been sampled—making such hash-based watermarks infeasible.\n\nTo address this limitation, the paper proposes injecting watermark signals at every diffusion step, operating directly on the distribution of context tokens rather than on specific sampled tokens. By leveraging the hash distribution derived from the model’s probabilistic context, the method adjusts the token probability distribution to embed the watermark. Experimental results show that this approach achieves higher detectability and greater robustness compared to baseline methods that apply watermarks only when all context tokens are fully sampled.", "questions": "Please refer to my earlier questions in the weaknesses.\n \nMy main concern is the practical significance of the proposed approach compared to Unigram when applied to diffusion language models (see Weakness 2). \n\nI also noticed that Appendix~B discusses the limitation of Unigram vulnerability to scrubbing attacks. However, it remains unclear to me why the proposed method would be more robust to scrubbing. This is especially questionable given that the proposed algorithm uses a small context size ($|\\mathcal{C}| = 1$ or $2$). Increasing the $|\\mathcal{C}|$  may increase the robustness against scrubbing for the proposed method, but it would further raise computational complexity (which is already high). \n\nIn addition, there is an inherent trade-off between robustness to scrubbing and spoofing, so criticism of the Unigram watermark in its scrubbing vulnerability is not fully convincing.", "limitations": "", "rating": 4, "confidence": 4, "soundness": 4, "presentation": 2, "contribution": 2, "strengths": "1. This is the first work to investigate watermarking for diffusion language models.\n2. The paper formulates watermark injection as a constrained optimization problem, offering a principled framework for embedding watermarks during the diffusion process. \n3. The theoretical analysis is well-grounded, providing clear mathematical justification for the proposed method.", "weaknesses": "1. The empirical comparison is not fair. The paper argues that the primary limitation of applying autoregressive watermarking methods to diffusion language models lies in the presence of masked context tokens, which prevent computing context-dependent hashes. Therefore, the most appropriate baselines for comparison should be Unigram and PatternMark, where the green–red list is determined independently of the context, and thus do not suffer from the limitation identified by the authors. For a fair evaluation and to substantiate the claimed limitation of autoregressive watermarking, the authors should compare their method against Unigram and PatternMark as the main baselines. Both methods can be straightforwardly adapted to diffusion language models.\n\n2. The additional benefit of the proposed algorithm appears marginal compared to the correct baselines. In Figure 5 of the Appendix, the authors compare their method with Unigram and PatternMark, claiming that “Our Watermark is More Performant than Prior Order-Agnostic Watermarks.” However, from the figure, when the TPR reaches $1$, the log perplexity of the proposed method is around $1.75$, while Unigram achieves approximately $1.85$. Considering that Unigram is also faster ($8.19$ s vs. $11.52$ s, as shown in Figure 17), the roughly 0.1 improvement in log perplexity may not be significant enough to justify the added complexity. In other words, the proposed approach does not resolve any limitation that Unigram-based methods do not already handle.\n\n3. The proposed method incurs significantly higher computational cost compared to the baselines. In each diffusion step and for every token position, the method requires solving a constrained optimization problem. Even with the introduced approximations, the time complexity per token per diffusion step remains high—on the order of ($O(|\\Sigma|\\log|\\Sigma|)$).\n\n4. In Equation (2), I think the derivation appears to hold only under the assumption that:\n$q(H_t(\\omega) = h, \\omega_t = v) \\approx q_t[v], h_t(q)[h].$ Under this assumption, the hash value at position is independent of the current token (\\omega_t). This implies that one cannot arbitrarily choose the hash function; for instance, self-hash, where the hash depends directly on the current token, would violate this assumption. The paper should clearly state this assumption and discuss under what conditions it holds and how it affects the validity of the proposed framework.\n\n5. Some parts of the paper are difficult to follow. For example:\n- In Section 2, ($\\omega$) is defined as the text sequence. However, in Equation (5), the hash value is computed as the sum of ($\\omega_{t+i}$). It is unclear what this summation means until Equation (32), where it becomes evident that ($\\omega_{t+i}$) refers to the token IDs.\n- In Equation (6), the notation SumHash is ambiguous. It appears to correspond to ($H^{\\text{SumHash}}$), but this should be stated explicitly. In addition, the symbol ($s$) (where $(s \\in \\mathcal{H})$) is introduced without explanation, leaving its role in the equation unclear.\n- The explanation of the hash function in Appendix G is much clearer and should be integrated or summarized in the main text for readability.\n- After Equation (32), it is unclear what ($\\mathcal{N}$) represents. I am also confused about the cardinality of the hash space—is ($|\\mathcal{N}|$ or $|\\mathcal{H}|$) intended to be equal to ($|\\Sigma|$)? Please clarify the definition and size of ($\\mathcal{H}$).", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "6b7Jb6SRg4", "reviewer": "Reviewer_mAXZ", "summary": "This paper investigates the problem of instantiating watermarks for diffusion language models as opposed to autoregressive models.  A watermark is a statistical signal hidden inside text that can be detected by anyone with access to a secret key but is intended to not distort the quality of text, thus remaining undetectable to observers without access to a secret key.  Many schemes have been instantiated for autoregressive models by modifying the sampling procedure of models, with a prominent such scheme being the green list approach, where a hash function looks at the recent context and returns a pseudorandom subset of the vocabulary to upweight in generation.  While this has proven effective in autoregressive models, it is not immediate how to instantiate this scheme with masked diffusion models.  This paper suggests casting watermarking as optimizing a detectability function subject to a KL constraint to ensure maintenance of text quality.  They then instantiate the watermarking scheme with two different hash functions and evaluate their approach empirically, demonstrating success relative to the naive baseline of using the autoregressive approach.", "questions": "See weaknesses.", "limitations": "", "rating": 4, "confidence": 3, "soundness": 3, "presentation": 3, "contribution": 2, "strengths": "This paper identifies a clear problem with the well known greenlist approach to watermarking autoregressive language models when applied to diffusion models and proposes a solution that is empirically effective relative to the naive baseline.  The scale of the experiments is reasonable and the results appear promising and relatively robust.", "weaknesses": "One major weakness is the much slower sampling time.  While the authors remark in the figure 17 caption that the watermarking overhead is negligible, their empirical results suggest that there is an almost 30% increase in sampling time for their fastest instantiation relative to no watermark, which does not seem negligible to me.  \n\nI also think that the paper could benefit from further comparison with existing baselines.  There exist watermarking approaches that embed the watermark directly in the weights of a model, with *GaussMark: A Practical Approach for Structural Watermarking of Language Models* being an example; it seems like such an approach could directly translate into diffusion models and potentially be competitive as it does not rely on the autoregressive nature of distribution.  Can the authors comment on this?\n\nFinally, I think a number of earlier works, including Synth-ID and the Gaussmark paper cited above suggest that perplexity alone is insufficient to measure text quality.  I wonder if the authors have experimented with more useful proxies for the extent of distortion of their approach?", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "GNY4XpgYLE", "reviewer": "Reviewer_3nzz", "summary": "This paper introduces the first watermarking framework tailored for Diffusion Language Models (DLMs), demonstrating its effectiveness through theoretical analysis and comparative experiments. The method achieves high robustness while maintaining text quality.", "questions": "1. The baseline in this paper adds watermarks only when the context is fully known, which seems unreasonable, as many tokens in the DLM generation process are masked. Why not use the Unigram or PatternMark methods from the appendix as the baseline?\n2. Since the entropy-based remask strategy performs better, why does the main experiment in the paper use a random-based remask strategy?\n3. The elegant decomposition of \"Expectation Boost\" and \"Predictive Bias\" heavily relies on SumHash and the special case $C ={-1}$. Does this intuitive explanation still hold for more general hash functions (e.g., MinHash) and larger context windows?\n4. The hash operation in this paper uses a global binary green list matrix, which seems very similar to the Unigram approach. Does this imply that without a global vocabulary, it is impossible to accurately recover the context of tokens during detection?\n5. In the ε-parametrization, the KL constraint does not effectively control the quality loss. Are there other more suitable constraints or objective functions that could be considered?\n6. Is there theoretical support for the convergence of the fixed-point iteration? Could it fail to converge under certain conditions? The observation that a single iteration is sufficiently good seems more like an empirical finding rather than a result derived from theory. Does this suggest that the optimization problem is not inherently complex?", "limitations": "", "rating": 4, "confidence": 5, "soundness": 3, "presentation": 2, "contribution": 2, "strengths": "1. The experiments in this paper are comprehensive, covering multiple dimensions of DLMs and comparing them with ARLMs.\n2. The paper provides a formal definition and derivation of watermarking for DLMs, which has significant theoretical implications.", "weaknesses": "1. The efficiency of the watermarking method proposed for DLMs is relatively low.\n2. The method in this paper is limited to the red-green vocabulary watermark and does not consider other watermarking schemes.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}]

Author rebuttal:

Decision: Accept (Poster)