Learning to Watermark: A Selective Watermarking Framework for Large Language Models via Multi-Objective Optimization
Authors: Chenrui Wang
Junyi Shu
Billy Chiu
YU LI
Saleh Alharbi
Min Zhang
Jing Li
Venue: NeurIPS 2025 poster ()
Keywords: LLM
watermark
Selective Watermarking
multi-objective optimization
Primary area: deep_learning
TL;DR: A selective watermarking method for LLMs via a lightweight network trained on multi-objectives

Abstract: The rapid development of LLMs has raised concerns about their potential misuse, leading to various watermarking schemes that typically offer high detectability. 
However, existing watermarking techniques often face trade-off between watermark detectability and generated text quality. 
In this paper, we introduce Learning to Watermark (LTW), a novel selective watermarking framework that leverages multi-objective optimization to effectively balance these competing goals. 
LTW features a lightweight network that adaptively decides when to apply the watermark by analyzing sentence embeddings, token entropy, and current watermarking ratio. 
Training of the network involves two specifically constructed loss functions that guide the model toward Pareto-optimal solutions, thereby harmonizing watermark detectability and text quality.
By integrating LTW with two baseline watermarking methods, our experimental evaluations demonstrate that LTW significantly enhances text quality without compromising detectability. 
Our selective watermarking approach offers a new perspective for designing watermarks for LLMs and a way to preserve high text quality for watermarks. The code is publicly available at: https://github.com/fattyray/learning-to-watermark

Learning to Watermark: A Selective Watermarking Framework for Large Language Models via Multi-Objective Optimization
Chenrui Wang' Junyi Shu'
Billy Chiu²
Yu Li
Saleh Alharbi Min Zhang' Jing Li
Harbin Institute of Technology, Shenzhen, China
Lingnan University, China *Zhejiang University, China *Shaqra University, Saudi Arabia jingli, jingli.phd@hooe phd@hotmail.com
wangchenrayl @gmail. com
Abstract
The rapid development of LLMs has raised concerns about their potential misuse, leading to various watermarking schemes that typically offer high detectability. However, existing watermarking techniques often face trade-off between water- mark detectability and generated text quality, In this paper, we introduce Learning to Watermark (LTW), novel selective watermarking framework that leverages multi-objective optimization effectively balance these competing goals. LTW features lightweight network that adaptively decides when to apply the watermark by analyzing sentence embeddings, token entropy, and current watermarking ratio. Training of the network involves two specifically constructed loss functions that guide the model toward Pareto-optimal solutions, thereby harmonizing watermark detectability and text quality. By integrating LTW with two baseline watermark- ing methods, our experimental evaluations demonstrate that LTW significantly enhances text quality without compromising detectability. Our selective watermark- ing approach offers a new perspective for designing watermarks for LLMs and a way preserve high text quality for watermarks. The code is publicly available https://ggtubb..oo////ttaaaalal waw.raamiinntt-tttamaatt
1 Introduction
The rapid progress of large language models (LLMs) [26, 40, 45] has brought both remarkable capabilities [4, 14, 27, 41] and serious risks of misuse, including copyright concerns [11, 31], academic misuse [36] and other malicious purposes [23, 24]. Consequently, watermarking techniques have emerged essential tools for detecting and tracing Al-generated texts.
These watermarking techniques embed signals that are imperceptible to humans yet detectable by watermark detection algorithms. Among these methods, the KGW watermark [12] partitions the vocabulary into "green-list" and "red-list" groups, subtly biasing token selection toward the "green- list". This allows detection of watermarked content via statistical hypothesis testing. However, despite their imperceptibility. these watermarks can degrade the text's semantic coherence, as the systematic bias influences token selection contrary natural model preferences. Recent attempts address these quality concerns include the TS-watermark [9], which adaptively adjusts watermark strength and green-list ratios; the NS-watermark [37], which limits the extent of watermarking improve text quality; and EXP- [16], which proposes a distortion-free watermark integrated during the sampling process.
Corresponding author.
39th Conference on Neural Information Processing Systems (NcurIPS 2025).

---

However, these methods introduce significant trade-offs. The TS-watermark restricts user flexibility by preventing manual adjustment of watermark parameters when either stronger or weaker water- marking is desired. The NS- -S-watermark significantly prolongs text-generation time, and its focus on maintaining text quality compromises detectability. Under its minimal watermarking constraints, the resulting z-score barely surpasses detection thresholds, leaving it vulnerable to trivial attacks as changing just one "green-list" token to a "red-list" token-to evade detection. The EXP-edit method, meanwhile, requires hundreds of inference steps for watermark detection, resulting in impractically long detection times, often minutes for even short texts. In contrast, selective watermarking methods aim to watermark only carefully chosen subset of tokens during text generation. This selective process relies on reproducible selection criteria at detection, enabling efficient watermark verification exclusively these marked tokens. One notable selective watermarking approach is SWEET [17], proposed specifically for code generation. However, its reliance on manual determination of en- tropy thresholds, derived through extensive grid-search analyses across sample datasets with varying watermark strengths and "green-list" ratios, hampers its practical usability in real-world applications.
In this paper, we propose novel selective watermarking approach. Rather than relying exclusively on entropy as selection criterion, we introduce a lightweight multilayer perceptron, referred to as the Selector Network. This network leverages the sentence embeddings of previously generated text, current token entropy, and the ratio of watermarked tokens within the generated content to adaptively determine when to apply the watermark. By training the Selector Network through multi multi-objective optimization -sing specifically constructed detectability and quality loss functions and the Multiple Gradient Descent Algorithm [5, 35]. we guide it toward Pareto-optimal watermarking decisions. This ensures an effective balance between maintaining high watermark detectability and preserving the quality of generated text. In summary, our contributions are follows:
Analysis: We find existing selective watermarking method [17] underexplored potentially informative factors that may be used as criterions for selection. We are the first to propose the method of utilizing a trained network to make decisions on whether to selectively apply watermark, unveiling a new perspective of selective watermarking strategies.
Method: We propose LTW, novel selective watermarking framework that uses trained lightweight network for selectively watermarking L.LMs. We introduce ITT-1 and LTW-0, by applying our selective framework to baseline watermark KGW and Unigram.
Evaluation: We conducted extensive experiments across multiple models, demonstrating the high text quality and detectability of our methods. We surpass previous watermarking methods in text quality, having the least perplexity while without compromising detectability.
2 Related Work
Watermarking Methods. The rapid advance of LLM and their potential misuse have prompted the need to watermark LLM-generated text to distinguish them from human- humann-witten written text. Currently, text watermarking methods can be divided into two types, watermarks for generated texts [1, 2, 28. 32, 34, 38, 39, 42] and watermarks for LLMs. The former modify existing text to produce watermarked text, these methods are usually format-based [2, 28, 32, 34), lexical-based [39, 42] or syntax based [1, 38). For the latter, which takes place during logits generation or during token sampling, implants the watermark during LLM generation. KGW-based [12] methods modifies the logits according to a green/red list, while sampling methods [3. 16] such as Christ's [3] and EXP-edit [16] watermark by using their pseudo random sampling methods. Recent works of LLM watermarking are proposed to reduce text quality degradation caused by adding watermarks or to increase their robustness under attacks. For example, Unigram [46] uses a fixed random key to produce their "green/red list", while SIR [19] uses semantics to determine the "green- "green-list" to improve robustness under attack. Speeiff Watermark [9] trains y and 8 generators to alter these hyperparameters during token generation to obtain better semantic coherence. To address the low-entropy limitation that previous works [3, 12, 17] has shown, SWEET [17] provided a selective watermarking method to improve the quality of watermarked codes generated by LLMs, and EWD [22] provided entropy- -basdd detection method to improve detection ability.
Balancing Detection and Quality. Recent works [8, 9, 18, 21, 37] indicate watermarking brings text quality decay and some works [6. 17] indicate that in some tasks such as coding, watermarking
2

---

Origialion Logit
Logits
oigia loght
Ivtrycne
LLM
Saler
Scany
Moall's usccode of The Mikkes Pedkaal
speacdE
prozuced the Niscoai
Adlaptive Threshold
Organd Legit
Scnoaaiion
Fetwork
ppoooo para TOOATO
watemmrked
CORTREE CONEM
mank-la oupus loo.
rileced dehenenaaaatinn
Owaroness
aurdariy
L22 C2 E12 Can Lnd DUK
Figure I: An illustration of our selective watermarking method. (1) The upper part illustrates that at each generation time step, the watermark collector concatenates the calculated input for the trained Selector Network, whose output is compared to an adaptive threshold to decide whether to watermark the current token. (2) The lower part is about constructing the detection oriented and text quality oriented loss, and utilize the MGDA algorithm to tackle this multi-objective optimization problem and optimize towards Pareto optimal.
causes notable performance decline. It is understandable some crucial tokens the LLM should generate were changes due to watermarking, resulting in the problem. Methods for solving this include detection trade-off methods [18, 37] and selective watermarking method [17]. NS-watermark [37] indicates that a minimal number of watermarked text is needed to achieve detection, and developed watermark meet the constraint; however, the method hurts detection ability because unlike selective watermarking, without the ability to determine which tokens are selected and only detecting them, the method has detect all generated text while only a fraction of them are watermarked, leaving their watermark with a low z-score and can be easily removed under their minimal constraint. Other detection trade-off method [18] add bias to tokens that are in the "red-list" but are deemed important. SWEET [17] proposed selective watermarking method based on entropy threshold, which only watermarks and detects high-entropy tokens. Some work [21] has applied the selective watermarking method as a pre-processing step for improving text quality in their watermark.
Multi-Objecd Optimization. In LLM watermarking, detection and text quality are a set of optimization objectives that often conflict with each other. Previous work [9] addressed this problem by using the Multiple-gradien Descent Algorithm (MGDA) [5, 35] to optimize the two objectives. They trained and d generators, which takes the embedding of the last token as input, to adaptively provide these two hyperparameters for watermarking and detection.
3 Methodology
We believe that the selective watermarking approach holds significant promise for simultane- ously achieving high watermark detectability while preserving text quality. However, existing research [17, 21] has focused primarily on using entropy as the sole criterion for selection, leaving other potentially informative factors underexplored. In this work, we followed prior work of selec- tive watermarking [17] and proposed incorporating semantic embedding of the previous text and watermarked ratio into the selection process together with entropy for selective watermarking. We
3

---

take them as input for our trained network to determine when to selectively add the watermark. We introduce our novel approach for selective watermarking, LTW.
3.1 Problem Formulation
Following the standard approach for watermarking LLMs [12] by modifying the output logits I E IR VI at each generation step t. for each token v in the vocabulary V. is assigned to the green-list or red-list according to a pseudorandom generator R, seeded with a key k. V is shuffled to divide the vocabulary V into two disjoint subsets Lgreen, Led = Shuffle(V, RR( (((), 7) where Lgreen = y V and
Y E (0, 1) is a hyperparameter controlling the green ratio and 5 is the hyperparameter controlling the watermark strength. From this partition, the green-list mask Mgreen Lgreen Lisa) E {0, 1 is denoted as:
1, Legreen 0.
m Mgreen [u]
(1)
V Lied
We introduce lightweight multi-layer perceptron Mo. which produces mask for deciding the appropriate generation time step to add the watermark. To enable gradient backpropagation during training, the network is designed binary values such as those from Bernoulli distribution that are not differentiable. To encourage the output values to be close to either or I, thereby promoting a mask-like behavior, we introduce regularization term in the loss function that penalizes uncertainty and drives the predictions towards binary-like extremes. We denote the token sequence at the current timestep as [&,, f.,..., En]. Eyem as a semantic model which produces a sentence embedding according to previous k tokens. Let E R denote the Shannon entropy of the token probability distribution the current generation step and let [0, 1] denote the current proportion of watermarked tokens in the generated sequence.
produce continuous output mam E [0, 1], rather than discrete
mwm Ma (5nm(nnk++mm) e, r) E [0, 1]
(2)
Combining the green-list and selective mask, we obtain the final perturbation mask over the logits, and the modified logits are given by:
1 14+1 + 5 m ml(+1] mgrecn (Lgreen Lead
The selectively watermarked logits, are then transformed into probability using the softmax function, and used for sampling the next token, the process continues until reaches the maximum number of new tokens or the cos token.
The detection process is formulated as z-test: 2 = 1ala->Tea The number of all the selected
/ x1n tokens are denoted as Twm Ei= Newm and |s|a "T mmm Pgr Per is the probability
[t]
"
[t]
of selecting a green token, relaxing |slc from the total number of green tokens to this makes differentiable for training.
The equations above are our relaxed version of the standard watermarking procedure [12] to make it consistent with our training process. The continuous and differentiable mwm together with differentiable approximation of enable gradient-based optimization of the model parameters. During inference, however, mwm is thresholded by a predefined hyperparameter
(0, 1) as the following Equation suggests, and [t] is either 0 or depending on whether the sampled token is in the gieen-list. Thus making our equation consistent with the standard process [12]. Further proof of our method is provided in Appendix A.
II if mmm > T
Mww
(4)
0. otherwise
3.2 Structure Design
Selector Network. The Selector Network is the module in our work that decides whether the current token should be watermarked or not. The network is multilayer perception [33] that takes the concatenated information during generation as input. These generation related information includes the sentence embedding of the previous tokens, the entropy of the probability distribution for the token to predicted, and the proportion of generated tokens that are selected to be watermarked. These information can also be obtained during detection, allowing trace back to those selectively

---

watermarked tokens to be detected on. We adopt SimCSE [7] to provide the sentence embedding which is used as input by our network. The obtained embeddings are first subjected to dimensionality reduction via the MIPP layers, then the reduced representations are concatenated with two additional features and fed into a second MLP for further dimensionality reduction. Finally, the model produces an output mmm E [0, 1]. Further discussion of its design is given in Appendix E.
Watermarking Information Collector. Compared to existing watermarking methods, our approach requires the collection of more comprehensive information during watermarking, To facilitate this, we implement an information collecting module that maintains contextual data from previously generated tokens. Specifically, at each generation step, the token id or embedding of the newly generated token appended to this module. and the sentence embedding constructed based on the last k tokens. Furthermore, the watermarked ratio information is updated according to whether the previous token was watermarked. We also compute the entropy of the probability distribution for the current token in this module. These provide the adequate information and are concatenated as input to be used by our network during training or watermarking.
Adaptive Threshold. In Section 3.1, we define Equation 4. During inference, we need to discretize the output mwm E [0, 1] into a binary decision of or 1, indicating whether to watermark the current token being generated or not. A dynamic threshold is applied for this purpose. A lower threshold is used when the current watermarking ratio is low, allowing more tokens to be watermarked and thus improving detectability; Conversely, higher threshold is adopted when having sufficiently high watermarked ratio.
3.3 Learning watermark: Training for Selective Watermark Insertion
Our goal is to train a selective watermarking module capable of making informed and appropriate watermark insertion decisions. The output of the selective watermarking module continuous rather than discrete binary value, serving as a differentiable signal that enables training through gradient- based optimization. Our objective is achieve high watermark detectability and high generation quality through selective watermarking, which naturally formulates as a multi-objective optimization (MO0) problem. To address this, we adopt the Multiple Gradient Descent Algorithm (MGDA) [5, 35] used by previous work [9] to effectively handle these competing objectives. Moreover, we expect the module to exhibit the following properties:
First, the output should exhibit mask-like behavior, encouraging the module to make sharp, near-binary decisions regarding whether to embed a watermark.
Second, the module should be entropy-aware. It should prefer watermarking when the entropy over the probability distribution of the next token is high, and avoid watermarking when it is low, This design is motivated by prior works [17, 22], which have demonstrated that applying watermarks to low-entropy tokens can lead to degraded text quality and reduced watermark detectability.
Third, the module should provide adaptive control based on the current watermarking ratio. Specifically, when the overall proportion of watermarked tokens is low, the module should be more inclined to add watermarks to maintain detectability. Conversely, when the ratio is already high, should reduce watermarking to better preserve text quality.
These goals and behavioral characteristics can be formulated as two distinct optimization objectives:
Preserving Quality. Part of this is trying maximize the semantic similarity between the marked text and the unmarked one, aiming to minimize the quality drop brought by adding watermarks. This is achieved by computing the cosine similarity between the sentence embedding of the watermarked output embedding and the wotemartrrdreed one. E., and E, denote the semantic embedding in the following loss function 5. The objective also incorporates entropy- aware behavior: the module is encouraged to apply watermarks to high-entropy tokens while avoid applying those with low-entropy we formulate the following loss function in Equation 6 to describe this objective. In which de and Me are a set of hyperparameters, and e = Σm= P4 log P. represents the current entropy.
Ls = COssim (Ew Es)
(5)
Lentropy
BCE (mmm σ (eee (s)))
(6)
5

---

Maximizing Defectability. Second, we seek to maximize watermark detectability, which involves |sla I To make the z-score differentiable, increasing the overall z-score used for detection v vTaa(11-7)
instead of the sum of the green tokens among those chosen to be watermarked, it is relaxed to the probability of producing green token among those chosen. This can be denoted as |s|a
Ei=1Ph mmm forming the following Equation 7. Furthermore, this objective promotes adaptive watermarking behavior: when the overall watermarking ratio is low, the model is encouraged to watermark for ensuring detectability; While when the ratio high, it prefers to watermark less, thereby preserving generation quality when detectability can be ensured, we denote Tr the current watermarked ratio, f 10. + (0, 1) as a monotonically decreasing function, we construct the following objective the loss function in Equation 8.
ET = Mwm
1= Par mI
1t] V VET
mmm 7(1 -7)
fuom_ ratio = MSE
mm/mm s(re))
(
(8)
The following loss function in Equation describes the need for mask-like like output, penalizing those unmask-like outputs. It is applied to both the following objectives to make the output mask-like.
1 T
Σ
(mmm 0.5
Loutput_fia fix
These objectives are jointly formulated as the loss functions in Equation 10 and 11. To achieve a Pareto-optimal solution between quality oriented LQ and detectability oriented Lp. the MGDA [5, 35] algorithm is adopted to determine a descent direction towards it. Further details are provided in the Appendix G. LQ -D= -> + humbuum_reee + Aoitpuut
Asim Ls + Nentropy-enttrope + Aiizooutpat_ffa fiz
(10)
(11)
where Ls denotes the similarity loss presented in Equation 5, Lentropy denoted in Equation is loss for achieving entropy awareness, Aeim and Xentropy are the weighting hyperparameter for the two loss functions. In Equation 11, 2 denotes our differentiable z-score described in Equation 7, Cum_rata presented in Equation 8 is the loss for incorporating ratio awareness behavior, X₂ and Awm are the weighting hyperparameter for them. The Loutput_fta in both functions is for promoting mask-like outputs, Azaz is its weighting hyperparameter.
4 Experiments
4.1 Experiment Setting
Datasets and Models. We trained and tested method on the RealNewsLike subset of the C4 dataset [29], following previous watermark works [8, 9, 12, 13, 21]. We used 10,000 texts for training, and randomly divided another 500 texts test set. We trained our selective watermarking module using OPT-1.3b [45]. Further discussion about training are provided in Appendix F.I. We then evaluated the performance of our watermark and other watermark methods using OPT-6.71 7b and GPT-J-6B [40]. After which, we conducted paraphrase attack on the watermarked texts using the Dipper model [15], we followed previous work's [13] parameter settings, using lex diversity of 60.
Baselines. We applied our selective watermarking method to KGW and Unigram to evaluate the effectiveness of our watermarking method, we will denote them LTW-1 and LTW-0 in the following text, the procedure of our selective method can be found in Appendix C. We compared our methods with KGW [12], Unigram [46], EXP-edit [16]. SWEET [17] and TokeifSpecifie [9] under fixed hyperparameters settings. To further amalyze the trade-offs between detectability and text quality, as well as between robustness and text quality, we varied the watermark strength 6, generated watermarked texts using our methods and KGW for each strength, and compared their performance by fitting a curve for comparing the trade-off between TPR and perplexity. Further discussion on hyperparameters settings and experiment details are provided in Appendix F.2.

---

Table 1: Performance comparison of different watermarking methods on OPT-6.7B in terms of Detectability, Robustness, and Text Quality. Higher performance when not under attack indicates higher detectability, higher performance under paraphrase attack shows the robustness of watermark method. Lower perplexity and higher similarity with reference text indicates higher text quality.
No Attack
Dipper Attack
Text Quality
Method
TPR @2
AUROC
Best F1
TPR@ 10
AUROC
Best F1
Perplexity
Similarity
KGW
0.998
0.99990
0,99900
0.910
0,96987
0.91465
20.5234
0.5472
Unigram
1.000
1.00000
1.00000
0.962
0.98217
0.95238
17.6761
0,5313
EXP-edit
0.972
0.98742
0,98008
0.790
0,90527
0.83871
23.9298
0,5046
SWEET
1.000
1.00000
1.00000
0.924
0,96955
0,92245
17.9301
0.5554
TS--wttemmar
0.996
0.99973
0,99598
0.810
0,93179
0.85076
14.0804
0.5645
LTW-1 (ours)
1.000
1.00000
1.00000
0.852
0,94034
0.87810
13.6205
0.5701
LTW-0 (ours)
1.000
1.00000
1.00000
0.954
0,98326
0.94190
13.6238
0.5735
Table 2: Performance comparison of different watermark methods on GPT-J-6B.
No Attack
Dipper Attack
Text Quality
Method
AUROC
Best
TPR@1
AUROC
Best F1
Perplexity
Similarity
KGW
1. 1,00000
1,00000
0.800
0.93668
0.87125
16.5485
0.4941
Unigram
1.00000
1.00000
0.944
0.97938
0.94586
13.0226
0,4634
EXP-edit
0.99996
0.99701
0.860
0.93651
0.88377
21.8741
0,4498
SWEET
1.00000
1.00000
0.878
0,94406
0.88777
15.2203
0,5049
TS-watermark
1.00000
1.00000
0.822
0,93150
0,86068
11.7829
0.5220
LTW-1 (ours)
1.00000
1,00000
0.858
0,94813
0.88303
11.6877
0.5286
LTW-0 (ours)
1.00000
1.00000
0.926
0.96238
0.91952
9.5571
0,4822
Evaluation Metrics. To evaluate watermark detection performance and robustness against para- phrasing attacks, we employed multiple evaluation metrics: AUROC, Best Fl score, and True Positive Rate (TPR) specified False Positive Rate (FPR) thresholds. We compared the quality of the wa- termarked text using two evaluation metrics: Perplexity and cosine similarity between the sentence embeddings of generated text and the reference text. For each text in the test 200 words following the prompt segment as human- written reference text. Sentence embeddings are obtained using the CotenceeBERR' model [30].
designate the
4.2 Experimental Results
Tablel and Table 2 show results of our water- mark method and baseline methods in terms of detectability when not under attack, robustness when under paraphrase attack as well as the text quality of the watermarked text generated us- ing OPT-6.7B and GPT-J-6B. Figure 2, and 4 present a detailed comparison of text quality in the main experiment described above. Figure 5 and 6 show the results of our method with KGW under different watermark strengths. An example of our watermark can be found in the Appendix D, in which compared with baseline methods, our method not only improved text quality, but also improved detection ability.
1.0
Watermark Methad
Cosine similarit
0.8
KGW
0.6
Unigram
EXP-EdIt
0.4
SWEET
0.2
TS-weterma
LTW-1
0.0
LTW-0
GPT]-6B
Figure 2: Comparison of the Cosine Similarity be- tween reference text and watermarked text across various LLMs.
Comparison on Text Quality. In Figure 3 and we compared the perplexity of our watermark method with the unwatermarked text and other baseline methods. A lower perplexity suggests better text quality. Among the watermark methods, our methods have the least perplexity, implying the least quality degradation among the baselines. In Figure 2, We evaluated text quality by computing the cosine similarity between the sentence embeddings of the original text and the generated texts, higher similarity suggests better quality and semantic coherence. We observe that, overall, methods have better semantic similarity compared

---

35-
35
30
Perpleetty
25
20
20
10-
10
Unwetermaring KSW
Unigram
EXP-6UI
SWEET
1Swratomark
LTW-3
Lnw-9
Unwatermarked KGW
Unigram
EXP Edi
SWEET
1S
unw.
Figure 3: Distribution of perplexity among the unwatermarked text generated using OPT-6.7b and that of texts watermarked using various wa- termark methods.
Figure 4: Distribution of perplexity among the unwatermarked text generated using GPT-J-6b and that of texts watermarked using various wa- termark methods.
FPR=19
FPR=10%
1.000
0.9
TPR (better -]
0.975
TPR (better
0.8
0.950
0.7
0.925
KGW
KGW
LTW-1
0.6
LTW-1
0.900
LTW-0
LTW-0
0.5
12.5
15.0
17.5
20.0
22.5
12.5
15.0
17.5
20.0
22.5
perplexity (better -)
perplexity (better
Performance comparison between our watermarking method and KGW watermark in terms of perplexity and TPR across various watermark strengths.
Figure 5:
Comparison of method and KGW under Dipper attack in terms of perplexity and TPR across various watermark strengths.
Figure 6:
with other watermarkmethods. Among which, LTW-1 achieved better semantic similarity than all five baseline methods, indicating better semantic coherence and text quality. This performance improvement can be attributed to our selective watermarking mechanism, which learns to insert watermarks based semantic embeddings and entropy, tending to watermark tokens which carry less semantic significance, while omitting those more critical tokens, thereby minimizing disruption to the overall sentence meaning.
Comparison Detectability. In Table and 2, compared the detectability of our watermarking methods with baseline methods on OPT-6.7B and GPT-J-6B under fixed watermark strength. We observe that, although most watermarks do well in detectability, our method consistently demonstrate highly competitive performance across all detection metrics. Furthermore, in Figure 5, we conducted additional experiments comparing the performance between our watermark and KGW across varying watermark strengths by increasing 8. The results show that our approach is clearly superior to KGW, achieving more favorable Pareto frontier than offering improved detectability at the same level of perplexity. Compared to KGW, our methods yield significantly lower perplexity at the same watermark strength and achieves TPR
much lower strength.
Comparison on Robustness. In Figure we compared the robustness our method and KGW against paraphrase attack across varying watermark strengths. The results show that our methods can achieve superior Pareto frontier, showing more robustness than the baseline method KGW. Results on Table and 2 also show that L.TW-O can outperform most baseline methods in robustness while still having lower perplexity. Though method like TS watermark can achieve text quality similar to they are less robust when under paraphrase attack.

---

Table 3: Performance comparison of detectability and quality between using the adaptive threshold module and after removing the adaptive threshold module under different watermark strengths.
Settings
Metries
6 1.5
2.5
8 3
6 3.5
Perplexity
11.686
12.354
13.003
13.410
13.981
14.029
With Z-score
6.844
9.293
11.243
12.670
13.776
14.497
Perplexity Z-score
11.801
12.524
13.217
13,403
13,922
14.308
Without
6.831
9.031
11.120
12.443
13,570
14.369
5 Analysis
Ablation Studies. In this section, we focus on assessing the effectiveness of our adaptive threshold module. We conduct ablation studies by removing this module and replacing it with a fixed threshold, and evaluate the perplexity and z-score across varying watermark strengths. The results in Table show that the adaptive threshold can improve detectability in all watermark strength settings, and having lower perplexity in 4 of 6 watermark strengths settings. Indicating that such module can help achieve higher detection abilities while having similar or even better text quality.
Analysis of Network Output. In this section, we explore the correlation between the output of the trained network and the input. Our network utilize the sentence embedding of previous text part of its input, though it is hard find a measurable representation for the embedding, it nevertheless is correlated with the part-of-speect (POS) at the current time step systematically. The results are shown in Figure 7. Our network tends not watermark adposition, conjunction, punctuations and symbols. Watermarking and altering them may yield semantically incoherent phrases, misinterpreted sentences and wrong equations. Our network more oftenly tends to watermark adverbs and adjectives, which often convey descriptive or qualitative information and have more replacements in the vocabulary. Watermarking them may have smaller influence on semantic coherence. We also evaluated the correlation between the output and the entropy input at the current time step, which has a positive correlation, encouraging watermarking when having high entropy, detailed discussion are in Appendix D. The results show that both semantic-leve features as well as entropy plays an important role in the process of making the selective decision our method.
Comparison on Our Two Methods. Unlike previous work [17] that only applied their selec- tive method to KGW, we applied our selective method to two representative watermarks, KGW and Unigram. In terms of text quality, both wa- termark variants exhibit significantly lower per- plexity and higher semantic similarity after in- corporating our selective watermarking method. KGW also shows improved detectability under the OPT model and enhanced robustness against paraphrasing attacks under GPT-J model after applying our selective method. While both of our methods yield similar text quality under the OPT model, they each excel in different aspects under the GPT-J model; Notably, the use of fixed green/red list in the Unigram approach proves beneficial to maintaining robustness under our selective watermarking Kinnn framework. The enhanced robustness of LTW-0 can be attributed to the use of fixed green/red list partition, mitigating the risk of detection failure caused by changes in the partition of the red- red-green list due to changed previous token when under paraphrase attacks. Moreover, selective watermarking alleviates the issue of systematically reduced probability of the important words in the fixed red list, minimizing degradation of text quality and semantic coherence for Unigram.
12
WHN PRON MOOW MONET CCON VIAA SCOrg)
XFU
JUDC
AEV
STM
1911
Figure 7: Distribution of the network output across different pfrt-of-speech categories of the token gen- erated at the current time step.
Analysis Measures taken for Robustness. Applying the hard binary mask instead of keeping the soft weights of the selector network is design for robustness. For the soft mask we used during training, the mask is also detection weight as Equation 7, token with a small mask is also given light detection weight. We believe that, comparatively, using the hard mask can better enhance the

---

robustness of the watermarking scheme under paraphrasing attacks. From a design perspective, it relatively simple to categorize the outputs of the selector into apply watermark and do not apply watermark, and only apply watermark to the first category. During detection, we only need to verify the outputs that were marked for watermarking, which a simple approach. However, if a soft mask was to be adopted at inference time, the detector would need to consider the soft mask values as per-token detection weights. In such a case, tokens with very low mask values would contribute minimally to detection, while the opposite for those with high mask value. While this offers finer granularity, it introduces more complexity and uncertainty-esseciilll making it more vulnerable to paraphrasing attacks.
Paraphrasing attacks are hard to change apply watermark to do not apply watermark compared with changing the output values. In our Equation 9, we penalize those unmasklike outputs, making most outputs further from the threshold, thus making it less sensitive to attacks. In contrast, soft mask values are continuous and more sensitive to subtle changes, meaning that a paraphrasing attack may cause small shifts in mask values (e-g.. a slightly increased value for a previously low-scored token, slightly decreased one for a previously high-scored token). These small perturbations can accumulate and increase the uncertainty of watermark detection, thus reducing robustness.
Choosing moderate window size for the sentence embedding used by the selector network is also important for robustness. A window size that is too small may cause the sentence embedding to degrade like those of a token-level representation, making it less robust under paraphrasing attacks. This is because paraphrasing typically preserves the overall sentence meaning while altering individual words through synonym substitution or structural changes. At the token level, such variations can affect the selector network's output, making less robust to these attacks. In contrast, by selecting an appropriately sized window, we can maintain semantic consistency with the paraphrased version, thus improving the robustness of the selector. On the other hand, with a too large window size, new tokens generated will have only minor influence to the content in the window, which may reduce the sensitivity the selector to the content being generated.
Justify for choice of metrics We have adopted perplexity and Similarity between the sentence embeddings as our metrics for evaluating Text Quality. Perplexity (PPL) a natural metric for judging watermark impact because it directly measures the negative log pogg-rbbbbyllty of the generated token sequence. If watermarking method perturbs the generation more, the generated outputs will typically become less likely to be generated and thus exhibit higher PPL. Generally, lower PPL indicates higher text quality[20), And it have been used as a metric in previous works [8, 12, 21, 22] as well. When the task has human reference answers such as the task of Text Completion, a higher similarity between the embedding of the reference text and the generated watermarked text captures whether the watermarked text still preserves task-relevant semantics and coherence, remaining appropriate and faithful to the reference text. Previous works [9, 25, 43, 44] have also adopted semantic score in their works and among which, some works [9, 44] also adopted sentence embedding models [0] for calculating the sentence-level score.
Conclusions
In this paper, we propose novel approach for selectively applying watermarks to LLMs, aiming to mitigate the quality degradation caused by watermarking while preserving high detectability. We constructed two distinet loss functions and employed the MGDA algorithm to solve this multi- objective optimization problem. Experimental results demonstrate that our approach achieves superior text quality without compromising detection performance. In summary, our method unveils a new perspective the design of selective watermarking strategies for LLMs.
Acknowledgements
This work was supported in part by National Natural Science Foundation of China (62476070), Shen- zhen Science and Technology Program (JCYJ20241 12022023000222030000000 GXWD20231 128 103232001, ZDSYS20033062099 1203008, 100 Department of Science and Tech- nology of Guangdong (202441510011540) and National Key R&D Program of China SS2204YFF0200055
128002300000033001
10

---

References
[1] M.J. Atallah, V. Raskin, M. Crogan, C. Hempelmann, Kerschbaum, D. Mohamed, and S. Naik. Natural language watermarking: Design, analysis, and a proof-of-concept implementation. In Information Hiding: 4th International Workshop, III 2001 Pittsburgh, PA, USA, April 25-27, 2001 Proceedings, volume 2137, pages 185-200, 2001.
[2] J. T. Brassil, S. Low, N. F. Maxemchuk, and L. O'Gorman. Electronic marking and iden- tification techniques discourage document copying. IEEE Journal on Selected Areas in Communications, 13(8): 1504, 1995.
[3] M. Christ, S. Gumn, and Zamir. Undetectable watermarks for language models. In The Thirty Seventh Anmual Conference on Learning Theory (COLT 2024), volume 247, pages 2024.
[4] H. W. Chung, L. Hou, S. Longpre, B. Zoph, Y. Tai, W. Fedus, Y. Li, X. Wang. M. Dehghani, Brahma, A. Webson, S. S. Gu, Z. Dai, M. Suzgun, X. Chen, A. Chowdhery, A. Castro-Ros, M. Pellat, K. Robinson, D. Valter, S. Narang, G. Mishra, A. Yu, V. Zhao, Y. Huang, A. Dai, H. Yu, Petrov, E. Chi, Dean, Devlin, A. Roberts, D. Zhou, Q. V. Le, and Wei. Scaling cnoruction-finetunne language models. J. Mach. Learn. Res., 25(1), 2024.
[5] J.-A Désidéri. Multiple- descent algorithm (mgda) for multiobjective optimization. Comptes Rendus Mathematique, 350(5):313-3185 2012.
[6] Y. Fu, D. Xiong, and Y. Dong. Watermarking conditional text generation for ai detection: Unveiling challenges and semantic-aware watermark remedy. Proceedings of the AAAI Conference on Artificial Intelligence, 18016))18003-1801 2024.
[7] T. Gao, X. Yao, and D. Chen. SimCSE: Simple contrastive learning of sentence embeddings. In Proceedings of the 2021 Conference on Empirical Methods in Natural Language Processing, pages 6894-6911 2021.
[8] Y. Guo, Z. Tian, Song, T. Liu, L. Ding, and D. Li. Context-aware watermark with semantic balanced green-red lists for large language models. In Proceedings of the 2024 Conference Empirical Methods in Natural Language Processing, pages 29663-226666,0. 2024.
[9] M. Huo, S. A. Somayajula, Y. Liang, R. Zhang, F. Koushanfar, and P. Xie. Token-specific watermarking with enhanced detectability and semantic coherence for large language models. In Proceedings of the 41st International Conference on Machine Learning (ICML 2024), 2024.
[10] M. Jaggi. Revisiting Wolfe: Projection-free sparse convex optimization. In Proceedings of the 30th International Conference on Machine Learning, volume 28, pages 427-435, 2013.
[11] A. Karamolegkou, J. Li, L. Zhou, and A. Søgaard. Copyright violations and large language models. In Proceedings of the 2023 Conference Empirical Methods in Natural Language Processing, pages 7403-7412, 2023.
[12] Kirchenbauer, J. Geiping, Y. Wen, Katz, I. Miers, and T. Goldstein. A watermark for large language models. In International Conference on Machine Learning, ICML 2023, volume 202, pages 17061-17000 -1084, 2023.
[13] J. Kirchenbauer, J. Geiping, Y. Wen, M. Shu, K. Saifullah, K. Kong, K. Fernando, A. Saha, M. Goldblum, and T. Goldstein. On the reliability of watermarks for large language models. In Proceedings of the International Conference on Learning Representations (ICLR), 2024.
[14] T. Kojima, S. Gu, M. Reid, Y. Matsuo, and Y. Iwasawa. Large language models are zero- shot reasoners. In Advances in Neural Information Processing Systems, volume 35, pages 22199-22213 2022.
[15] K. Krishna, Y. Song, M. Karpinska, J. Wieting, and M. lyyer. Paraphrasing evades detectors of ai-generated text, but retrieval an effective defense. In Proceedings of the 37th International Conference Neural Information Processing Systems, 2023.

---

[16] R. Kuditipudi, J. Thickstun, T. Hashimoto, and P. Liang. Robust distortion- free watermarks for language models. Transactions on Machine Learning Research. 2024.
[17] T. Lee, S. Hong. J. Ahn, I. Hong, H. Lee, S. Yun, J. Shin, and G. Kim. Who wrote this code? watermarking for code generation. In Proceedings of the 62nd Annual Meeting of the Association for Computational Linguisties (Volume 1: Long Papers), pages
2024.
[18] Y. Li, Y. Wang, Z. Shi, and C.-J. Hsich. Improving the generation quality of watermarked large language models via word importance scoring. arXiv preprint arXiv:231 1.09668, 2023.
[19] A. Liu, L. Pan, X. Hu, S. Meng, and L. Wen. A semantic invariant robust watermark for large language models. In The Twelfth International Conference on Learning Representations (ICLR 2024), 2024.
[20] A. Liu, L. Pan, Y. Lu, J. Li, X. Hu, X. Zhang, L. Wen, I. King, H. Xiong, and P. Yu. A survey of text watermarking in the era of large language models. 2024.
[21] Y. Liu and Y. Bu. Adaptive text watermark for large language models. In Proceedings of the 41st International Conference on Machine Learning, volume 235, pages 3071 18-3773777, 2024.
[22] Y. Lu, A. Liu, D. Yu, J. Li, and I, King. An entropy-based text watermarking detection method. In Proceedings of the 62nd Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), pages 11724-11735, 2024.
[23] D. Megías, M. Kuribayashi, A. Rosales, and W. Mazurczyk. Dissimilar: Towards fake news detection using information hiding, signal processing and machine learning. In Proceedings of the 16th International Conference on Availability, Reliability and Security, 2021.
[24] M. Mozes, X. He, B. Kleinberg, and L. D. Griffin. Use of Ilms for illicit purposes: Threats, prevention measures, and vulnerabilities, 2023.
[25] T. Munyer, A. Tanvir, A. Das, and X. Zhong. Deeptextmark: A deep learning- driven text watermarking approach for identifying large language model generated text, 2024.
[26] OpenAl. Gpt-4 technical report, 2024.
[27] A. Plaat, A. Wong, S. Verberne, J. Broekens, N. van Stein, and T. Bäck. Reasoning with large language models, survey. CoRR, abs/2407.11511 2024.
[28] L. Y. Por, K. Wong, and K. O. Chee. Unispach: A text-based data hiding method using unicode space characters. Journal of Systems and Software, 177 2012.
[29] C. Raffel, N. Shazeer, A. Roberts, K. Lee, S. Narang, M. Matena, Y. Zhou, W. Li, and P. J. Liu. Exploring the limits of transfer learning with unified text-to-text transformer. Journal of Machine Learning Research, 21 21(140):1667, 2020.
[30] N. Reimers and I. Gurevych. Sentence-BERY Sentence embeddings using Siamese BERT- networks. In Proceedings of the 2019 Conference on Empirical Methods in Natural Lan- guage Processing and the 9th International Joint Conference on Natural Language Processing (EMNLP-IJCNLP), pages 9982-3992, 2019.
[31] M. C. Rillig, M. Agerstrand, M. Bi, K. A. Gould, and U. Sauerland. Risks and benefits of large language models for the environment. Environmental Science & Technology. 5709):36643466, 2023.
[32] S. G. Rizzo, F. Bertini, and D. Montesi. pontent-preservin text watermarking through unicode homoglyph substitution. In Proceedings of the 20th International Database Engineering & Applications Symposium, pages 97-104, 2016
[33] Rosenblatt. The perceptron: A probabilistic model for information storage and organization (1958). In Ideas That Created the Future: Classic Papers of Computer Science. The MIT Press, 2021.
[34] R. Sato, Y. Takezawa, H. Bao, K. Niwa, and M. Yamada. Embarrassingly simple text watermarks. arXiv preprint arXi::2310.08920 2023,
12

---

[35] O. Sener and V. Koltun. Multi-task learning as multi- optimization. In Proceedings of the 32nd International Conference on Neural Information Processing Systems, page 525-536, 2018.
[36] C. Stokel-Walker, Ai bot chatgpt writes smart essays should professors worry? Nature, 2022.
[37] Y. Takezawa, R. Sato, H. Bao, K. Niwa, and M. Yamada. Necessary and sufficient watermark for large language models. Transactions Machine Learning Research (TMLR), 2025.
[38] M. Topkara, U. Topkara, and M. J. Atallah. Words are not enough: Sentence level natural language watermarking. In Proceedings of the 4th ACM International Workshop on Contents Protection and Security (MCPS '06), pages 37-46, 2006.
[39] U. Topkara, M. Topkara, and M. J. Atallah. The hiding virtues of ambiguity: Quantifiably resilient watermarking of natural language text through synonym substitutions. In Proceedings of the 8th Workshop on Multimedia and Security, pages 174-174, 2006.
[40] B. Wang and A. Komatsuzaki. GPT-J-6B: A Billion Parameter Autoregressive Language Model, 2021.
[41] Wei, X. Wang, D. Schuurmans, M. Bosma, B. Ichter, F. Xia, E. Chi, Q. Le, and D. Zhou. Chain-of-thought prompting elicits reasoning in large language models, 2023.
[42] X. Yang, K. Chen, W. Zhang, C. Liu, Y. Qi, J. Zhang, H. Fang, and N. Yu. Watermarking text generated by black-box language models. arXiv preprint arXiv:2305.0888. 2023.
[43] X. Yang, K. Chen, W. Zhang, C. Liu, Qi, Zhang, H. Fang, and N. Yu. Watermarking text generated by black-box language models, 2023.
[44] K. Yoo, W. Ahn, J. Jang, and N. Kwak. Robust multi-bit natural language watermarking through invariant features. In Proceedings of the 6lst Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), 2023.
[45] S. Zhang, S. Roller, N. Goyal, M. Artetxe, M. Chen, S. Chen, C. Dewan, M. Diab, X. Li, X. V. Lin, T. Mihaylov, M. Oit, S. Shleifer, K. Shuster, D. Simig, P. S. Koura, A. Sridhar, T. Wang, and I.. Zettlemoyer. Opt: Open pre-trained transformer language models, 2022.
[46] X. Zhao, Ananth, Li, and Y.-X. Wang. Provable robust watermarking for ai ai-gennrated text. In The Twelfth International Conference on Learning Representations (ICLR 2024), 2024.
13

---

NeurIPS Paper Checklist
1. Claims
Do the main claims made in the abstract and introduction accurately reflect the paper's contributions and scope?
Question:
Answer: [Yes]
Justification:
As shown in Section 1, our claims and contributions are clearly described.
Guidelines:
The answer NA means that the abstract and introduction do not include the claims made in the paper.
The abstract and/or introduction should clearly state the claims made, including the contributions made in the paper and important assumptions and limitations. A No or
NA answer to this question will not perceived well by the reviewers.
The claims made should match theoretical and experimental results, and reflect how much the results
be expected to generalize other settings.
is fine to include aspirational goals as motivation as long is clear that these goals are not attained by the paper.
2. Limitations
Question: Does the paper discuss the limitations of the work performed by the authors?
Answer: [Yes]
Justification:
The limitations are shown in Appendix H.
Guidelines:
The answer NA means that the paper has no limitation while the answer No means that the paper has limitations, but those not discussed in the paper.
The authors are encouraged to create separate "Limitations" section in their paper.
The paper should point out any strong assumptions and how robust the results are to violations of these assumptions (e.g., independence assumptions, noiseless settings, model well-specification. asymptotic approximations only holding locally). The authors should reflect how these assumptions might be violated in practice and what the implications would be.
The authors should reflect on the scope of the claims made, e.g., if the approach was only tested on few datasets or with few runs. In general, empirical results often depend on implicit assumptions, which should be articulated.
The authors should reflect the factors that influence the performance of the approach. For example, facial recognition algorithm may perform poorly when image resolution low images are taken in low lighting. Or speech-to-text system might be used reliably to provide closed captions for online lectures because it fails to handle technical jargon.
The authors should discuss the computational efficiency of the proposed algorithms and how they scale with dataset size.
If applicable, the authors should discuss possible limitations of their approach address problems of privacy and faimess.
While the authors might fear that complete honesty about limitations might be used by reviewers as grounds for rejection, a worse outcome might be that reviewers discover limitations that aren't 't acknowledged in the paper. The authors should use their best judgment and recognize that individual actions in favor of transparency play an impor- tant role in developing norms that preserve the integrity of the community. Reviewers will be specifically instructed to not penalize honesty concerning limitations.
3. Theory assumptions and proofs
Question:
For each theoretical result, does the paper provide the full set of assumptions and complete (and correct) proof?
Answer: [Yes]
14

---

Justification:
They are provided in Section 3.1, Section 3.3 and Appendix A.
Guidelines:
The answer NA means that the paper does not include theoretical results.
All the theorems, formulas, and proofs in the paper should be numbered and cross- referenced.
All assumptions should be clearly stated referenced in the statement of any theorems.
The proof' can either appear the main paper or the supplemental material, but they appear in the supplemental material, the authors are encouraged to provide short proof sketch to provide intuition.
Inversely, any informal proof provided in the core of the paper should be complemented by formal proofs provided in appendix or supplemental material.
Theorems and Lemmas that the proof relies upon should properly referenced.
4. Experimental result reproducibility
Question: Does the paper fully disclose all the information needed to reproduce the main ex- perimental results of the paper to the extent that affects the main claims and/or conclusions of the paper (regardless of whether the code and data are provided or not)?
Answer: [Yes]
Justification:
We provide the experiment details in Section 4.1, and additional details are provided in Appendix F.1 and Appendix F.2 and dataset details in Appendix to reproduce the main experimental results.
Guidelines:
The answer NA means that the paper does not include experiments.
If the paper includes experiments, a No answer to this question will not be perceived well by the reviewers: Making the paper reproducible is important, regardless of whether the code and data are provided or not.
If the contribution is dataset and/or model, the authors should describe the steps taken make their results reproducible or verifiable.
Depending on the contribution, reproducibility can be accomplished in various ways. For example, if the contribution is novel architecture, describing the architecture fully might suffice, or if the contribution a specific model and empirical evaluation, it may be necessary either make it possible for others to replicate the model with the same dataset, or provide access to the model. In general. releasing code and data is often one good way accomplish this, but reproducibility can also be provided via detailed instructions for how to replicate the results, access to a hosted model (e.g., in the case of large language model), releasing of a model checkpoint, or other means that appropriate to the research performed.
While NeurlPS does not require releasing code, the conference does require all submis- sions to provide some reasonable avenue for reproducibility, which may depend on the nature of the contribution. For example
(a) the contribution is primarily a new algorithm, the paper should make clear how reproduce that algorithm. (b) If the contribution is primarily new model architecture, the paper should describe the architecture clearly and fully.
(c) the contribution is a new model (e.g., a large language model), then there should either be way to access this model for reproducing the results or way to reproduce the model (e.g., with an open-source dataset instructions for how to construct the dataset).
(d) We recognize that reproducibility may be tricky in some cases, in which case authors are welcome deseribe the particular way they provide for reproducibility. In the case of closed-source models, it may be that access to the model is limited in some way (e.g., to registered users), but it should possible for other researchers to have some path to reproducing or verifying the results.
5. Open access to data and code
15

---

Does the paper provide open access to the data and code, with sufficient instruc- tions faithfully reproduce the main experimental results, as described in supplemental material?
Question:
Answer: [Yes]
Justification:
The code and data available in supplemental material.
Guidelines:
The answer NA means that paper does not include experiments requiring code.
Please see the NeurIPS code and data submission guidelines (https://nips.co cc/ Beapi///guidee//oeeSubbbba
www.iiiissssiiiiiiiooooiiisiiiii for more details.
While we encourage the release of code and data, we understand that this might not be possible, "No" is an acceptable answer. Papers cannot be rejected simply for not including code, unless this is central to the contribution (e.g., for a new open-source benchmark).
The instructions should contain the exact command and environment needed to run to reproduce the results. See the NeurlPS code and data submission guidelines (https: / /nips. wttpccccpppccccceececc..ccccccct httpppululuudlssappp/ppppppaaaae /Ct////oooeeeeeeeeeeeeeeeee /ttea////iiieee/eeeutiiiiiie
onPolicy) for more details.
The authors should provide instructions on data access and preparation, including how access the raw data, preprocessed data, intermediate data, and generated data,
The authors should provide scripts to reproduce all experimental results for the new proposed method and baselines. If only subset of experiments reproducible, they should state which ones are omitted from the script and why.
At submission time, to preserve anonymity, the authors should release anonymized versions (if applicable).
Providing as much information as possible in supplemental material (appended to the paper) recommended, but including URLs to data and code permitted.
6. Experimental setting/details
Does the paper specify all the training and test details (e.g., data splits, hyper- parameters, how they were chosen, type of optimizer, etc.) necessary to understand the results?
Question:
Answer: [Yes]
Justification: We provide the dataset details in Appendix B, and implementation details in F.1, byperparameter details in Section 4.1 and Appendix F.2.
Guidelines:
The answer NA means that the paper does not include experiments.
The experimental setting should be presented in the core of the paper to a level of detail that is necessary to appreciate the results and make sense of them.
The full details can be provided either with the code, in appendix, as supplemental material.
7. Experiment statistical significance
Does the paper report error bars suitably and correctly defined or other appropriate information about the statistical significance of the experiments?
Question:
Answer: [No]
Justification: We used a fixed random key for our main experiments. When our watermark class initializes, it sets fixed random seed for reproducible. Through out our training and experiments we fixed the same random key as that of used in previous work [9].
Guidelines:
The answer NA means that the paper does not include experiments.
The authors should answer "Yes" if the results are accompanied by emror bars, confi- dence intervals, or statistical significance tests, least for the experiments that support the main claims of the paper.
16

---

The factors of variability that the error bars are capturing should be clearly stated (for example, train/test split, initialization, random drawing of some parameter, or overall run with given experimental conditions).
The method for calculating the error bars should be explained (closed form formula, call to a library function, bootstrap, etc.)
The assumptions made should be given (e.g.. Normally distributed errors).
It should be clear whether the error bar is the standard deviation the standard error of the mean.
It is OK to report 1-sigma error bars, but one should state it. The authors should preferably report a 2-sigma error bar than state that they have 96% CI, if the hypothesis of Normality of errors is not verified.
For asymmetric distributions, the authors should be careful not show in tables figures symmetric error bars that would yield results that are out range (e.g. negative error rates).
If error bars are reported in tables or plots, The authors should explain the text how they were calculated and reference the corresponding figures or tables in the text.
8. Experiments compute resources
For each experiment, does the paper provide sufficient information on the com- puter resources (type of compute workers, memory, time of execution) needed reproduce the experiments?
Question:
Answer: [Yes]
Justification:
As is provided in Appendix F.3.
Guidelines:
The answer NA means that the paper does not include experiments.
The paper should indicate the type of compute workers CPU or GPU, internal cluster, or cloud provider, including relevant memory and storage.
The paper should provide the amount compute required for each of the individual experimental runs as well estimate the total compute.
The paper should disclose whether the full research project required more compute than the experiments reported in the paper (e.g., preliminary or failed experiments that didn't make into the paper).
9. Code ethics
Question: Does the research conducted the paper conform, in every respect, with the NeurlPS Code of Ethics httpps//neurs: newrips.....///
ines?
Answer: [Yes]
Justification:
The research is conducted in the paper conform, with the NeurIPS Code of
Ethics.
Guidelines:
The answer NA means that the authors have reviewed the NeurIPS Code of Ethics.
the authors answer No, they should explain the special circumstances that require deviation from the Code of Ethics.
The authors should make sure to preserve anonymity (c.g.. if there special consid- eration due to laws or regulations in their jurisdiction..
10. Broader impacts
Question: Does the paper discuss both potential positive societal impacts and negative societal impacts of the work performed?
Answer: [Yes]
Justification:
The socictal impacts of watermarks, ours included, are as shown in Section 1.
Guidelines:
The answer NA means that there is no societal impact of the work performed.
17

---

If the authors answer NA or No, they should explain why their work has no societal impact or why the paper does not address societal impact.
Examples of negative societal impacts include potential malicious or unintended uses (e.g., disinformation, generating fake profiles, surveillance), fairness considerations (e.g., deployment of technologies that could make decisions that unfairly impact specific groups), privacy considerations, and security considerations.
The conference expects that many papers will be foundational research and not tied to particular applications, let alone deployments. However, if there is direct path to any negative applications, the authors should point it out. For example, it is legitimate to point out that improvement in the quality of generative models could be used to generate deepfakes for disinformation. On the other hand, it is not needed to point out that generic algorithm for optimizing neural networks could enable people to train models that generate Deepfakes faster.
The authors should consider possible harms that could arise when the technology is being used as intended and functioning correctly, harms that could arise when the technology is being used as intended but gives incorrect results, and harms following from (intentional or unintentional) misuse of the technology.
If there are negative societal impacts, the authors could also discuss possible mitigation strategies (e.g., gated release of models, providing defenses in addition attacks, mechanisms for monitoring misuse, mechanisms to monitor how system learns from feedback over time, improving the efficiency and accessibility of ML).
11. Safeguards
Does the paper describe safeguards that have been put in place for responsible release of data models that have high risk for misuse (e.g., pretrained language models, image generators, or scraped datasets)?
Question:
Answer: [NA]
Justification:
We do not have such risks.
Guidelines:
The answer NA means that the paper poses no such risks.
Released models that have high risk for misuse or dual-use should be released with necessary safeguards to allow for controlled use of the model, for example by requiring that users adhere to usage guidelines or restrictions to access the model or implementing safety filters.
Datasets that have been scraped from the Internet could pose safety risks. The authors should describe how they avoided releasing unsafe images.
We recognize that providing effective safeguards challenging, and many papers do not require this, but we encourage authors to take this into account and make best faith effort.
12. Licenses for existing assets
Are the creators or original owners of assets (e.g.. code, data, models), used in the paper, properly credited and the license and terms of use explicitly mentioned and properly respected?
Question:
Answer: [Yes]
Justification:
As is shown in Section and Appendix B.
Guidelines:
The answer NA means that the paper does not use existing assets.
The authors should cite the original paper that produced the code package dataset.
The authors should state which version of the asset used and, possible, include URL
The name of the license (e.g., CC-BY 4.0) should be included for each asset.
For scraped data from a particular source (e.g., website), the copyright and terms of service of that source should be provided.
18

---

If assets are released, the license, copyright information, and terms of use in the package should be provided. For popular datasets, paperswi thcode. cowccoo//aaoo/o datasatssss has curated licenses for some datasets. Their licensing guide can help determine the license of dataset.
For existing datasets that are re-packaged, both the original license and the license of the derived asset (if has changed) should be provided.
this information is not available online, the authors are encouraged to reach out to the asset's s creators.
13. New assets
Are new assets introduced in the paper well documented and is the documentation provided alongside the assets?
Question:
Answer:
[Yes]
Justification: We proposed novel MLP trained for selectively apply watermarks, as described 3.2.
Guidelines:
The answer NA means that the paper does not release new assets.
Researchers should communicate the details of the dataset/code/mmode as part of their submissions via structured templates. This includes details about training, license, limitations, etc.
The paper should discuss whether and how consent was obtained from people whose asset is used.
At submission time, remember anonymize your assets (if applicable). You can either create an anonymized URL or include an anonymized zip file.
14. Crowdsourcing and research with human subjects
Question: For crowdsourcing experiments and research with human subjects, does the paper include the full text of instructions given participants and screenshots, if applicable, as well as details about compensation (if any)?
Answer: [NA]
Justification:
This work doesn't 't involve crowdsourcing and research with human subjects.
Guidelines:
The answer NA means that the paper does not involve crowdsourcing research with human subjects.
Including this information in the supplemental material is fine, but if the main contribu- tion of the paper involves human subjects, then as much detail as possible should be included in the main paper.
According the NeurIPS Code of Ethics, workers involved in data collection, curation, or other labor should be paid at least the minimum wage in the country of the data collector.
15. Institutional review board (TRB) approvals or equivalent for research with human subjects
Question: Does the paper describe potential risks incurred by study participants, whether such risks were disclosed to the subjects, and whether Institutional Review Board (IRB) approvals (or equivalent approval/teview based the requirements of your country or institution) were obtained?
Answer: [NA]
Justification: Crowdsourcing and research with human subjects are not involved in this work.
Guidelines:
The answer NA means that the paper does not involve crowdsoureing nor research with human subjects.
19

---

Depending on the country in which research is conducted, IRB approval (or equivalent) may be required for any human subjects research. If you obtained IRB approval, you should clearly state this in the paper.
We recognize that the procedures for this may vary significantly between institutions and locations, and we expect authors to adhere to the NeurIPS Code of Ethics and the guidelines for their institution.
For initial submissions, do not include any information that would break anonymity (if' applicable), such as the institution conducting the review.
16. Declaration of LLM usage
Does the paper describe the usage of LLMs if is an important, original, or non-standard component of the core methods in this research? Note that if the LLM is used only for writing, editing, or formatting purposes and does not impact the core methodology, scientific rigorousness, originality of the research, declaration is not required.
Question:
[NA]
Justification:
We did not use L.M to develop core method in this work.
Guidelines:
The answer NA means that the core method development in this research does not involve LLMs as any important, original, or non-standard components.
Please refer to our LLM policy (https: httpa//////eeeeeeeeeeeeeeeea ct/ ccC.coneeeeeecee/2225555555 for what should or should not be described.
20

---

A Theoretical Justification of LTW as a Selective Watermark
If watermarking scheme is designed to be selective, it must not rely on random mechanisms or rules that are unverifiable during the detection phase. Otherwise, without complementary detection strategy capable of identifying which tokens generated sequence were chosen to be watermarked during applying watermark, the watermark detection process becomes unreliable.
Specifically, consider the hypothesis testing formulation:
S|cc 7T
Soriy (T>(1->)
where (0, 1) is the green-list ratio, T is the total number of tokens to be detected, and |S a denotes the number of tokens detected which are the green-list G. Suppose that we define the original test statistic as Sorig corresponding to total of I tokens, among which |S|G are green tokens.
If, during detection, token that not selected for watermarking during generation is nonetheless included in the detection process, then the total token count increases to T + 1, while the expectation of green tokens becomes |S|c G +
The updated test statistic becomes:
(|S]c + + 1T
|S|o -I (T + (1 7) V(T + 11( 77
Znew
It is evident that Knew < Zorig suggesting that including tokens that were not selected for watermarking into the detection process will, in expectation, degrade the detectability of the watermark.
Therefore, in selective watermarking schemes, it is essential that the selection rule used during watermark embedding be reproducible during detection. Only by reliably identifying the subset of tokens that were eligible for watermarking at generation time can we perform valid and powerful hypothesis testing on that subset alone, ensuring accurate detection outcomes.
Let the generated sequence up to step n be denoted as s = [t f2..t.....fff. .t]] Let E as the Sentence as the vocabulary containing i tokens and Ma our
T2,,..... ................................ .Kk Embedding model, V {21. trained network. Specifically, during the watermark embedding phase, we leverage three key signals:
1. The semantic embedding, obtained by passing the preceding tokens to the sentence embedding model:We denote the obtained sentence embedding E Esem (sm()nn++1:n)
2. The Shannon entropy of the probability distribution at the current step as P(r,) log p(z,);
3. The current proportion of tokens selected for watermarking in the generated sequence as r | - Selected(t,)
The probability of selecting the current token for watermarking is given by:
Wwm MM ([E, e, r]) E [0,
During the detection phase, determine whether each token in the generated sequence was likely selected for watermarking by re-evaluating the same signals used during generation. The sentence embedding E. can be recomputed based the i tokens before the current token, the Shannon entropy e can be reconstructed using the model's token probability distribution at that position, and the watermark ratio can be tracked incrementally based watermark decisions of the tokens before it. Since all three inputs to Mg can be obtained with prior sequence, the selection logic is fully reproducible during detection.
21

---

B Dataset details
We followed previous works [8, 9, 12, 13, 21] utilizing the RealNewsLike Like subset of the C4 dataset [29] our dataset. Because the method we proposed requires training, need devide training set as well as test set for the experiments. We selected the first 10,000 pieces of text in the first RealNewsLike subset as training set. From another RealNewsLike subset, randomly shuffld using a random seed provided in our code. Subsequently, 500 samples of text with sufficient length were then filtered and selected ensure that each long enough be divided into a prompt segment and reference text. They are later used as the test set in our experiments.
C Selective Watermarking Algorithm and Detection Algorithm of LTW
The following Algorithms show the detailed steps of how our selective watermark is applied and detected with the already trained Selector Network.
Algorithm Watermark Injection
1: Input: LLM f. prompt tokens window size k;, green-list ratio y, watermark bias 8, max length
L. Sentence Embedding model &, our trained MLP Mg. thresholds Tlww Tmid Thigh
2: Output:
3: S+ æ:
(sequence &generated sequence watermark record)
4: for = 1 to L do
5:
4 + f(S)
(logits)
6: Pt softmax(4+)
7:
et t En Pe. log Pr[v]
(entropy)
t S0,
-
8:
me),(W), E. E(Si-k k :))
otherwise
9:
(last token for getting sentence embeddings) (selector output E [0, 1))
10:
m. Mo (1E,, rr,
Thow
Tt < Low Ratio Threshold,
11:
Te
Thigh:
ri > High Ratio Threshold,
Tmidd
otherwise
12:
if m, > then
m. +-
14:
G. 4 GreenMask(44, 7)
15:
for each G2 do
16:
4e[u] + 44[0] +
end for
18:
else
m4 <- 0
20:
[a[v] + [.[e]
end if
22:
Append m, to W
23:
Vt ~ Sample(softmax((.)
24: Append to S
if Ui = (FOS) then break
26:
end for
27: Return
22

---

Algorithm 2 Selective Watermark Detection
Input:LLM f. prompt I, test sequence(prompt included) y (J0....,9NN-1. window size
green-list ratio 7,, Sentence Embedding model &, trained MLP Maa thresholds Now Tmid Thigh
2: Output: z-score 2
3: W + D: Recored f- 0;
Agreen t 0
4: for æ to N 1 do
5: 4 f1/(0:1 :00)) 1))
6: Pt softmax(()
7:
G t E. log
(entropy)
0,
W 0,
8:
(W),(W), otherwise
9:
F. + [l/[max(0, k) t))
(sentence embedding)
10:
m. + Ma m.)
Now
< LowRatio,
11:
TE
Thigh:
rz HighRatio,
Tmid=
otherwise
if m. > then
13:
my 1: Reconed + Rescorod
Ge BreenMask(yyo111
15:
if y(] G, then
16:
Regreen + Ngreen + -1
end if
18: else
Me 0
20: end if
Append m, to W
22:
end for
+ ZScore (ggreen: Recored)
24:
Return
D Further Discussions on Experiment Results
D.1
ROC Curves
Figure 8 and 9 illustrate the ROC curves and the AUC values of our watermark methods and baseline methods under paraphrase attack. The result demonstrate the robustness of our methods compared to baseline methods.
ROC Curves on OPT-6.7B
ROC Curves on GPT]-6B
1.0
1.0
True Positive Rate
0.8
True Positive Rate
0.8
0.6
0.6
0.4
KGW (MUC 0.970
Laigram
0.4
Utiggam 2.27)
KüW (MUC - 0-335)
LAUC
0.2
Mill. 3.11:
(Ta (ilC 5.16;
U.00 rouk 2386)
U'00:::: (AU 2992)
Randim Guesn
Random Gunm
0.6
False Positive Rate
False Positive Rate
Figure Comparisons of ROC curves and AUC values of different watermark methods generated using OPT-6.7B when under dipper attack.
Figure 9: Comparisons of ROC curves and AUC values of different watermark methods generated using GPT-J-6B when under dipper attack.
23

---

D.2 Watermarked Text Example
We present example of text watermarked using our method and baseline methods. Figure 10 shows the prompt used for generation using OPT-6.7B as well as the watermarked generations using our method, KGW and SWEET. The unwatermarked text is highlighted in gray, those highlighted in red or green are watermarked tokens, red represents the sampled token is in the red list, and green represents the sampled token is in the green list.
Prompt
KGW (z-score: 14.22, ppl: 20.85)
Congrats to everyone featured in the videos and also all of us, for being recognized as an Arts & Culture hub of the world! Of course we' 'ee all known this for a while now, but i's great that Google is catching on. you're re opening up this newsletter your inbox for the first time you' reading through stories on the web, you' 'e stumbled across a new project called Maker City KCC. This is our weekly newsletter. Thanks for finding us. Maker City many makers and entrepreneurs, creating audience for the city's 'ss maker spaces and the spirit of innovation in our city. We depict and discuss the many dimensions of the 4th Industrial Revolution, and the opportunities it creates for Kansas City. And, as part of increasing civic engagement, want
If you don 1 have a newsletter make for one What 6 going on in the world The International Date line is a place where the date that people on the surface see is different than the date that goes past the sun 1 per ist alt ic motion inside the earth at some midnight past the International Date Line and back in to Kansas City Missouri again Well we had this crazy notion that the International Date lines goes past Kansas City our date for World Maker aire would be the past too a World Maker F aire Kansas City will be happening this Friday and our W KC model is completely rooted and organized by YOU so please help spread the word about it Create a shirt that only Kansas City M akers If you don already have shirt make sure to get
sign up
a place highlights the
SWEET (e-score: 14.38, ppl: 17.64)
LTW (ours) (x-score: 14.67, ppl: 15.22)
Know anyone that should added our list of makers| are maker and would like to get some coverage in the weekly newsletter let us know HERE What You Should Know D rain our reservoir D rain Kansas City has launched response to hundreds of thousands of illegal dump ers. who are el ogging our reservoirs with the waste of others As city should be fighting this instead of toler ating it Visit drain our res erv oir com to read up on the history of the problem the City has Nisit drain our res erv our ive in com for survey you casily take and share with your friends Visit drain our res our now com for map of what all the problems really are Let the government know If you live in the City of Kansas City proper J please take the time to yoice your concerns about what our government says
This week episode of The M akers Podcast features an episode of The M akers Podcast produced by Dave De ans and Rebecca Miller at the Missouri History Museum This week s guest is Brent Schmidt CEO at City Fund KC Our conversation with Brent spans City Fund 's mission
fund through City Fund KC entrepreneurship and economic development in Kansas City Dave and Rebecca also talk about being in the Museum Brent 's background as a video producer and much more This podcast is worth listen If you' re the market for some fresh affordable fresh handmade affordable gifts make sure you re checking out Shop Now an up start maker that sells 100 % handmade fresh affordable gifts that you actually enjoy This week s episode The Maker Town Podcast features
ability to drive
Figure 10: A real example of watermarked text generated on our C4 RealNewslikee test set. The figure shows comparison between of method and baseline method. In this example, the generated watermarked text using our method has lower perplexity and a higher z-score than baseline methods.
D.3 Correlation Between Entropy and Network Output
We conducted additional analyses by plotting a scatter diagram investigate the relationship between the entropy of the input and the output of the trained model during selective watermarking. The results indicate clear overall positive correlation between input and output entropy. Notably, when the entropy is either very low or very high, the results exhibit clustering within a narrow range. In these cases, other input factors, such as semantic embeddings, either amplify the influence of entropy or play a comparatively minor role in determining the outcome.
24

---

Specifically, when the entropy is below I or above 4, the output consistently falls into well-defined categories: low entropy values are classified as indicating that the current token should not be watermarked, while high entropy values are classified requiring the application of the watermark. However, for entropy values within the range of 1.5 to 3.5, a broader spectrum of outputs is observed. Under our dynamic thresholding approach, the outputs under the same entropy includes the possible decision to selectively watermark or not applying the watermark to the token being generated.
0.8
0.6
Out (0-1)
0.4
0.2
0.0
0.5
1.0
1.5
2.0
2.5
3.0
3.5
4.0
Entropy
Figure 11: Scatter diagram between the entropy input and the output of our Selector Network.
E
Further discussion on Selective watermark designs
To compute the final selection weight mm [0. 1]. design compact yet expressive neural network based on a multi-layer perceptron (MLP) [33]. The input to the model consists of a sentence embedding of dimension 768, along with input features: entropy and ratio. In order to handle the high-dimensional nature of the embedding and promote efficient learning, the embedding first passed through several fully connected layers to reduce its dimensionality. This reduced representation then concatenated with the two features to form a concatenated input vector, which subsequently fed into another MLP responsible for computing the final prediction. The output layer applies sigmoid activation function to constrain the result to the range (0, 1]. The model functions as selector module in our system, leveraging both semantic information from the sentence embedding and statistical cues from the two watermarking related features to make a learnable, data-driven selection decision.
F
Experimental Details
F.1
Training Details
In our proposed Selector Network, implemented as multi-layer perceptron (MIP), we adopt two-stage processing pipeline for the input. Specifically, each input sentence embedding E E 768 is
25

---

first passed through a two-layer MLP for dimensionality reduction. The resulting livee-dimensional representation is then concatenated with the other two additional features: the entropy e E RR of the predicted token distribution and the watermarked ratio r E [0. 1.. Letting E,, denote the reduced embedding, the concatenated vector eF used as input to a second two-layer MI.,, which produces a single output in the output layer. All hidden layers in both MLP modules use the Leaky ReLU activation function. To ensure the output lies in the interval [o, 1I, the output layer uses a sigmoid activation function. We train the Selector Network for one epoch on our training set. We used O.T-1.3B during training, generating a maximum of 75 tokens for each prompt provided during training. During training, the parameters in the LLM well as the sentence embedding model are frozen, the parameters in our trained MLP are the only parameters being updated. The training procedure uses a batch size of 5, the Adam optimizer with a learning rate of 1 10 4, and input sentence embeddings computed from the previous & = 6 tokens in the previous sequence. The watermark hyperparameters experiments. We save model checkpoints every 200 steps and retain the final model weights after one epoch for use in experiments on the effectiveness of our method.
set to 8 3.0 and 7 = 0.25, the same as those in the main
F.2 Experiment Details
In our main experiments, we set 0.25 and 5 3.0 for methods that can adjust these hyperparam- eters, and we set the entropy threshold as 1.2 for SWEET. We used the default model provided in their code for TS-tattemmrk, and for EXP-edit, the default hyperparameters provided in their code. For all watermarks, we used Multinomial Sampling with a top-k of 100 a top-p of 0.95 and no repeat n-gram size of 8 to generate 200 tokens for each prompt.
For fair and consistent comparison, we uniformly set the tunable hyperparameters = 3.0 and y 0.25 across all watermarking schemes that allow manual configuration of these values in our main experiment. This decision was motivated by the hyperparameter configuration adopted in the baseline method SWEET, which introduces an additional hyperparameter, entropy, to determine whether to add watermark at each step. The authors conducted a comprehensive hyperparameter search and ultimately selected entropy 1.2 with 5 3.0 and y 0.25 as their main setting which was the only set of hyperparameter that achieved both high detectability and high text quality under their predefined criteria on one of the evaluated datasets. The setting ō 3.0 and 7 0.25 was adopted as the primary configuration throughout their main and subsequent experiments in their work.
Consequently, we fix entropy to 1.2 for the SWEET baseline and apply = 3.0 and y = 0.25 to all applicable watermarking methods in our evaluation. In the context of our main experiment, consider a watermarking approach to be inferior to ours under the current evaluation setup if: (1) it does not outperform ours in either detectability methods perform comparably, and (2) it yields lower text quality.
robustness (considering some metrics where all
One of the fundamental advantages of large language model (LLM) watermarking lies in its high detectability under sufficient watermark strength. Our proposed watermarking method exhibits excellent detectability: on 500-sample test set all watermarked samples were successfully identified, with no false positives on unmarked texts. We also did extra experiments to compare our method with KGW under various watermark strengths varying from 6 = 1.0 to 8 = 3.5 to evaluate the performance of our watermark with the baseline method in terms of detectability and text quality, the results show that our methods have better pareto frontier than theirs.
The improved robustness of LTW-0 can be attributed to its fixed green-red token partitioning strategy. which mitigates failure cases in using hash-function that is based on previous generated text, where rewriting of previous tokens affects green/red list assignment and causes detection to fail. The implementation of LTW-0 in our work is simple, the difference between our implementation of LTW-1 and is changed function for obtaining the green-list, changing the hashing method to using a fix has.-key. We adopt the same hash-key that was used in previous works. [12, 17].
F.3 Computational Resources and Runtimes
We performed our experiments on Nvidia 1220 GPUs. We trained the our network using 3 L,0 GPUs, among which the second and third GPU are mainly used to speed up the generation of the L.L.M takes about 60 hours for training epoch. In our other experiments, evaluating the performance of
26

---

our watermark method and baseline methods, only one L20 is used, without using parallelism. About 2 hours is needed for finishing the watermarking generation task for our method, baseline methods expect EXP-edit, which is slow requires about the same time for finishing generation watermarked text using the test set.
G Discussion on Multiple-Gradient Descent Algorithm
To train the network proposed, formulated two loss functions Lo and CD to describe task. In our work, the training objective is to minimize those two loss functions, Lo (G) and LD(G), However these two losses often exhibit competitive relationship, meaning that reducing one may lead to an increase in the other. The optimization problem is formulated as:
min LQ(G) and min LD(G)
This is typical multi-objectivv optimization problem, where the ideal solutions are characterized by Pareto optimality.
Given feasible solutions G and G, we say that G dominates G if:
To(G)) s LQ(G), Lp(G) < LD(G),
and at least one of these inequalities is strict. A solution G is called Pareto optimal if there does not exist another solution that dominates G
The Multiple Gradient Descent Algorithm (MGDA) [5, 35] is designed to find descent direction that leads the optimization towards the Pareto front. Specifically, first compute the gradients of Lo and Lo with respect to G, denoted as 9Q and Jp. respectively. MGDA seeks a direction g that is convex combination of sQ and gD:
gD + (1 X² X*990.
where E 10 chosen to minimize the norm of g:
x argmin + (1 A)goll2.
Previous works [35] suggests, approach is applicable in the case of two losses, where the Frank- Wolfe algorithm [10] can be effectively utilized since the line search step admits a closed-form solution.
Algorithm 3 Closed-form solution for X"
Require: Gradients 9Q- 9n
if 9.90 2 ID9D then
2:
A >==1 1
3: else if 9D9 2 9690 then
4: =
5: else
(gQ )n)
9Q
6:
X
1|gD
7:
end if
By updating the model parameters along the direction g, the optimization process is guided towards solutions that Pareto optimal with respect both LQ and LD-
H Discussion about Limitations and Future Work
Our method presents two main limitations, primarily due to constraints in computational resources and time.
First, due to limitations in computational resources and time, our experiments are conducted on relatively small-scale models and only one dataset. Specifically, we utilize the RealNewsLike
27

---

subset of the C4 dataset which has been widely adopted in previous works [8, 9, 12, 13, 21] on large language model watermarking.
Second, the selector network in our work is trained on generations produced by the OPT-1.2 model using C4 RealNewsLike subset input for evaluating our method with baseline the text continuation task. In future work, we may extend this framework to broader scenarios which may require adapting the selector different generation tasks, which may need train our network different tasks using LLMs which may experts in those tasks.
28


Reviews:
[{"review_id": "ptKgdflUKA", "reviewer": "Reviewer_Ezdj", "summary": "This paper introduces \"Learning to Watermark\" (LTW), a novel selective watermarking framework for Large Language Models (LLMs) that leverages multi-objective optimization to balance watermark detectability and text quality. Traditional watermarking methods often face a trade-off between these two objectives, leading to degraded text quality or reduced detectability. LTW addresses this by employing a lightweight neural network, the Selector Network, which adaptively decides when to apply watermarks based on sentence embeddings, token entropy, and the current watermarking ratio. The network is trained using two specifically designed loss functions that guide it toward Pareto-optimal solutions, ensuring high detectability without compromising text quality.\n\nThe authors evaluate LTW by integrating it with two baseline watermarking methods, KGW and Unigram, and demonstrate significant improvements in text quality while maintaining or even enhancing detectability. Experimental results show that LTW achieves lower perplexity and higher cosine similarity with reference texts compared to existing methods, indicating better semantic coherence and text quality. Additionally, LTW exhibits superior robustness against paraphrase attacks, making it a promising approach for practical LLM watermarking applications. The paper contributes a new perspective on selective watermarking strategies, emphasizing the importance of adaptive decision-making in maintaining both security and usability.", "questions": "1. The current model primarily controls text quality and watermarking capability through loss functions. Does the weight of different losses have an impact on the model's performance? Additionally, does the number of tokens selected or the window size used in the model affect its effectiveness?\n\n2. How does the model ensure robustness against paraphrasing or deletion attacks? It seems that the method does not explicitly consider these types of attacks, which could be a potential vulnerability.", "limitations": "yes", "rating": 4, "confidence": 4, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": 3, "clarity": 3, "significance": 3, "originality": 3, "strengths_and_weaknesses": "## Strengths\n\n- The paper introduces a novel selective watermarking framework, LTW, which leverages multi-objective optimization to balance watermark detectability and text quality. This approach is innovative and addresses a significant challenge in the field of LLM watermarking.\n\n- The authors employ the Multiple Gradient Descent Algorithm (MGDA) to solve the multi-objective optimization problem, ensuring that the framework achieves Pareto-optimal solutions. This method effectively harmonizes the competing goals of high detectability and high text quality.\n\n- The paper provides extensive experimental evaluations across multiple models and datasets, demonstrating that LTW significantly enhances text quality without compromising detectability. The results are compelling and show that LTW outperforms existing watermarking methods in both text quality and robustness.\n\n## Weaknesses \n\n- The paper only evaluates the watermarking method against a single type of attack, specifically the Dipper Attack. It lacks experiments on other types of attacks, such as scrubbing and spoofing, which are important for a comprehensive understanding of the method's robustness.\n\n- The experiments are limited to one dataset, which may not be sufficient to demonstrate the generalizability of the proposed method across different datasets and domains.\n\n- In Table 3, as the threshold increases, the effect of the selector appears to diminish. The paper does not include ablation studies on other components of the model, making it difficult to assess the individual contributions of each module to the overall performance.", "was_revised": true, "final_justification": "4: Borderline accept", "rebuttal": "**Reply to Reviewer Ezdj**\n\nWe thank the reviewer for the insightful and valuable comments. We respond to your comment and problems as follows and sincerely hope that our rebuttal could properly address your concerns.\n\n> **Q1:** The experiments are limited to one dataset, which may not be sufficient to demonstrate the generalizability of the proposed method across different datasets and domains.\n\n**Reply:** **We did an additional experiment using the ELI5 dataset to demonstrate the generalizability of the proposed method across different datasets.**\n\n The ELI5 dataset is a long form Question Answering dataset, we randomly selected 500 piece of question-answer pairs from the dataset and did the extra experiment on it.\n\nWe employed the selective watermarking network previously trained on the C4 dataset, without training it on the new task and dataset. In evaluating detectability, we followed suggestion of reviewer GajM and adopted the use of p-values under a small false alarm rate, which provides a more rigorous measure of detectability instead of more commonly used metrics in watermarking papers such as FPR@1 and FPR@2 . As for text quality, we used the recall variant of ROUGE-L, a simpler and more computationally efficient metric.\n\n| Method       | LLaMA3.1-8B    |           | OPT-6.7B        |           |\n| ------------ | -------------- | :-------- | --------------- | --------- |\n|              | p-value < 1e-8 | Rouge-L   | p-value < 1e-10 | Rouge-L   |\n| KGW          | 0.984          | 0.230     | 0.994           | 0.244     |\n| Sweet        | 0.986          | 0.221     | **0.996**       | 0.252     |\n| TS-watermark | 0.934          | 0.225     | 0.992           | 0.251     |\n| LTW-1 (ours) | 0.982          | **0.251** | 0.992           | **0.256** |\n| LTW-0 (ours) | **0.988**      | 0.242     | 0.994           | 0.249     |\n\n Our experimental results in the table demonstrate that our method demonstrate high detectability as well as text quality. Specifically, while Sweet has a performance slightly better than ours in terms of detectability when generated using OPT-6.7b on this task, our methods shows better overall performance, **especially the text quality when generated using LLaMA3.1-8B under this task**. **Compared with TS-watermark ,which was trained to adaptively provide gamma and delta hyperparameters using the C4 dataset**, **our method shows better generalization ability on a different task**.\n\n---\n\n>  **Q2:** Does the weight of different losses have an impact on the model's performance? Additionally, does the number of tokens selected or the window size used in the model affect its effectiveness? \n\n**Question about Weight of different Losses:**\n\n**Reply:** The weights of these loss functions indeed play a important role in our method. Through extensive experiments, we have developed some empirical guidelines.\n\nFirst, for the loss term in Eq. 9, its weight should not be set too high. We typically set it in the range of 0.1–0.15, as its main function is to encourage the output to be more mask-like. This not only helps during training but also contributes to the robustness of the watermarking method—a point we will elaborate on later. Second, the weight corresponding to Eq. 7 should neither be too large nor too small. Improper weighting can lead to unreasonable watermarking ratios (either too high or too low), which directly affects both utility and detectability. The other loss terms are relatively more flexible and can be adjusted with less sensitivity to overall performance.\n\n**Question about Window Size:**\n\n**Reply:** We deliberately chose a **moderate window size**—neither too small nor too large—for the sentence embedding used by the selector network. A window size that is **too small** may cause the sentence embedding to degrade like those of a token-level representation, making it less robust under paraphrasing attacks. This is because paraphrasing typically preserves the overall sentence meaning while altering individual words through synonym substitution or structural changes. At the token level, such variations can affect the selector network’s output, making it less robust to these attacks. In contrast, by selecting an appropriately sized window, we can maintain semantic consistency with the paraphrased version, thus improving the **robustness** of the selector. We will further elaborate later.\n\nOn the other hand, if the window size is **too large**, each new token generation results in only a minor change to the window—dropping the oldest token and adding the most recent one. From a semantic perspective, this leads to minimal variation between consecutive windows, which may reduce the **sensitivity** of the selector. As a consequence, both during training and inference, the selector may fail to be of use to the token currently being generated.\n\nTherefore, choosing a **moderate and well-balanced window size** is crucial to ensure both robustness to paraphrasing and effectiveness in guiding selective watermarking.\n\n---\n\n> **Q3:** How does the model ensure robustness against paraphrasing or deletion attacks? It seems that the method does not explicitly consider these types of attacks, which could be a potential vulnerability.\n\n\n**Reply:** We appreciate your concerns and questions about robustness which we didn't explain clearly in the paper due to page limitations. We genuinely value the opportunity to clarify and address your concerns about robustness. We will **explain** **the measures during watermark design to prevent ill impact on robustness step by step** :\n\n**Measures to ensure Robustness**:\n\n Reviewer GajM asked us out of *Curiosity* that why have we chosen a hard binary mask instead of keeping the soft weights of the selector network like during training, which can be more flexible. This is a design for robustness. For the soft mask used during training, the mask is also a detection weight as in Eq.7, a token with a small mask is also give light detection weight, however, though this brings more flexible may be potentially better in terms of text quality. We believe that, comparatively, using the hard mask can better enhance the robustness of the watermarking scheme under paraphrasing attacks. From a design perspective, it is relatively simpler to categorize the outputs of the selector into two distinct classes—\"apply watermark\" and \"do not apply watermark\"—and apply watermarking accordingly. During detection, we only need to verify the outputs that were marked for watermarking, which is a simple approach.\n\nHowever, if a soft mask, as used during training, were to be adopted at inference time, the detector would likewise need to consider the soft mask values as per-token detection weights. In such a case, tokens with very low mask values would contribute minimally to detection, while the opposite for those with a high mask value. While this offers finer granularity, it introduces more complexity and uncertainty—especially more vulnerable under paraphrasing attacks.\n\nThe inputs of our selector includes a suitable window size k for obtaining the sentence embedding, using a k that is too short or if simply using the previous token embedding which would be easier to obtain and no need for the extra sentence would make it vulnerable to paraphrasing attacks since the previous may change due to the attack. Previous work[1] have shown that sentence semantic meanings are robust under paraphrasing attacks. (In their work ,they replaced KGW's[2] hashing method with a semantic one, improving robustness). This is a method done to enhance the robustness of our network output.\n\nParaphrasing attacks is hard to change \"apply watermark\" to \"do not apply watermark\" than changing its values because in our Eq.9, we penalizing those unmarklike outputs, making most outputs further form the threshold. In contrast, soft mask values are **continuous and more sensitive** to subtle changes, meaning that a paraphrasing attack may cause small shifts in mask values (e.g., a slightly increased value for a previously low-scored token, or a slightly decreased one for a previously high-scored token). These small perturbations can **accumulate** and increase the **uncertainty of watermark detection**, thus reducing robustness.\n\nThese are some of the measures taken during watermark design to ensure robustness against paraphrasing. In addition we aim to provide a selective watermarking method which can be adopted to different watermarking method. For example, the work of SIR[1] can be combined with ours by changing the hashing method from the previous tokens (like the case of KGW) to using a semantic based one for further improving robustness under paraphrasing attacks.\n\n[1] A Semantic Invariant Robust Watermark for Large Language Models，ICLR2024\n\n[2] A Watermark for Large Language Models，ICML 2023"}, {"review_id": "hxIyRo8SO2", "reviewer": "Reviewer_GajM", "summary": "In this paper, the authors propose a methodology to improve Green-list based watermarking schemes for LLMs. The method is based on the design of a so-called \"selector network\", that is, a regressor mapping a series of features to a mask. The main idea is to prevent watermarking during steps where the token distribution is almost deterministic and thus would lead to both a high quality loss and few gains in detectability if any.\n\nTo achieve this, the authors leverage a multi-objective optimization objective, balancing quality and detectability and claiming to achieve Pareto optimality. Quality is taken into account using embedding similarity (between watermarked and non-watermarked sentence), entropy of \n\nFinally, the authors compare their scheme to other classical methods, as well as comparable methods based on the entropy.", "questions": "1. Could the authors improve their evaluation by focusing on the TPR at a low FPR (at least less $10^{-5}$ ) instead of providing many insignificant metrics such as AUC, F1 etc..? The best course of action would be to switch to non-asymptotic p-values if times and resources allow it -- since I understand this would entail changing the loss of the selector network.\n2. I would forego EXP-Edit in favor of another distortion-free scheme which can actually provide theoretical guarantees \n3. Can you provide a rationale for the choice of LLMs and maybe provide an empirical analysis of the distribution of their entropy for each generation step and for different texts following my \"**Uninformed choice of LLM**\" comment ?\n4. An empirical analysis of the probability of drawing a green token under $\\mathcal{H}_1$ with and without the selector token would be welcome to get a better insight on the impact of the selector network on detection.\n5. *(Curiosity)* Why have you chosen a hard binary mask instead of keeping the soft weights of the selector network? I would guess a soft mask to be more flexible, allowing to tune the bias $\\delta$ automatically for a given distribution. Have you tried it? If yes, was it worse than the binary mask?\n\n**Score increase or decrease** : Given the current state of the paper, I see no reason to decrease the score since I am confident in the author's results and since the methodology looks sound to me. I will definitely increase the score if: 1) better metrics are provided (most important), 2) a better selection of LLM is made with regard to their entropy (if time and resources permit) and 3) the proposed improvements to the experimental settings are included.  \n\n[3] S. Aaronson and H. Kirchner. Watermarking GPT outputs, 2023. URL https://\nscottaaronson.blog/?m=202302.", "limitations": "Nothing to report", "rating": 4, "confidence": 4, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": 2, "clarity": 3, "significance": 3, "originality": 2, "strengths_and_weaknesses": "## Qualities\n\n**Clear and well-written paper**: Each step of the method is well motivated and articulated. Each loss is well described. I was able to answer most my questions during second and third read of the paper. I am confident I should be able to try replicating the results with the information contained in the paper (and in the appendix for the MLP training details).\n\n**Pertinent choice for the losses**: The collection of features and loss design for the selector network looks very complete and well-designed to me, taking into account all necessary properties for a watermark that is both robust and imperceptible. \n\n**Nice analysis of the selector outputs**: I find Figure 7 and its corresponding paragraph very insightful. I appreciate the time taken to study the function the words chosen by the selector network.\n\n## Weakness\n\n**Detection score of Green-list tokens is flawed**: The authors use the original z-score of the KGW method as their basis for detection. This assumes asymptotic normality of the score and a text long enough that normality can be invoked to compute sound statistics (p-value and confidence interval). Such assumptions are known to be unnecessary, as shown in [1]. Indeed, the sum of token scores for green-list watermarks follow a binomial distribution $\\mathcal{B}(\\gamma, l)$ under $\\mathcal{H}_0$ ($\\gamma$ being the probability of the token to be in the green list for non-watermarked text and $l$ the size of the text). This means an **exact, non-asymptotic** p-value can be computed for KGW and Unigram, leading to a precise control over Type I errors. Currently the authors have to rely on empirical measurements which are less rigorous. \n\n**Flawed choice of metrics**: This is a direct consequence of the previous problem. The authors use F1, AUC and TPR at (very!) high FPR. Obviously, watermarking schemes are good enough so that the detectability under no attack is always close to 1 for each metric. This makes comparison difficult and variations between schemes insignificant in Table 1. A better way to compare the schemes would be to fix a **low** probability of false-alarm, say $10^{-6}$, and compute the power of the test at this guaranteed rate. This can be done easily if a theoretically sound score is used during detection. In particular, using the $p$-value approach I outlined in the previous section allows to do this analysis without any problem. See any of [1,2] for a more thorough description of the method. Also note that $10^{-6}$ might still be too high a $P_{FA}$ to get significant difference in powers between watermarking schemes, in which case, I would advise the authors to go even lower.\n\n**Uninformed choice of LLM**: Two LLMs are studied, OPT-6.7B and GPT-J-6B. No rationale is provided for why these models have been chosen. It would have been a lot more pertinent to choose three different LLMs, one with low, medium and high entropy on average and see the difference in gains using the authors method. I would make the author's method even more enticing as I expect the performance to be high for low entropy models. For an idea of what kind of LLM to choose, I know [1] has performed such a study in Appendix J: *Phi* models have notoriously high entropy, *Llama-2* very low entropy and *Llama-3.1* is between the two. At the very least, an analysis of the distribution of the entropy of currently used models should be provided \n\n**Lack of link to theoretical analysis**: The distribution of the score for KGW follows a binomial distribution under $\\mathcal{B}(\\gamma^{\\prime}, l)$ under  $\\mathcal{H}_1$ too ($\\gamma^{\\prime}$ being the probability of the token to be in the green list for a watermarked text and $l$ the size of the text). The parameter $\\gamma^{\\prime}$  usually cannot be computed theoretically but is simple to estimate empirically. It would have been interesting to provide an estimation of this parameter with and without the selector network to estimate a theoretical gain/loss in detectability. More difficult, but certainly most insightful, a theoretical analysis on the distribution of the $\\gamma^{\\prime}$ within a text and its impact on detectability would have been welcomed in order to provide conditions on when focusing on certain token distributions while disregarding others is beneficial to detectability.\n\n[1] Fernandez, P., Chaffin, A., Tit, K., Chappelier, V., & Furon, T. (2023, December). Three bricks to consolidate watermarks for large language models. In 2023 IEEE International Workshop on Information Forensics and Security (WIFS) (pp. 1-6). IEEE.\n\n[2] Giboulot, E., & Furon, T. (2024, December). WaterMax: breaking the LLM watermark detectability-robustness-quality trade-off. In NeurIPS 2024-38th Conference on Neural Information Processing Systems (pp. 1-34).", "was_revised": true, "final_justification": "I increase my score due to the fact that the authors have diligently answered all my questions. Citing my own comments:\n\n> From what I can see, the TPR @ small FPR is still very high for every method. It is once again difficult for me to see any difference between either of the proposed scheme. The only two reasons I can think of is either too high of a temperature of too long of a text (how long is the text by the way?). This is why I would have appreciated to have an average p-value instead of a TPR, this would have allowed a better discrimination between the different schemes. Similarly, the ROUGE score differences do not seem significant to me.\n\n> For the analysis of $\\gamma$, I was just asking the author to compute the actual/empirical $\\gamma^{\\prime}$ on a sample of texts. This would have actually been a great way to compare schemes (since a higher $\\gamma^{\\prime}$ leads to lower p-values by definition.\n\n> For Q4, I understand the choice from the authors. I believe they could have kept a detector which takes a 0-1 decision (does the token belongs to the green-list) while keeping the soft mask during the embedding to improve text quality. But maybe I am mistaken.\n\nConsequently, the experimental design, though now better, is still somewhat lacking. As as such, I cannot go as high as an accept, but will gladly raise my score to a Borderline accept.", "rebuttal": "**Reply to Reviewer GajM**\n\n Thank you for your thorough review and for pointing out reference error issue which will be fixed by us. We are deeply grateful for the time and effort you have invested in reviewing our paper. Your valuable questions and insights have significantly contributed to enhancing our work.  To address your queries and concerns, we have prepared detailed responses for your questions and concerns and **we genuinely value the attention you give to our work.** We will explain your concerns point by point and provide you with the additional experiments results concerning using better metrics for the sake of better comparison.\n\n> **Q1:** Could the authors improve their evaluation by focusing on the TPR at a low FPR (at least less 10−5 ) instead of providing many insignificant metrics such as AUC, F1 etc..? The best course of action would be to switch to non-asymptotic p-values if times and resources allow it -- since I understand this would entail changing the loss of the selector network.\n\nWe are glad that you have pointed out a better metric for us with the aim of better comparison  between the watermarking methods, we have adopted using p-value method **in our additional experiment**. We used a probability of false-alarm small enough to make out the difference between different methods in terms of detectability instead of metrics such as tpr@1, tpr@2, which are frequently used by previous works, but may not be sufficient enough in our case. \n\n\n\n**Additional Experiments:**\n\n**We genuinely value the opportunity to clarify and address your concerns about  better detection metrics. Taking advantage of the rebuttal period, we have conducted additional experiments to provide a more comprehensive evaluation of our watermarking method and further validate the effectiveness of the proposed method.** We have adopted the ELI5 dataset which is a long form Question Answering dataset, we randomly selected 500 piece of question-answer pairs from the dataset. \n\nWe employed the selective watermarking network previously trained on the C4 dataset. Using it is sufficient for the limited time being. In evaluating detectability, we followed your suggestion and adopted the use of p-values under a small false alarm rate, which provides a more rigorous measure of detectability instead of more commonly used metrics in watermarking papers such as FPR@1 and FPR@2 . As for text quality, we used the recall variant of ROUGE-L, a simpler and more computationally efficient metric.\n\n| Method       | LLaMA3.1-8B    |           | OPT-6.7B        |           |\n| ------------ | -------------- | :-------- | --------------- | --------- |\n|              | p-value < 1e-8 | Rouge-L   | p-value < 1e-10 | Rouge-L   |\n| KGW          | 0.984          | 0.230     | 0.994           | 0.244     |\n| Sweet        | 0.986          | 0.221     | **0.996**       | 0.252     |\n| TS-watermark | 0.934          | 0.225     | 0.992           | 0.251     |\n| LTW-1 (ours) | 0.982          | **0.251** | 0.992           | **0.256** |\n| LTW-0 (ours) | **0.988**      | 0.242     | 0.994           | 0.249     |\n\n Our experimental results in the table demonstrate that our method demonstrate high detectability as well as text quality. Specifically, while Sweet has a performance slightly better than ours in terms of detectability when generated using OPT-6.7b on this task, our methods shows better overall performance, **especially the text quality when generated using LLaMA3.1-8B under this task**. Compared with TS-watermark ,which was trained to adaptively provide gamma and delta hyperparameters using the C4 dataset, **our method shows better generalization ability on a different task**.\n\nWe hope that our rebuttal could properly address your concerns and questions.  If so, we would deeply appreciate it if you could kindly raise your score (Rating: 3: Borderline reject), If not, please let us know your further concerns. Thank you for your invaluable time and consideration.\n\n---\n\n\n\n> **Q2:**  Can you provide a rationale for the choice of LLMs and maybe provide an empirical analysis of the distribution of their entropy for each generation step and for different texts following my \"**Uninformed choice of LLM**\" comment ?\n\nThank you for your valuable feedback concerning the choice of LLMs', especially regarding choosing low, medium and high entropy models. In our additional experiment, we have incorporated using Llama3.1-8B together with OPT-6.7B under the task of ELI5, an Long Form Question Answering, we also analysis of the distribution of the entropy of currently used models across the two tasks. As for the reason that we have chosen the OPT model is because it is one of the most frequently used model in the works[1-5] on watermarking ever since the KGW watermark[1]. \n\nBelow is the **analysis of entropy :**\n\nFor the original experiment using OPT and GPT-J on the C4 dataset, they have similar entropy distributions, OPT-6.7B has an average entropy of 1.9 while GPT-j has an average entropy of 2.2.  \n\nAs for task of  Long Form Question Answering using ELI5 dataset in the additional experiments, the task has a higher entropy. For Llama3.1-8b, it has an average entropy of 2.2 while OPT-6.7B has an average entropy of 2.7. \n\nFrom the entropy across the two tasks using different models , you can find out that models that have a higher entropy tends to yield better overall detectability,\n\n[1] A Watermark for Large Language Models，ICML 2023\n\n[2] Context-aware Watermark with Semantic Balanced Green-red Lists for Large Language Models. emnlp 2024\n\n[3]  Robust distortion-free watermarks for language models,TMLR\n\n[4]  Token-Specific Watermarking with Enhanced Detectability and Semantic Coherence for Large Language Models,ICML 2024\n\n[5] Adaptive Text Watermark for Large Language Models ,ICML2024\n\n---\n\n\n\n> **Q3:** Some theoretical analysis about probability of drawing a green token and impact of the selector network on detection.\n\nThank you for been interesting for for your valuable feedback concerning theoretical analysis about $\\\\gamma'$ as well as impact on detectability . We would like to provide you with a simple analysis hoping to clarify your worries.\n\nIn our method, we like to analysis $\\\\gamma'$ from two perspectives,the generated text as a whole as well as only those that have been chosen to be watermarked and detected. The parameter $\\\\gamma$ is the green-list ratio for KGW type watermarks, it is often a fixed hyperparameters throughout the generation and can be used for estimate empirically. For our watermark and other selective watermark, you can consider the $\\\\gamma'$ became 0 when decided not to add watermark to the current token being generated and the preset $\\\\gamma$ when chosen to be watermarked from the whole text perspective. The distribution of $\\\\gamma'$ depends on the selection rule, in our case the output of our model. At this time,it may being you concern about how to compute the distribution of the score for the watermark, but if from the perspective of detection, at detection period we only detect that were chosen and added a bias. Those selected tokens were added bias to according to the green-list ratio $\\\\gamma'$, thus it also follows $\\\\mathcal{B}(\\\\gamma', l)$, except $l$ in our case is the tokens being selected instead of the whole text.\n\n---\n\n\n\n> **Q4:** *(Curiosity)* Why have you chosen a hard binary mask instead of keeping the soft weights of the selector network? I would guess a soft mask to be more flexible, allowing to tune the bias δ automatically for a given distribution. Have you tried it? If yes, was it worse than the binary mask?\n\nThis is a design for robustness. For the soft mask we used during training,  the mask is also a detection weight as in Eq.7, a token with a small mask is also give light detection weight, however, though this brings more flexible may be potentially better in terms of text quality. We believe that, comparatively, using the hard mask can better enhance the robustness of the watermarking scheme under paraphrasing attacks. From a design perspective, it is relatively simpler to categorize the outputs of the selector into two distinct classes—\"apply watermark\" and \"do not apply watermark\"—and apply watermarking accordingly. During detection, we only need to verify the outputs that were marked for watermarking, which is a simple approach. \n\nHowever, if a soft mask, as used during training, were to be adopted at inference time, the detector would likewise need to consider the soft mask values as per-token detection weights. In such a case, tokens with very low mask values would contribute minimally to detection, while the opposite for those with a high mask value. While this offers finer granularity, it introduces more complexity and uncertainty—especially more vulnerable under paraphrasing attacks.\n\nParaphrasing attacks is hard to change \"apply watermark\" to \"do not apply watermark\" than changing its values. In our Eq.9, we penalizing those unmarklike outputs, making most outputs further form the threshold. In contrast, soft mask values are **continuous and more sensitive** to subtle changes, meaning that a paraphrasing attack may cause small shifts in mask values (e.g., a slightly increased value for a previously low-scored token, or a slightly decreased one for a previously high-scored token). These small perturbations can **accumulate** and increase the **uncertainty of watermark detection**, thus reducing robustness."}, {"review_id": "gnKeQGE6YL", "reviewer": "Reviewer_v5TR", "summary": "This paper introduces Learning to Watermark (LTW), a novel selective watermarking framework for large language models (LLMs). The method uses a lightweight neural network (Selector Network) trained via multi-objective optimization to decide dynamically when to watermark tokens during generation. By incorporating sentence embeddings, token entropy, and the watermark ratio, LTW aims to balance text quality and detectability more effectively than prior approaches.", "questions": "Can the selector network trained on one LLM generalize to other LLMs? In line 278, the text refers to Table 3 and Table 4, but Table 4 is missing from the paper. In Tables 1 and 2, the number of decimal places is inconsistent. Could the authors explain the reason for this formatting choice?", "limitations": "1. The method emphasizes the need for an adaptive threshold to decide whether to watermark a given token. However, Table 3 shows that the gains in detectability and quality from this adaptive threshold are quite limited. This raises questions about whether this component is really necessary.\n\n2. Since LTW requires an additional sentence embedding model and selector network during both generation and detection, it introduces higher computational and memory overhead. The authors should provide a discussion and experiments on the time and space costs of the approach.\n\n3. From the proposed method, it appears that the watermark ratio is adjusted based on the portion of text already generated. This results in a higher watermark ratio in the earlier parts of the text and a lower ratio in later parts. This design may naturally expose the watermark to truncation attacks, where keeping only the later segments could reduce detectability. Could the authors clarify whether this would significantly impact detection performance?", "rating": 4, "confidence": 5, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": 3, "clarity": 2, "significance": 2, "originality": 3, "strengths_and_weaknesses": "**Strengths:**\n1. The paper is clearly written and well-structured and easy to follow.\n2. LTW proposes learning selective watermarking to balance text quality and detectability, going beyond entropy thresholds and introducing adaptive criteria based on richer contextual signals.\n\n**Weaknesses:**\n1. Eq.5 shows that the objective is to maximize the semantic similarity between the watermarked and unwatermarked text. However, it is unclear what exactly the unwatermarked text refers to and how it is generated in practice. The authors should clarify this point to avoid confusion about the optimization target.\n2. The paper mentions that LTW achieves strong robustness. Why is this not discussed in the introduction? More importantly, why does LTW improve robustness?  For example, Table 1 shows that LTW achieves higher robustness than KGW, but the reasons for this advantage are not clearly discussed.\n3. According to Table 1, Table 2, Figure 8, and Figure 9, LTW outperforms KGW in both detectability and text quality. Yet from Equations 3 and 7, LTW seems like a relaxed version of KGW. Why does this weaker formulation yield a better results?\n4. There is limited discussion and no ablation study on the impact of the losses defined in Eq.(10) and Eq.(11), which are central to the optimization process.", "was_revised": false, "final_justification": "", "rebuttal": "**Reply to Reviewer v5TR**\n\nThank you for your thorough review and for pointing out the Table number reference issues. It was a reference mistake that we will correct. We are deeply grateful for the time and effort you have invested in reviewing our paper. To address your queries and concerns, we have prepared detailed responses for your questions and concerns and **we genuinely value the attention you give to our work.** \n\n> **Q1:** The authors should clarify what exactly the unwatermarked text refers to and how it is generated in practice to avoid confusion about the optimization target.\n\n**Reply:** We apologize for any misunderstanding that may have arisen regarding the meaning of the unwatermarked text and we will further explain it and how it is generated. In Eq.5 we aim to design a metric to quantify the impact of watermarking on generation quality compared to non-watermarked outputs generated by the LLM. By optimizing this objective during training, we seek to minimize the degradation introduced by watermarking. Naturally, one effective approach is to compute the cosine similarity between the generated embedding of the tokens that were generated without the watermark and that of the watermarked one. Therefore, during training, we generate both the non-watermarked and watermarked embeddings for the same prompt to enable this comparison. The cosine similarity is then computed and used as a loss term, encouraging the watermarked content to remain semantically closer to its non-watermarked counterpart. \n\n---\n\n\n\n> **Q2:** Why is robustness not discussed in the introduction? More importantly, why does LTW improve robustness?\n\n**Reply:** We appreciate your concerns and questions about robustness which we didn't explain clearly in the paper due to page limitations. We genuinely value the opportunity to clarify and address your concerns about robustness. We will explain **the measures during watermark design to prevent ill Impact on robustness step by step** :\n\nReviewer GajM asked us out of *Curiosity* that why have we chosen a hard binary mask instead of keeping the soft weights of the selector network like during training, which can be more flexible. This is a design for robustness. For the soft mask used during training,  the mask is also a detection weight as in Eq.7, a token with a small mask is also give light detection weight, however, though this brings more flexible may be potentially better in terms of text quality. We believe that, comparatively, using the hard mask can better enhance the robustness of the watermarking scheme under paraphrasing attacks. From a design perspective, it is relatively simpler to categorize the outputs of the selector into two distinct classes—\"apply watermark\" and \"do not apply watermark\"—and apply watermarking accordingly. During detection, we only need to verify the outputs that were marked for watermarking, which is a simple approach. \n\nHowever, if a soft mask, as used during training, were to be adopted at inference time, the detector would likewise need to consider the soft mask values as per-token detection weights. In such a case, tokens with very low mask values would contribute minimally to detection, while the opposite for those with a high mask value. While this offers finer granularity, it introduces more complexity and uncertainty—especially more vulnerable under paraphrasing attacks.\n\nThe inputs of our selector includes a suitable window size k for obtaining the sentence embedding, using a k that is too short or if simply using the previous token embedding which would be easier to obtain and no need for the extra sentence would make it vulnerable  to paraphrasing attacks since the previous may change due to the attack. Previous work[1] have shown that sentence semantic meanings are robust under paraphrasing attacks.(In their work ,they replaced KGW's[2] hashing method with a semantic one, improving robustness). This is a method done to enhance the robustness of our network output.\n\nParaphrasing attacks is hard to change \"apply watermark\" to \"do not apply watermark\" than changing its values. In our Eq.9, we penalizing those un-masklike outputs, making most outputs further form the threshold. In contrast, soft mask values are **continuous and more sensitive** to subtle changes, meaning that a paraphrasing attack may cause small shifts in mask values (e.g., a slightly increased value for a previously low-scored token, or a slightly decreased one for a previously high-scored token). These small perturbations can **accumulate** and increase the **uncertainty of watermark detection**, thus reducing robustness.\n\nThese are some of the measures taken during watermark design to prevent ill impact on robustness, we hope this helps you better understand the measures we have taken in our design to enhance robustness though they may not be something novel enough to boast about.\n\n [1] A Semantic Invariant Robust Watermark for Large Language Models，ICLR2024\n\n [2] A Watermark for Large Language Models，ICML 2023\n\n---\n\n\n\n> **Q3:**  Why does this weaker formulation yield a better results?\n\n**Reply:** We need to point out that a relaxed formula doesn't mean that it would provide less results, our relaxed version is designed to selectively apply and detect watermark based on a \"rule\", which is the output of the selector network. The previous selective watermark, Sweet[1] , can also be seen as a relaxed version of KGW. The rule of which is whether the token has passed the entropy threshold. The method has achieved better coding ability and detectability than KGW though it is also a relaxed formulation.\n\nWe believe the reason that our methods are that they can grasp the tokens for which watermarking can bring more benefit. For example, taking entropy as an example, when the entropy is low, the model’s output distribution becomes highly concentrated on a few specific tokens. In such cases, watermarking is more prone to introducing errors, as perturbing the logits may lead to the selection of low-probability tokens that were unlikely to be chosen otherwise. Additionally, due to the sharp concentration of probabilities, bias added  to tokens in the green list tends to be more insignificant, as these tokens already have low baseline probabilities compared to the original high-probability ones . As a result, the sampling process remains dominated by high-probability tokens—which are often located in the red list (as the choice of the gamma value typically is smaller than 0.5). This increases the likelihood that the sampled tokens are in the red-list, thereby reducing detectability. We believe that our method can take grasping when to  make the decision watermark further, watermarking those that improves detectability and afflicts text quality less.\n\n [1] Who Wrote this Code? Watermarking for Code Generation, ACL 2024\n\n---\n\n\n\n> **Q4:** Can the selector network trained on one LLM generalize to other LLMs? \n\n**Reply:**  Yes ,The main results of our method using GPT-J used the selector trained using the OPT model. Part of the reason we used a separate sentence embedding model and obtain its sentence embedding is so that the network trained on one model can generalize to other LLMs. The results show that it can generalize to other LLMs. \n\n---\n\n\n\n> **Q5:** The authors should provide a discussion and experiments on the time and space costs of the approach.\n\n**Reply:** The additional memory introduced by our watermarking method primarily stems from the use of an extra sentence embedding model and a lightweight selective neural network. In practice, the watermark generation process incurs approximately **0.5 GB** of additional GPU memory consumption, most of which is attributed to the sentence embedding model. \n\nAs for inference time, we report  generated 100 samples using various watermarking methods for exactly 200 tokens, here is average time required for each piece of text:\n\n- **Ours (LTW)**: 6.25 seconds / text\n- **KGW**: 5.81 seconds / text\n- **SWEET**: 5.93 seconds / text\n- **TS-watermark**: 6.04 seconds / text\n- **EXP-edit**: 34.80 seconds / text\n\n---\n\n\n\n> **Q6:**  This design may naturally expose the watermark to truncation attacks, where keeping only the later segments could reduce detectability. Could the authors clarify whether this would significantly impact detection performance?\n\n**Reply**: Your suggestion is very professional, and we appreciate your insight In addition to our theoretical analysis, we have conducted the following supplementary experiment to address it.\n\nIn our watermarking method, we introduce the watermarking ratio as a controllable factor not only for the reasons stated in the paper, but also because we aim to design a watermarking approach that is more practical and generalizable than SWEET for real-world applications. Specifically, unlike SWEET, which requires task-specific hyperparameter tuning (e.g., determining the optimal entropy threshold through extensive search) prior to deployment, our method is designed to work without such tuning, making it more plug-and-play.\n\nThe watermarking ratio serves as a balancing mechanism between entropy and semantics. It allows us to encourage watermarking when the overall ratio is too low and discourage it when the ratio is too high—rather than merely encouraging watermarking indiscriminately. \n\nTo verify this, we recorded 500 generations using our watermarking method and tracked whether each generated token was watermarked. For each generation, we generated 200 tokens and split them into the **first 100 tokens** and the **last 100 tokens**. We then calculated and compared the proportion of tokens that were watermarked in both segments. The results show that the first half had a watermarking ratio of **38%**, while the second half had **35.4%**, indicating that no significant imbalance occurred for the first half and the later part. Thus, doing a simple truncation may not bring too much impact."}, {"review_id": "u1Peo0lsa4", "reviewer": "Reviewer_9Hqn", "summary": "This paper trains a neural network to achieve the selective watermarking strategy by analyzing sentence embeddings, token entropy, and the current watermarking ratio, which achieves a better balance between text quality and detectability.", "questions": "Please see above.", "limitations": "Please see above.", "rating": 4, "confidence": 3, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": 2, "clarity": 3, "significance": 2, "originality": 2, "strengths_and_weaknesses": "Strengths:\n\n1. This paper provides an optimized method to achieve a selective watermarking approach.\n\n2. This paper provides extensive experiments to demonstrate the effectiveness of the proposed methods.\n\nWeaknesses:\n\n1. Could the authors consider using additional datasets, such as long-form QA, to further validate the effectiveness of the proposed method? \n\n2. OPT-6.7b is too old compared with the most recent open-source LLM, like Llama-3.\n\n3. Table 1 and 2 provide a performance comparison of different watermark methods. Why does Table 2 not provide TPR@2? Moreover, this paper strengthens the trade-off between the text quality and detectability. One better way to demonstrate this trade-off is to provide TPR with a lower FPR, such as 1%, 0.1%, 0.01%.\n\n4. Could the authors explain the reason for using the similarity metric? It seems that this metric can not accurately represent the quality of watermarked text, because it is not necessary to have a high similarity between watermarked text and reference text especially for the open generation dataset (C4).\n\n5. Is it possible to report the rate of watermarked tokens?", "was_revised": true, "final_justification": "In general, most of the concerns have been resolved.", "rebuttal": "**Reply to Reviewer 9Hqn**\n\nThank you for your valuable and insightful comments. We will explain your concerns and questions point by point. In addition, we will provide you with **the results of the additional experiment which may address your concern**. We hope that our rebuttal could properly address your concerns and questions.  If so, we would deeply appreciate it if you could kindly raise your score (Rating: 3: Borderline reject), If not, please let us know your further concerns. Thank you for your invaluable time and consideration.\n\n> **Q1:** Could the authors consider using additional datasets, such as long-form QA, to further validate the effectiveness of the proposed method?\n\n**Reply:** We thank you for your advice of suggesting us to use additional  long-form QA dataset to further validate the effectiveness of the proposed method, which will be provided in the following additional experiment part using the ELI5 dataset. The original dataset, C4 realnewslike dataset was frequently used in previous works [1-5] of LLM watermarking ,sometimes the only dataset used in the paper ,thus we used it for our original experiments.\n\n[1] A Watermark for Large Language Models，ICML 2023\n\n[2] Context-aware Watermark with Semantic Balanced Green-red Lists for Large Language Models. emnlp 2024\n\n[3]  Robust distortion-free watermarks for language models,TMLR\n\n[4]  Token-Specific Watermarking with Enhanced Detectability and Semantic Coherence for Large Language Models,ICML 2024\n\n[5] Adaptive Text Watermark for Large Language Models ,ICML2024\n\n---\n\n\n\n> **Q2:** OPT-6.7b is too old compared with the most recent open-source LLM, like Llama-3.\n\nWe thank you for your advice of suggesting that we use more recent open-source LLM such like Llama3,  we will provide the results of using it in our additional experiments.  We have to point out that we have chosen the OPT model is because it is one of the most frequently used model in the works[1-5] on watermarking ever since the KGW watermark[1] , and thus is a reason we have included it.\n\n[1] A Watermark for Large Language Models，ICML 2023\n\n[2] Context-aware Watermark with Semantic Balanced Green-red Lists for Large Language Models. emnlp 2024\n\n[3] Robust distortion-free watermarks for language models,TMLR\n\n[4] Token-Specific Watermarking with Enhanced Detectability and Semantic Coherence for Large Language Models,ICML 2024\n\n[5] Adaptive Text Watermark for Large Language Models ,ICML2024\n\n\n\n**Additional experiments to provide a more comprehensive evaluation of LTW:**\n\nWe appreciate the reviewer's insightful observation regarding the need for comparisons using additional dataset and additional model to evaluate the effectiveness of our watermark method. **We genuinely value the opportunity to clarify and address your concerns. Taking advantage of the rebuttal period, we have conducted additional experiments to provide a more comprehensive evaluation of our watermarking method and further validate the effectiveness of the proposed method.** We have adopted the ELI5 dataset which is a long form Question Answering dataset, we randomly selected 500 piece of question-answer pairs from the dataset. \n\nWe employed the selective watermarking network previously trained on the C4 dataset. Due to the limited time available for the rebuttal period, we were unable to retrain a network using data from the QA dataset which would undoubtedly  bring better performance. However, the model trained on C4 demonstrates sufficient generalization ability, using it is sufficient for the limited time being. In evaluating detectability, we followed Reviewer GajM’s suggestion and adopted the use of p-values under a small false alarm rate, which provides a more rigorous measure of detectability instead of more commonly used metrics in watermarking papers such as FPR@1 and FPR@2 . As for text quality, we used the recall score of of ROUGE-L, a simpler and more computationally efficient metric.\n\n| Method       | LLaMA3.1-8B    |           | OPT-6.7B        |           |\n| ------------ | -------------- | :-------- | --------------- | --------- |\n|              | p-value < 1e-8 | Rouge-L   | p-value < 1e-10 | Rouge-L   |\n| KGW          | 0.984          | 0.230     | 0.994           | 0.244     |\n| Sweet        | 0.986          | 0.221     | **0.996**       | 0.252     |\n| TS-watermark | 0.934          | 0.225     | 0.992           | 0.251     |\n| LTW-1 (ours) | 0.982          | **0.251** | 0.992           | **0.256** |\n| LTW-0 (ours) | **0.988**      | 0.242     | 0.994           | 0.249     |\n\n Our experimental results in the table demonstrate that our method demonstrate high detectability as well as text quality. Specifically, while Sweet has a performance slightly better than ours in terms of detectability when generated using OPT-6.7B on this task, our methods shows better overall performance, **especially the text quality when generated using LLaMA3.1-8B under this task**. Compared with TS-watermark ,which was trained to adaptively provide $\\\\gamma$ and $\\\\delta$ hyperparameters using the C4 dataset, our method shows better generalization ability .\n\n---\n\n\n\n> **Q3:**  Why does Table 2 not provide TPR@2? Moreover, this paper strengthens the trade-off between the text quality and detectability. One better way to demonstrate this trade-off is to provide TPR with a lower FPR, such as 1%, 0.1%, 0.01%.\n\n**Reply:** We used tpr@2 because metrics like tpr@1 and tpr@2 are frequently used in previous works ,for Table 1,the results of using tpr@1 are as follows shows, only the metrics of EXP-edit and TS-watermark became lower. As for Table 2, we didn't provide tpr@2 because at 2% fpr they all achieved 100% tpr when not under attack, this can also be observed from the AUROC metrics value. In the additional experiment, however, we will set a low probability of false-alarm,such as 1e-8, and use $p-value < P_{FA}$ as the metric in the additional experiments to show the high detectability our method possess.\n\n|       | kgw-1 | unigram | EXP-edit | sweet | TS-watermark | LTW-1 | LTW-0 |\n| ----- | ----- | ------- | -------- | ----- | ------------ | ----- | ----- |\n| tpr@1 | 0.998 | 1.000   | 0.854    | 1.000 | 0.992        | 1.000 | 1.000 |\n\n---\n\n\n\n> **Q4:** Could the authors explain the reason for using the similarity metric? \n\n**Reply:** We understand the reviewer's  question over the similarity metric. In this paper, we used  the cosine Similarity between the sentence embeddings for assessing the text quality and semantic coherence between the generated text and the original part that was used as reference text. In the work of one of our baseline methods, TS-Watermark[4] ,also adopted similar approach for evaluate the semantic coherence of the watermarked text under the C4 dataset.  In our case, we use data from the C4 realnewslike subset, which are more like news articles for the LLMs to generating continuation writing based on the part of text that is given. A higher similarity indicates a more contextually appropriate and more coherent connection with the preceding context.\n\n---\n\n\n\n> **Q5:** Is it possible to report the rate of watermarked tokens?\n\n**Reply:** We appreciate the reviewer's concerns regarding the rate of the  watermarked tokens, the average watermarked percentage  of text generated under the c4 realnewslike subset is about 36.73%, while those generated under the ELI5 dataset is about 47.35%.  This may be due to the difference between the tasks."}]

Author rebuttal:

Meta-review: This paper received all positive reviews. After discussion, reviewers agreed to accept this paper.  The authors are encouraged to address the major concerns in the camera-ready version.
1. Add experimental evaluation on other types of attacks
2. Justify the choice of evaluation metrics
3. Add more ablation studies.

The authors’ rebuttal and messages were carefully ready, discussed, and considered.
Decision: Accept (poster)