Title: Enhancing LLM Watermark Resilience Against Both Scrubbing and  Spoofing Attacks
Venue: NeurIPS 2025
Accepted: yes
Keywords: machine learning security

llm watermarking
Abstract: Watermarking is widely regarded as a promising defense against the misuse of large language models (LLMs); however, existing methods are fundamentally constrained by their vulnerability to scrubbing and spoofing attacks. This vulnerability stems from an inherent trade-off governed by watermark window size: smaller windows resist scrubbing better but are easier to reverse-engineer, enabling low-cost statistics-based spoofing attacks. This work expands the trade-off boundary by introducing a novel mechanism, equivalent texture keys, where multiple tokens within a watermark window can independently support the detection.  Based on the redundancy, we propose a watermark scheme with **S**ub-vocabulary decomposed **E**quivalent t**E**xture **K**ey (**SEEK**). SEEK achieves a Pareto improvement, enhancing robustness to scrubbing attacks without sacrificing resistance to spoofing.


Peer reviews:
{"IS_META_REVIEW": false, "comments": "SUMMARY:\nThe paper addresses the vulnerability of watermarking techniques used to protect large language models (LLMs) from misuse. Traditional watermarking methods face challenges from scrubbing and spoofing attacks, which can undermine their effectiveness. The authors introduce a novel mechanism called equivalent texture keys, which allows multiple tokens within a watermark window to independently support watermark detection. This innovation leads to a new watermarking scheme called Sub-vocabulary decomposed Equivalent tExture Key (SEEK).\n\nIn summary, the paper's contributions include:\n- Introducing the equivalent texture key mechanism to enhance watermark robustness.\n- Proposing SEEK, a new watermarking scheme that balances scrubbing and spoofing defenses while maintaining text quality.\n- Demonstrating through experiments that SEEK offers a Pareto improvement over prior watermarking schemes .\n\nSTRENGTHS AND WEAKNESSES:\nStrengths\n- Proposes a novel mechanism (equivalent texture keys) and the SEEK scheme to overcome the inherent scrubbing-spoofing robustness trade-off in LLM watermarking. Achieves Pareto improvement by enabling large watermark windows (spoofing resistance) while maintaining scrubbing robustness via hash-space optimization.\n- Demonstrates state-of-the-art results across 5+ datasets (WikiText, C4, Dolly-CW, MMW):\n   - +82–92% spoofing robustness (FPR reduction)\n   - +6–24% scrubbing robustness (TP@1% gain)\n   - Maintains text quality (perplexity/log-diversity ≈ baselines).\n- Formally proves key properties:Expected erased tokens for watermark removal increases with smaller hash space (Prop 4.2), Hash collision probability bounds (Prop 4.1), Robustness guarantees under adversarial modifications (Appendix D).\n\nWeakness\n\n- While the paper shows strong performance across various datasets, it could benefit from a more detailed discussion on how the proposed method generalizes to different types of language models and real-world scenarios. Additional experiments on diverse models and tasks could strengthen this aspect.\n- SEEK’s sub-vocabulary decomposition assumes uniform token distribution. Performance may degrade for low-entropy/specialized domains (e.g., medical/legal jargon) where sub-vocabulary imbalance could affect watermark stability.\n- Experiments only compare KGW variants (e.g., KGW-Min/Left/Sum). Lacks comparison with other methods, eg.  SIR[1]\n- This paper lacks ablation study. No component-wise analysis to disentangle contributions of: Equivalent texture keys (hash-space reduction), Sub-vocabulary decomposition, Obscures whether both innovations are essential for Pareto gains.\n\n[1] Liu, Aiwei, et al. \"A semantic invariant robust watermark for large language models, 2024.\" URL https://arxiv. org/abs/2310.06356.\n\nQUESTIONS:\n- While Figure 2 shows spoofing robustness improves with larger windows for KGW, Table 2 reports SEEK results only for (h=6, d=6) and (h=16, d=16). How does SEEK's scrubbing/spoofing trade-off specifically vary with window size?\n- Table 6 tests SEEK on 7B–13B models, showing consistent gains. Is SEEK's improvement consistent across scales (e.g., <1B or >30B models)?\n- The paper only compares against KGW-family methods. Yet recent works like UNIGRAM (fixed green list), SemStamp (semantic hashing), and CATER (contextual watermarks) also address spoofing/scrubbing trade-offs. Do you compare with these methods?\n\nLIMITATIONS:\nyes", "TITLE": "Official Review", "RECOMMENDATION": 5, "REVIEWER_CONFIDENCE": 4, "CLARITY": 3, "IMPACT": 3, "ORIGINALITY": 3, "DATE": "2025-06-30"}

{"IS_META_REVIEW": false, "comments": "SUMMARY:\nThe paper introduces a novel watermarking algorithm designed to be robust against both spoofing and scrubbing attacks. The core idea leverages insights: increasing the context window enhances resistance against spoofing attacks, while reducing the cardinality of the hash space used to generate the watermark key enhances robustness against scrubbing attacks. Consequently, the proposed algorithm, SEEK, minimizes the output space of the hashing function while maintaining a large context window size for watermark key construction. Additionally, SEEK adapts the method for partitioning vocabulary into green and red sets based on this refined watermark key construction. Extensive evaluations demonstrate SEEK’s robustness against both spoofing and scrubbing attacks.\n\nSTRENGTHS AND WEAKNESSES:\n## Strengths\n1. The algorithm effectively reduces the trade-off between robustness against spoofing attacks and scrubbing attacks, which is a common limitation in existing watermarking algorithms.\n2. The paper is clearly written and easy to follow. Figure 4, in particular, provides an excellent overview of the differences between SEEK and KGW.\n3. The motivation behind the algorithm, specifically how the window size and hash space cardinality impact watermark robustness against spoofing and scrubbing attacks, is clearly articulated. This analysis highlights the potential reasons for the observed trade-offs.\n\n## Weaknesses\n1. The evaluation of text quality has significant limitations, resulting in an unclear assessment of SEEK's actual impact on generated text quality. Additional experimental results are required to conclusively demonstrate that SEEK-generated text remains usable and maintains high quality. \n2. The paper claims reduced hash space cardinality improves robustness against scrubbing attacks without affecting spoofing resistance. However, an implicit trade-off exists and should be explicitly analyzed.\n3. SEEK may incur increased computational overhead during text generation, potentially impacting its practical application, particularly with larger vocabulary sets. This concern should be explicitly addressed in the evaluation to clarify the practical limitations.\n\nQUESTIONS:\n1. In Algorithm 1, Line 5, how is the vocabulary partitioned into sub-vocabularies? Does the partition depend solely on \\$\\zeta\\$? Please explicitly clarify this in Line 221 and Algorithm 1, Line 5.\n\n2. The paper claims, \"reducing the hash space size does not influence the watermark window size, thereby maintaining similar spoofing resistance.\" However, there seems to be an implicit relationship between the window size (\\$h\\$) and the cardinality of the hash space (\\$d\\$). My question: With a fixed hash space cardinality (\\$d\\$), isn't there a limit to how large or small the window size (\\$h\\$) can practically be?\n   1. If \\$h\\$ is significantly larger than \\$d\\$, it becomes likely that all hash values appear within the window, making SEEK behave like a unigram watermark (assuming sub-vocabularies partitioned solely by \\$\\zeta\\$). For example, with \\$d=6\\$ (as in the paper), if \\$h=30\\$, the probability of encountering all hash values is already greater than 0.96. Could you verify and discuss this explicitly?\n   2. Conversely, if \\$h\\$ is much smaller than \\$d\\$, collisions become less likely (as indicated by Proposition 4.1), potentially decreasing robustness against scrubbing attacks.\n   Given these points, isn't there still an explicit trade-off between robustness against spoofing and scrubbing attacks by varying \\$h\\$ for a fixed \\$d\\$? If this reasoning is correct, please explicitly state it in the paper and provide evaluations of SEEK against spoofing and scrubbing attacks for different combinations of \\$h\\$ and \\$d\\$.\n\n3. There are significant concerns about the text quality evaluation:\n\n   1. The P-SP scores reported in Figure 6 are very low (<0.4). Previous literature (e.g., \\[29]) indicates that a score above \\~0.7 is necessary to maintain linguistic fidelity. Thus, current results do not support the claim of preserving linguistic fidelity. Clarify or provide new evidence supporting your claims.\n   2. The paper uses OPT-2.7B as the oracle model to evaluate LLaMA2-13B-generated text. Typically, the oracle model should have higher quality than the target model (e.g., \\[28], \\[31]). Using OPT-2.7B might bias results unfairly against LLaMA2-13B. Clarify this and justify the oracle model choice.\n   Given this, Figure 6, as presented, inadequately demonstrates SEEK’s true impact on text quality compared to KGW. Proper and additional evaluations or metrics are necessary to substantiate your claims clearly.\n\n4.  Although SEEK aims to increase vocabulary diversity, log-diversity metrics are missing. Include these results explicitly.\n\n5. The paper mainly evaluates the case where \\$\\delta=5.0\\$ and \\$\\gamma=0.25\\$. These settings inherently cause significant performance drops (as indicated in Figure 6) and simplify spoofing attacks. Include evaluations using more practical values of \\$\\delta\\$ that maintain acceptable text quality.\n\n6. What is the time complexity of text generation for SEEK compared to KGW? How does complexity scale with increased vocabulary size, given the necessity of partitioning vocabularies and assigning different keys? Provide empirical timing results explicitly for SEEK and KGW.\n\nLIMITATIONS:\nyes", "TITLE": "Official Review", "RECOMMENDATION": 5, "REVIEWER_CONFIDENCE": 4, "CLARITY": 2, "IMPACT": 2, "ORIGINALITY": 4, "DATE": "2025-07-02"}

{"IS_META_REVIEW": false, "comments": "SUMMARY:\nThe authors study LLM watermarking, aiming to address vulnerability to spoofing and scrubbing attacks. To this end, they propose SEEK, a variant of the KGW approach that simultaneously improves the resistance to both, while not degrading text quality. The authors support this with extensive experiments.\n\nSTRENGTHS AND WEAKNESSES:\nStrengths: \n- The authors focus on an important problem (security and robustness of LLM watermarking).\n- Experiments are well set up and thorough along many dimensions (properties of interest, models, baselines, experimental parameters), and the key results seem to convincingly demonstrate that SEEK has benefits over the main baselines.\n- While SEEK is heavily based on KGW, the proposed modifications are well-motivated and to the best of my knowledge novel.\n- The paper is well-written. I especially appreciate how the authors introduce their method by first motivating and addressing the security-robustness tradeoff by reducing the hash space size (4.1), and then identifying and resolving a new issue that appears in this regime, which is the lack of diversity (4.2). This contextualizes SEEK and provides new insight into the parameter space of KGW.\n\nWeaknesses:\n- In my opinion, the spoofing-scrubbing tradeoff is misrepresented in the paper which may confuse readers as this is the key motivation behind the paper. In particular, my interpretation of [26] slightly differs from the authors'. While it is true that the context size controls the scrubbing-spoofing tradeoff, I believe the point of [26] was that when stealing-based spoofing becomes possible, scrubbing is also much easier as using the learned knowledge enables \"assisted scrubbing attacks\". Thus \"too small h\" => \"both spoofing and scrubbing are easy\", while \"too large h\" => \"spoofing hard, scrubbing easy\". As the authors' main motivation is the second regime and they successfully address this I do not find this to be a major weakness, but I believe the tradeoff should be stated more precisely. On top of that, for completeness, the experiments should include assisted scrubbing and not only blind scrubbing attacks---I do not expect them to work on SEEK as the spoofing also fails.\n- It is not fully clear to me what the tradeoffs of (d,h) are with SEEK as SEEK-L seems sometimes better and sometimes worse than the main variant chosen by the authors. The paper would be stronger with such a discussion, in particular: what parameter choices are obviously bad? Do authors always suggest setting d==h? If d>>h, I assume running watermark stealing as if the scheme was Unigram would recover the split given by $\\theta_{default}$? Such explorations seem to be missing and would strengthen the paper. \n\nIn conclusion, despite the weaknesses above I believe the paper is a valuable addition to the conference and will inspire meaningful discussion as it provides a new perspective on key design components of LLM watermarking schemes and the security-robustness tradeoff.\n\n- Nit: the authors use both 1e-2 and 1% to denote the FPR of 0.01 across the paper, it would be cleaner to unify this.\n\nQUESTIONS:\n- Can you comment on the questions in the second weakness above related to tradeoffs of SEEK?\n- Why does Figure 2 use both 1e-2 and 1e-3 on different axes as the FPR? \n- How are the numbers used in the abstract computed? I did not find this clarified.\n\nLIMITATIONS:\nYes", "TITLE": "Official Review", "RECOMMENDATION": 5, "REVIEWER_CONFIDENCE": 4, "CLARITY": 4, "IMPACT": 3, "ORIGINALITY": 3, "DATE": "2025-07-05"}

{"IS_META_REVIEW": false, "comments": "SUMMARY:\nThis paper introduces SEEK, a watermarking scheme for large language models that resolves the inherent trade-off between resistance to scrubbing and spoofing attacks. The key insight is the use of equivalent texture keys, which enable multiple tokens within a window to independently support watermark detection. By decomposing watermark construction across sub-vocabularies, SEEK preserves text quality while improving robustness. The method establishes a new design paradigm for secure and resilient watermarking in generative language models.\n\nSTRENGTHS AND WEAKNESSES:\n### Strengths:\n- The paper introduces the concept of equivalent texture keys, where multiple tokens in a watermark window can independently support watermark detection. This mechanism enables redundancy in detection, making it harder for scrubbing attacks to eliminate the watermark fully.\n\n- The paper addresses a core limitation in existing LLM watermarking methods by eliminating the trade-off between scrubbing and spoofing robustness. Prior methods typically required choosing between short windows and long windows, but SEEK manages to improve both simultaneously.\n\n- The proposed SEEK scheme uses sub-vocabulary decomposition to maintain a high density of equivalent texture keys while ensuring a diverse set of green-red partitions. This allows the method to achieve robustness without degrading text generation quality, which is a common issue in reduced-hash watermark designs.\n\n- The threat model is clearly defined and aligns with real-world deployment scenarios, where attackers can access model outputs via APIs and have full knowledge of the watermarking scheme. This makes the evaluation credible and practically relevant.\n\n### Weaknesses:\n- The method relies on reducing the hash space to increase the likelihood of equivalent texture keys, which introduces a potential risk of overfitting to specific green-red list patterns. Although the sub-vocabulary decomposition aims to mitigate this, the paper does not thoroughly analyze long-term distributional shifts or adversarial adaptation under this structure.\n\n- While the notion of equivalent texture keys is novel, the concept builds heavily on existing KGW-Min mechanisms and hash-collision strategies. The technical advancement, though well-executed, may be viewed as incremental rather than fundamentally transformative.\n\n- The empirical evaluation, though extensive, is limited to statistical and distillation-based spoofing attacks. It does not explore more advanced or adaptive spoofing strategies that may emerge once attackers are aware of the SEEK design, such as gradient-based spoofing or token-level substitution with semantic constraints.\n\n- The use of disjoint sub-vocabularies introduces a structural assumption about token distribution and semantic independence across partitions. The paper does not investigate whether this assumption holds consistently across languages, domains, or low-resource settings.\n\n- SEEK requires careful tuning of hash space size and watermark window length. The method's sensitivity to these hyperparameters is acknowledged but not deeply explored, which may limit reproducibility and practical deployment without further ablation studies.\n\nQUESTIONS:\n1. The method leverages hash space reduction to promote equivalent texture keys, which introduces redundancy into detection. However, it remains unclear how this redundancy behaves under distribution shift or adaptive attackers who may exploit repeated patterns. Could the authors provide a theoretical or empirical analysis of how SEEK performs under longer sequences, multilingual corpora, or adversaries trained to detect and exploit common green list configurations? A more thorough analysis here could significantly strengthen confidence in the method’s generality.\n\n2. The sub-vocabulary decomposition approach assumes that splitting the vocabulary into disjoint subsets does not harm semantic coherence or introduce artifacts. Could the authors provide an empirical study or qualitative examples across different domains to support this assumption? For instance, is there a drop in quality for tasks involving specialized vocabulary or in low-resource domains? Clarification here would increase confidence in SEEK's applicability across a broader range of use cases.\n\n3.The method introduces several hyperparameters (e.g., watermark window size and hash space cardinality), but their tuning appears somewhat ad hoc. Could the authors provide a sensitivity analysis or offer guidance for selecting these parameters in practice? In particular, how does the method perform when these parameters are misaligned with the dataset’s characteristics? More insight here would improve reproducibility and strengthen the case for real-world adoption.\n\nLIMITATIONS:\nPlease see the weaknesses.", "TITLE": "Official Review", "RECOMMENDATION": 5, "REVIEWER_CONFIDENCE": 4, "CLARITY": 3, "IMPACT": 2, "ORIGINALITY": 2, "DATE": "2025-07-06"}