Title: Practical and Effective Code Watermarking for Large Language Models
Venue: NeurIPS 2025
Accepted: yes
Keywords: Large Language Model

LLM Watermarking

Code Watermarking
Abstract: The rapid advancement of Large Language Models (LLMs) in code generation has raised significant attribution and intellectual property concerns. Code watermarking offers a potential solution but faces unique challenges due to programming languages' strict syntactic constraints and semantic requirements.
To address these challenges, we introduce ACW (AST-guided Code Watermarking), a novel adaptive framework that leverages Abstract Syntax Tree (AST) analysis during training to learn watermark embedding strategies. Our framework identifies substitutable code components and strategically biases token selections to embed watermarks. We also propose a novel sampling scheme that distributes tokens between green/red lists according to semantic context, ensuring statistical distinguishability while preserving code functionality. Extensive experiments demonstrate that ACW achieves a significant improvement in watermark detection accuracy compared to existing methods, with negligible impact on code functionality. This adaptive framework offers a promising solution for effective and practical code watermarking in the age of LLMs. Our code is available at: https://github.com/TimeLovercc/code-watermark.


Peer reviews:
{"IS_META_REVIEW": false, "comments": "SUMMARY:\nIn this work, the authors propose some improvement over the Green-Red approach for watermarking LLM-generated code. There are two main ideas: (1) to consider the code AST and some entropy-based selection on which tokens to apply the watermark; (2) to partition the green/red token such that valid solutions are likely present in both sets. The proposed method is evaluated against three LLMs (two of them are 1.5B and the other one 8B) and two common code benchmarks (HumanEval and MBPP) and the experimental results suggest that the proposed approach outperforms the baseline approaches.\n\nSTRENGTHS AND WEAKNESSES:\nStrength\n\nThe draft is fairly easy to follow. The idea is explained sufficiently and so is the analysis of the experimental results. \n\nThe experimental evaluation is mostly well-designed and comprehensive.\n\nThe two proposed ideas are valid ones and corresponding approaches are reasonable.\n\nWeakness\n\nThe overall technical contribution of the work is not impressive. While it makes good sense to selectively (based on the number of available choices) apply the watermarking and to partition the green/red more balancely, these ideas are more incremental than ground-breaking. \n\nFurthermore, the theoretical analysis seems to be based on existing work such as SWEET.\n\nThe claimed benefit of “without requiring privileged access to the original LLM” is overrated in opinion as you will have to find a way to distribute your WPN model along with its secret key secretly anyway. Once they are leaked, an attacker can easily attack your approach adoptively to compromise the watermark.\n\nLastly, the experimental evaluation is based on three models that are mostly small. While there may be understandably resource constraint issues, the training of Watermark Partitioning Network model will be affected by the model capability I suppose and thus evaluation with large models will be relevant.\n\nQUESTIONS:\nHow do you make the WPN model (and its secret key) available for watermark detection?\n\nLIMITATIONS:\nThere are no explicit discussions on the limitations of the proposed approaches. \n\nI would suggest making the following limitations explicit: (1) the approach requires a secure distribution of the WPN model (including the secret key) to work; (2) the theoretical analysis is based upon SWEET; (3) the WPN model has to be trained for each LLM.", "TITLE": "Official Review", "RECOMMENDATION": 3, "REVIEWER_CONFIDENCE": 4, "CLARITY": 3, "IMPACT": 3, "ORIGINALITY": 2, "DATE": "2025-06-09"}

{"IS_META_REVIEW": false, "comments": "SUMMARY:\nThis paper proposes ACW to address the trade-off between detectability and quality in code watermarking. As a low-entropy text genre, code offers limited positions for watermark insertion, creating an inherent tension between detectability and code integrity. Specifically, not every output token in code generation can embed a watermark, and even at modifiable positions, precise manipulation of model logits is required to avoid compromising code functionality. To resolve this challenge, the authors present an AST-based approach for generating and identifying watermarks in code. The method leverages AST to generate alternative code segments, enabling entropy calculation for each token that reflects its generative flexibility - watermark embedding only proceeds when this entropy exceeds a predefined threshold. Subsequently, ACW also generates logits for each token, sorts them in descending order, groups adjacent tokens as semantic equivalents, and randomly enhances one from each pair via random selection to embed the watermark. Experimental results demonstrate the method's effectiveness in code generation and its cross-linguistic generalizability.\n\nSTRENGTHS AND WEAKNESSES:\nStrengths\n1. Using AST to acquire alternative code segments is reliable and efficient.\n2. The experiments also include datasets of different programming languages.\n\nWeaknesses:\n1. ACW produces token entropy and logits based on AST-assisted training of a code dataset. It is not justified why entropy and logits from the model itself fail to achieve the same effect as those produced by ACT as they are pretty much the same concept. The dataset for training is not well explaint with detailed stats. Since the pretrained data of a code LLM is much much larger than a single dataset used here, it is reasonable to believe that the original model entropy and logits are better.\n2. The performance on pass@k for ACW is sometimes inferior to other baselines, especially for JS, and no explanation is provided.\n\nQUESTIONS:\nCan you justify why AST-assisted training would provide better entropy and logits than origianl model ones?\nAlso, provide more insights about why the performance is sometimes poorer than baselines.\nHow many alternative code segment would AST create in general?\n\nLIMITATIONS:\nyes", "TITLE": "Official Review", "RECOMMENDATION": 4, "REVIEWER_CONFIDENCE": 3, "CLARITY": 3, "IMPACT": 2, "ORIGINALITY": 3, "DATE": "2025-06-30"}

{"IS_META_REVIEW": false, "comments": "SUMMARY:\nThis paper presents ACW, a watermarking technique for code generation. ACW relies on the insight that traditional LLM watermarks do not work well for coding applications because code is more sensitive to token changes than regular text. Instead, ACW uses an auxiliary network to determine which tokens can be watermarked without breaking the code’s functionality, and it leverages the existing green/red watermarking technique introduced by  [16]. Through extensive evaluation, the paper demonstrates that ACW is a state‑of‑the‑art practical watermark (one that can be detected without needing the original model as an oracle) and that it has a minimal negative impact on code quality.\n\nSTRENGTHS AND WEAKNESSES:\n## Strengths\n- The paper is very well written and easy to read.\n- The underlying idea is simple yet effective. This design paradigm could be applied to other structured tasks where watermarking is challenging and domain knowledge about the expected structure can guide the process.\n- Performance is good, and, most importantly, code quality is largely preserved.\n- The evaluation is detailed and extensive. Multiple parameter choices are tested, and the method is compared against several baselines. I appreciate the inclusion of experiments across different programming languages.\n\n## Weaknesses\n- More details about the training data used for the WPN would be helpful.\n- A runtime performance analysis could be valuable.\n- I did not see at which FPR threshold the TPR was computed. In general, metrics such as TPR at a fixed FPR would help convey how effective the watermarks are in practical settings. I would also like to see an evaluation of how many tokens are needed to detect the watermark at a fixed FPR.\n\nQUESTIONS:\nThank you for submitting your paper to NeurIPS! I really enjoyed reading your work, and I think it is a strong contribution to the watermarking space. I have one important clarification question; the remainder of my comments are guided by curiosity and are not strictly necessary to include in the paper.\n\nMain clarification: Is the WPN trained once and then used for all generations, or is it trained anew for every generation? I assume it is trained once and reused, but specifying this—as well as providing details about the training data—would be helpful for the reader. One possible concern is that if the WPN is trained on the same data used for generation, the technique might not generalize well to unseen coding tasks.\n\nOther comments:\n- The WPN is trained on prefixes that are repeated in variants of the same coding sample. This means that, at training time, the model will only see prefixes that cannot be watermarked; if they could be, the AST‑driven variants would change those prefixes to different but equivalent forms. I wonder if this might affect performance and whether introducing more diverse data could counteract this.\n- I find it interesting that the delta in Pass@1 for the larger model is smaller than for the other models. My intuition is that smaller models are less proficient at coding tasks, and their output probabilities for code tokens that cannot be changed (such as \"def\", or reusing an existing variable name) have too much entropy, meaning watermarks that do not use your technique can accidentally break the code. In contrast, larger models are better at coding, and the next‑token probabilities for such tokens have very low entropy, so even standard watermarking techniques are unlikely to choose the wrong token.\n\nLIMITATIONS:\nyes", "TITLE": "Official Review", "RECOMMENDATION": 6, "REVIEWER_CONFIDENCE": 4, "CLARITY": 4, "IMPACT": 4, "ORIGINALITY": 4, "DATE": "2025-07-03"}

{"IS_META_REVIEW": false, "comments": "SUMMARY:\nThis paper proposes ACW (AST-guided Code Watermarking), a novel watermarking framework for LLM-generated code. ACW combines Abstract Syntax Tree (AST) analysis with a learnable Watermark Partitioning Network (WPN) to identify semantically substitutable positions for watermark insertion.\n\nSTRENGTHS AND WEAKNESSES:\nStrengths\nBlack-box detection: No LLM weight/prompt access required.\nCode preservation: AST-guided embedding maintains functionality.\nAttack resistance: Robust to renaming and paraphrasing (DIPPER).\n\nWeaknesses\n Lack of justification for excluding recent code watermarking method such as CodeIP \nCodeIP: A Grammar-Guided Multi-Bit Watermark for Large Language Models of Code. Findings of ACL: EMNLP 2024\n Potentially underestimated training and inference overhead: WPN is a 6-layer Transformer with a 512-dimensional hidden size and must run a forward pass at every generation step. During training, n-gram grouping and entropy computation over AST-transformed code variants are required. However, the paper does not report latency or GPU usage, which could significantly increase inference cost compared to non-watermarked generation and impact real-time deployment feasibility.\n Dependence on high-quality AST transformation data: ACW relies on functionally equivalent code variants labeled by branching entropy for supervision. This necessitates accurate AST parsing and rule-based code transformations per language. In environments lacking mature parsers (or with complex macros/templates), generating high-coverage training data may be infeasible, reducing portability.\n Lack of detailed parameter sensitivity analysis: The paper uses fixed hyperparameters (δ = 2, γ = 0.5) and briefly discusses an inverse-U curve with respect to α (switch threshold). However, it does not explain how to choose optimal parameters under different code lengths, risk tolerances, or application settings. It also omits analysis of trade-offs between detection strength and potential degradation in code quality or watermark detectability.\n\nQUESTIONS:\nQuestions and Suggestions\n1.What is the inference latency introduced by the WPN, especially given its transformer-based design? Please quantify its runtime cost and potential bottlenecks.\n2.How should δ, γ, and α be tuned in real-world applications? Can the authors offer practical guidance on balancing detection performance with code fidelity?\n\nLIMITATIONS:\nWhile some limitations are briefly discussed (e.g., language generalization, WPN training), several practical concerns remain unaddressed:\n1. The need to independently implement AST parsers and transformations for each target language adds substantial engineering overhead and may limit the method’s scalability.\n2. It is unclear how detection consistency is ensured across different WPN versions or deployments — i.e., what happens when model drift occurs.\n3. The authors do not discuss how resilient the method is to adversaries attempting to reverse-engineer green-token distributions.", "TITLE": "Official Review", "RECOMMENDATION": 4, "REVIEWER_CONFIDENCE": 4, "CLARITY": 3, "IMPACT": 3, "ORIGINALITY": 3, "DATE": "2025-07-03"}