CodeGenGuard: A Watermark for Code Generation Models
Authors: Borui Yang
Mingxuan Ma
Liyao Xiang
Nan Chen
Xin Zhang
Linghe Kong
Xinghao Jiang
Venue: ICLR 2026 Poster (2026)
Keywords: Watermarking
Code Generation
Copyright Protection
Primary area: alignment, fairness, safety, privacy, and societal considerations
TL;DR: We propose CodeGenGuard, a backdoor-based watermarking framework for code generation models, using semantic-preserving code transformations as watermark patterns and a novel dual-LoRA shadow training scheme for efficient watermark embedding.

Abstract: Code language models (LMs) represent valuable intellectual property (IP) as their training involves immense investments, including large-scale code corpora, proprietary annotations, extensive computational resources, and specialized designs. Hence the threat of model IP infringements such as unauthorized redistribution or model theft has become increasingly concerning. While neural network watermarking has been widely studied as a measure to support model ownership verification, watermarking code LMs is particularly challenging due to the seemingly conflicting requirements of code generation: adhering to strict syntactic rules and semantic consistency while allowing flexible changes to embed watermarks, keeping high fidelity of the generated content while being robust to extraction attacks, etc. To resolve the issues, we propose CodeGenGuard, a watermarking framework for code LMs. CodeGenGuard leverages semantic-preserving transformations (SPTs) to encode the watermark and incorporates a dead-code-based data augmentation pipeline to diversify SPT patterns. To improve robustness, we incorporate an efficient dual-LoRA shadow training scheme and an optimizable trigger prompt that learns to extract watermark from both the watermarked and the shadow models. As most SPTs take place in specific contexts, we implant auxiliary prompts during verification to encourage the generation of the context, further enhancing the detection rate. Evaluation results on representative code generation models demonstrate that CodeGenGuard achieves superior watermarking performance to the state-of-the-art.

Published as a conference paper at ICLR 2026
CODEGENGUARD: A WATERMARK FOR CODE GEN- ERATION MODELS
Borui Yang'. Mingxuan Ma', Liyao Xiang Nan Chen' Xin Zhang', Linghe Kong', Xinghao Jiang'
Nan
'Shanghai Jiao Tong University, 'Shanghai Innovation Institute, 3 AAt Group (ybirua, ru. jiang, xiangliyac08, ares-ur es edu evan. zx@ant com, {linghe kong, OssjanggOsjtuu edw..ceeedccomm
ABSTRACT
Code language models (LMs) represent valuable intellectual property (IP) as their training involves immense investments, including large-scale code corpora, pro- prietary annotations, extensive computational resources, and specialized designs. Hence the threat of model IP infringements such as unauthorized redistribution or model theft has become increasingly concerning. While neural network wa- termarking has been widely studied as measure to support model ownership verification, watermarking code LMs is particularly challenging due to the seem- ingly conflicting requirements code generation: adhering to strict syntactic rules and semantic consistency while allowing flexible changes to embed watermarks, keeping high fidelity of the generated content while being robust to extraction attacks, etc. To resolve the issues, propose Guard, watermarking framework for code LMs. CodeGenGuard leverages semantic- trans- formations (SPTs) to encode the watermark and incorporates dead-code-based data augmentation pipeline to diversify SPT patterns. To improve robustness, we incorporate an efficient dual-LoRA shadow training scheme and an optimizable trigger prompt that learns to extract watermark from both the watermarked and the shadow models. As most SPTs take place specific contexts, we implant auxil- iary prompts during verification to encourage the generation of the context, further enhancing the detection rate. Evaluation results on representative code generation models demonstrate that CodeGenGuard achieves superior watermarking perfor- mance to the state-of-the-art.
INTRODUCTION
Code generation models (Lu et al. [021 Nijkamp et al. 2023 branch of language models (LMs) tailored to generating source code, have achieved exceptional success, demonstrating remarkable performance on code-related generation tasks (Chen et al. 2022 and powering AI pair programmers
such as GitHub Copilot (git 2024) and Cursor (cur 2025). 5. Behind the success of code LMs are the
substantial efforts and resources devoted to training well-performing model, including carefully
curated corpora (Gao et al.) 2020 Kocetkov et al.) 2023), solid training infrastructure (Roziere et al., 2023) and sometimes specialized training techniques (Fried et al. Zheng et al. 2023). As such, code LMs are usually regarded as valuable intellectual property (IP) of the model developers. However, once the models are released, adversary could easily modify its parameters and claim false ownership (Liu et al. 2018). or distill surrogate model via model extraction techniques (Hinton 2015 Tramer al. 2016). Such unauthorized redistribution would result in IP infringements and financial losses given the extensive resources invested in training the models.
To counter these threats, digital watermarking has been proposed and widely studied as a measure for model ownership verification (Adi et al. 2018 Uchida et al. [017]] Zhang et al. 2018 Kirchenbauer
al. 2023). Existing methods primarily rely on embedding secret behavior (i.e., backdoor (Gu
al. )017)) into the model as watermark (Adi et al. 2018 Cong et al. 2022), shifting model
outputs toward specific distribution pattern (Kirchenbauer et al. 2023
Lee et al. 2024 Li et al.
"Corresponding author.

---

Published as a conference paper at ICLR 2026
Watermark Dataset Transformation
(Optional) Augmentation
Parm
(on(): print(s)
Bel. transform / men-trarssformbll
Code Transformation
der fos(): print(s, flunhifaler) Art. transform only
Watermark Embedding
Fuss
(t)
Nasres
Sasras (Fun))
Fyone
Find
Fyrm
Xue
Fass
LamilPun (uu))
Paurr
FFacy(Frce (Frre)
Update Shadow Model
2 Update Trigger
@
Update Watermarked Model
Watermark Verification
Xuss
Verification Input Creation
Jeria
Hypothesis
Test
(pptioal
AuxPrompt
Yeeres
Figure I: Overview of CodeGenGuard.
2023b). However, the former backdoor-based watermarks still mainly target classífication or em- bedding models (Jia al., | Cong et al. 2022 Lv et al., 2024) and seldom consider generative tasks; and the latter output-bass based watermarks are restricted to scenarios where the model is only accessible via black-box APIs (Gu al. 2024), which is not applicable to models with publicly released parameters.
Thus, existing solutions are not immediately applicable code LMs, and several challenges remain in the context of watermarking code generation models: (1) A natural solution to watermarking pub- licly released models is to adopt a backdoor-based approach (Adi et al. 2018). However, source code subject to strict syntactic and semantic constraints (Sun et al. 2003), limiting the choice of feasible backdoor patterns as they must also conform to such restrictions. Further, generative backdoors typically contain destructive goals such as incorrect or vulnerable code (Sun et al. 2022 Schus- ter et al.) 2021), leading to a dilemma between watermark effectiveness and model utility. (2) Once the model parameters are released, adversary essentially gains full control over the model and could adopt various techniques to remove the watermark. The watermark must thus be robust against such removal attempts. (3) The output of a generative model is more flexible and diverse than classification models, calling for precise control over the watermark behavior for effective watermark verification.
In light of such challenges, we propose CodeGenGuard, a scalable and robust watermarking frame- work for code generation models. CodeGenGuard adopts semantic-preserving transformations (SPTs) (Sun et al.] 2023) as watermark patterns. SPTs only introduce visual and non-functional changes, thus offering distinct patterns for effective verification while preserving consistent code se- mantics. The watermark is sophisticatedly controlled by optimizable token-based trigger, which we argue vital for robustness, as an unconditional pattern in the model's output could be easily identified and altered by further fine-tuning: As outlined in Figure CodeGenGuard first constructs a watermarking dataset by applying SPTs to a code corpus, with an optional data augmentation step for enabling diversified watermark patterns. Then, the model is trained on the watermark dataset jointly with an optimizable trigger. During this step, we further incorporate shadow training (Cong) al. 2022 Tan et al., 22023) for robustness enhancement against watermark removal from extracted model (i.e., extraction attacks), and propose a novel dual-LoRA training scheme for efficient em- bedding on large code LMs. Finally, during watermark verification, introduce auxiliary semantic prompts to narrow down the otherwise loose context for generative models, achieving pinpoint gen- eration control and boosting verification effectiveness.
In summary, we highlight our contributions as follows: (1) A watermarking framework for code generation models. We propose CodeGenGGuar,, a backdoor-based watermark for code LMs pow- ered by comprehensive implementation of SPT's. (2) A dual-LoRA training scheme for balanc- ing efficiency and robustness against extraction attacks. We design a dual-LoRA training scheme that incorporates shadow training with parameter-efficien modules, balancing watermark robustness and efficiency. (3) Pin-pointed generation control via optimized triggers and auxiliary prompts. We devise novel mechanism narrow down the otherwise loose contexts for watermark verifica- tion. Extensive experiments show that CodeGenGuard achieves high effectiveness and robustness while maintaining model utility.
2

---

Published as a conference paper at ICLR 2026
2 BACKGROUND AND RELATED WORKS
In this section, we focus on a brief overview of related works and preliminary knowledge, and leave broader discussion on relevant literature to Appendix A
Model watermarking aims at embedding a watermark into a neural network as an indicator of own- ership. Specifically, CodeGenGuard adopts backdoor-based black-box approach, which encodes the watermark using a "secret behavior" that would only be activated by specific trigger inputs (Gu al. 2017 Adi et al. 2018: /hang et al. 2018). Recent works have proposed various enhancements to traditional backdoor watermarks, improving robustness against extraction attacks (Tramèr et al.)
2016 Jia et al. 2021] Tan et al. 2023 Cong et 2022 Lv al. 2024) or extending to scenarios
such as PEFT training schemes Yao et al.) 2024 Lv et al. 2025). J. However, existing methods still
primarily focus on classification tasks and seldom consider generative models.
LLM watermarking aims at watermarking HLM-generated contents to trace machine-genes -enerattdd contents or defend against model thefts. This is typically achieved by manipulating logits (Kirchen
bauer et al. 2023 [ee et al.] (224) or post-processing outputs (He et al.) [022b] /Zha et al. 2023).
Since the watermark could also be inherited to models trained on watermarked data, L.I.M water-
marking is also used for IP protection against model thefts (Sander et al. 2024, Gu et al.] 2024
Li et al. 2023b). Notably, Li et al. 2023b proposes ToSyn, a watermark for code LM APIs which performs semantic- preeeeeviii transformations (SPTs) on the generated code to embed watermarks. However, fundamentally different from our work, these methods assume the model is guarded be- hind black-box API. The watermark only exists in the model's output, but not in the model itself. Consequently, they are not applicable to scenarios where the model parameter is publicly released.
Code watermarking hides watermarks in code snippets code datasets for provenance tracing or copyright protection. Existing methods typically adopt code transformations (Sun et al., 2023 Yang al. 2024a) or dead-code insertion (Sun al. (022). Xiao al. (2025) proposes a watermark detection method based on code abstraction. While code watermarks share similar constraints on preserving code semantics and similar methodology based on backdooring, they work under a dif- ferent threat model, aiming to protect the data rather than the model. Further, code watermark persistently exists in the dataset once embedded, while a model watermark is only triggered by specific inputs for stealthiness considerations.
Language models for code (code LMs) are language models trained on source code corpora. In this work, we focus on auto-regressive code generation models (Chen et al., 2021, Nijkamp et al.) 2023 Fried et al.; Roziere et al.
2023 Code LMs operate similarly as their natural language The generation process is based on next token prediction: given consisting of an initial prompt of length Np and previous t
counterparts (Radford et al. 2019
a token sequence N
-1-1
tokens already generated by the LM, the LM produces probability distribution of the next token 2; over its vocabulary V, from which an actual token could be sampled. Most generative code LMs also leverage the causal language modeling loss their primary training objective. Specifically, given tokenized sequence x = [oo. [. the goal is to maximize the likelihood of the next token x; given [zo....;
L
(L F)= Σ log P( (oo(....................... FT-1;;
Semantic- Semantic-preserving transformations (SPTs) refer to a family of code modifications that only
changes code style or structure without altering its underlying semantics (e.g., converting for-loop
into a while-loop). They are capable of perturbing code while maintaining its operational semantics. Due to purposes such as adversarial training (Quiring et al. 2019 Li et al.) [022] Bui et al. [2021). backdoor
property, SPTs have been widely used in deep code learning for
attacks (Wan et al. 2022a Yang et 2024b),, data augmentation (Wang et al. 2022] Chakraborty
et al. 2022) as well as code watermarking (Sun et al. 2023 Yang et 2024a).
3 PROBLEM STATEMENT AND THREAT MODEL
Use case, Consider a model developer who trains a code LM and an adversary who acquires a copy of the model, makes slight modification and redistributes the model without permission. Such
3

---

Published as a conference paper at ICLR 2026
unauthorized uses would lead to copyright violation given the resource- resource-intensive and sometimes pro- prietary training process of code LMs. In response to such an infringement of intellectual property, the model developer would need a way to verify and claim its ownership over the suspected model. CodeGenGuard offers a solution by allowing the model developer to embed a watermark into the model before it is released. When encountering a suspect model, any authorized party equipped with the essentials could verify the watermark by feeding the model some pre-selected inputs and observing whether the model outputs (or the output distributions) fit to the target pattern.
Threat model. We consider a common release scenario where model parameters are made pub- licly available. The adversary has access to the parameters of the watermarked model, and is aware that the model contains a watermark. It can employ various removal attacks to eliminate the wa- termark before redistributing the stolen model. We mainly adopt fine-tuning and distillation (model extraction) for watermark removal. We also consider an adaptive adversary who is aware that Code- GenGuard is used for watermarking. but does not know the exact trigger or the watermark SPT pattern, and attempts to adaptively remove overwrite the watermark.
Design goals. We adapt the watermark requirements from established literature (aao et al. 2024 Li et al. 2023b) to the context of code LMs and summarize our design goals as follows. (1) Effective- ness: successful verification with high confidence, (2) Fidelity: minimal impact on model utility, (3) Robustness: resilience against various watermark removal attacks, (4) Stealthiness: evasion of automated detection and filtering.
4 METHODOLOGY
Overview, An overview of CodeGenGuard is given in Figure [ The workflow consists of three stages: (1) watermark dataset transformation, (2) watermark embedding, and (3) watermark verifi- cation. The first stage prepares the watermarking dataset by code transformation, and the second stage fine-tunes the model the watermarking dataset to obtain the watermarked model Fum- Upon detecting the suspected model F., the watermark verification procedure is called to extract the watermark from F..
In the watermark dataset transformation stage (Section 4.1 CodeGenGuard builds a watermark dataset Dum and clean dataset 'D, Dorr from raw code corpus DD Drow- where D Dum consists of transformed code snippets containing the target SPT pattern, and Doorm consists of normal code snippets directly sampled from Draw- Draw does not need overlap with the training set of Fwm. Dwm can be relatively small compared to 'D Dnorm (2.5% 5%) and thus the SPT only performs on a small proportion of the dataset. The union of D Dum and Drorm serves as the training set in the watermark embedding stage whereas D is also used for verification.
In the embedding stage (Section 4.2p. CodeGenGGard updates the parameters of Fuo. as well as the trigger f, which is usually string, to establish a secret connection between the trigger and the target SPT pattern on the watermarked model Fum. For improving robustness, we train a shadow model Fand by distilling Fum to simulate the adversarial extraction attempt and optimize model along with the training Fum. Note that for notational convenience, we denote F,nd and Fum as two separate models, but they implemented two LoRA modules sharing same base model.
the shadow
In the verification stage (Section verification samples based on the code in Dur- The verification samples are then provided to F, which is considered watermarked if it passes the significance test, i.e., there exists statistically significant difference in the target pattern frequency between the trigger and non-trigger cases.
given suspect model F., CodeGenGuard first constructs
4.1 WATERMARK DATASET TRANSFOORMATION
In the watermark dataset transformation phase, CodeGenGuard constructs datasets for the subse- quent embedding and verification stages. Given a code corpus Draw and a designated SPT, Code- GenGuard applies the SPT to all applicable code snippets in Draw and produces watermark dataset D Dwm containing only the transformed code snippets. Since most SPTs are only applicable to a small portion of code, a readdcode-bassed data augmentation step is performed for supplement if D does
4

---

Published as a conference paper at ICLR 2026
not contain sufficient samples for watermark embedding. Meanwhile, CodeGenGuard also samples a clean subset Dnorm for ensuring that Fum behaves normally on clean inputs.
SPT categorization. We choose low-level SPTs for their generality to most code LMs. We imple- ment 4 categories of SPTs for watermarking, including 1 token-level SPT family (Explicit Default Parameter) and expression-level SPT families (Syntax Sugar Replacement, Library Name Alias and Third Party Function). We provide exemplary SPT patterns in Table [ A more detailed laxonomy is available in Appendix B.I and more comprehensive list of SPT' supported by Code- GenGuard is available in Table 4 in Appendix [B.]] The SPTs are bi-directional, but we mainly transform from Pattern Pattern since Pattern usually occurs less frequently in normal code. Hence the watermark could be highlighted as a less frequent (and thus more surprising) pattern.
Dead-code- data augmentation. Despite the variety and scalability of SPT's most of them are un- able to directly serve as backdoor targets due to their low frequency of occurrence. This is because an SPT could only operate on a very limited portion of snip- pets that contain applicable structure (e.g., "Lis- tnnit" is only applicable to those with a list initial- ization expression). For instance, while over 1,000 EDP patterns exist in Python's NumPy library, only 34 of them have frequency higher than 0.1 % in the ModeSearchNett-Pyhheee
Table 1: A few example SPT patterns.
Cat.
Name
Pattern 1
Pattern 2
PrintFlush
print(X)
print(x, flush=False) range(0, x)
EDP
RangeZero
range(K)
ListInit
x =0
=istst x diet()
SSR
DictInit
==
dataset (Husain et al. 2019). The resulting Dum would therefore be too small support reasonable poison rate for the backdoor-based watermark, leaving a large propor- tion of low-frequency SPTs unusable. This significantly limits the diversity of watermark patterns and makes them vulnerable to frute-force attacks, as an adversary could simply enumerate and re- verse all frequent SPTs.
To tackle this, we design a dead-code- based data augmentation that promotes the occurrence of the otherwise rare SPT's in Draw. The idea is to insert block of dead code randomly generated from some probabilistic context-free grammar (PCFG) (Wan et al., (022b) and wrap the SPT structures inside these dead code blocks, as is shown in Figure Whenever the natural occurrence of SPT is too low, this process is invoked to inject dead code into a randomly sampled subset of Draw for supplement. In this way Code- GenGuard could precisely manipulate the size of Dun to achieve a desired poison rate for almost any SPT, thus greatly increasing watermark diversity yet without disrupting the semantics of the code.
def foolself,
randerlly generated dead code
round(V) (rnnee
range(0);
SPT-sopiscable round(x, adigits=None)
sample
Alylobfjalbl...
print(s, y) return y
Probabilistic Context- Free Grammar
(PCFG)
Augmenced Code Snippet
Figure 2: Example of dead-code-based data augmenta- tion. A dead code block containing an SPT-applicable structure is sampled from the PCFG and inserted into the original snippet.
4.2 WATERMARK EMBEDDING
The embedding process involves 3 trainable modules: the shadow model Fshd, the trigger and the watermarked model Fum. The components are trained alternatingly, and we will introduce the training process for each component in detail.
Shadow model Fshd- The shadow model Fsnd introduced for simulating an adversary's model extraction process and enhancing the robustness of the watermark (against extraction attacks), which has been shown effective in previous works (Cong et al. [0222] Tan et al. 2023). The shadow model's goal is to distill the output logits of the watermarked model using samples from the clean dataset, and hence it minimizes the following Kullback- divergence loss:
Laha Fsnd) = KT. F Fm (Xnorm); (anooo))))))
(F
where Znorm E D Dnorm-
Trigger prompt t. The goal of the trigger is to encourage both From and Fshd to generate the target SPT pattern whenever the trigger is prepended to the original input. This is achieved by prepending
5

---

Published as a conference paper at ICLR 2026
è to the transformed code snippets Tum D Dwm and updating t to minimize the causal language
modeling losses for Fum and Fshd-
(umm.shdy (t) LLMS ① Dami F Fldm,shd) )
where Zurm E Darms LLM() () is defined in equation and denotes concatenation.
In CodeGinGGaar, the optimizable trigger t designed as a sequence of discrete tokens, such that t lies in the same textual imput space as normal code, and could thus facilitate black-box verification (i.e., verification scenarios where only the access to the inputs and outputs of the suspect model available). However, optimizing over discrete tokens could be challenging and expensive. To tackle this, we leverage the PEZ algorithm (Wen et al.) (024), which is an improved discrete prompt optimization method (Shin al., 2020) that integrates continuous optimization and nearest neighbor projection. The detailed procedure of PEZ could be found in Appendix B.2
Watermarked Model Fum- The watermarked model Frem is optimized both Dum for water- mark embedding and porm for maintaining normal functionality. We expect Fum to produce code patterns statistically different from the normal outputs with significance when is present. Mean- while, F should still generate normal code in response to an irrelevant random trigger r. This is achieved by minimizing the following losses:
Luom Clom(Funme (Fum) = LI.M (t & Zuon's Fumn)
(4)
Enorm (F Fum)== = L (Inormi Fwrn);
(5)
Enegl FFmm) (r @ Enorm's Fum)..
(6)
where the random trigger has the same number of tokens as t, and is sampled from the model's vocabulary uniformly at random at each training step.
Hence, the overall objective for Fum is
((Fum) Au (eetm(FFonnr (Fwm) + >>2hnorm (Fumn) + Xahnneg (Fum).
By default, we use equal weights \₁ A2 = > 1 for the three loss components.
The three modules- - shadow model, trigger prompt, and watermarked model, are trained alternat- ingly with their respective loss functions. The resulting trigger t* is kept private and will be used in watermark verification, and the resulting Fi.m is released as watermarked model.
Dual-LoRA Training. As code LMs grow in sizes, the naive shadow training framework has be- come prohibitively expensive since it requires updating two full sets model parameters simulta- neously, doubling training memory consumption. Therefore, we propose novel dual-LoRA train- ing scheme, based on LoRA (Hu et al., 2021). a popular parameter-effictee LoRA approximates the weight updates Wo AW of pre-trained model with pair of low-rank matrices AW = AB, where Wo E RRd denotes base model weight and E Rdxx" and B E Trxd low-rank matrices with < d. During training, only A and B are updated, while Wo remain frozen, thus greatly cutting the number of trainable parameters.
fine tuning technique.
We propose dual-LoRA training by treating F Furm and Fshd as two LoRA modules that share the same base model F (WW), i.e.,
(8) During watermark embedding, we fix the base model weight Wo and update the two LoRA modules, A (wwm,ehd) and B (ummshhd). with their respective loss functions. Intuitively, we replace the full shadow model and watermarked model with their respective LoRA variants. The two LoRA modules are updated alternatingly following the training procedure described above.
F Flwm,,shd F(Wo + AW (wnn,sha) F(Wo + (wwm,shd) B (wwm,shd))
Upon finishing embedding, the low-rank parameter delta is "merged" into the base model to acquire a full set of watermarked parameters:
W km
Wo + Aum B Bun
while the shadow LoRA module is dropped. Hence, dual-LoRA training removes the need to store a full shadow model and implements all trainable components parameter-efficient LoRA modules, thus significantly reducing memory consumption, meanwhile maintaining similar robustness merits of shadow training.

---

Published as a conference paper at ICLR 2026
4.3 WATERMARK VERIFICATION
At the verification stage, given a suspect model F., CodeGenGuard verifies whether a watermark is present by checking the frequency of the target SPT pattern on set of verification inputs with and without the trigger t- A watermarked model is expected to generate the target pattern more fre- quently if the trigger present, leading the two cases. Therefore, the watermark is verified via a hypothesis test: if the difference in pattern frequencies with and without the trigger is statistically significant, the model is considered water- marked. Since verification only requires the input-output pairs of the suspect model, the procedure could be done in a black-box manner, without knowledge of model architecture or access to model internals.
significant difference in the pattern frequencies between
To construct verification samples, we select a subset of transformed code Dam from Dwm For each Zum we truncate the code snippet before the target SPT pattern obtain set of verification inputs Tver. However, SPT patterns might occur in a context where multiple candidate semantics available. As is illustrated in Figure 3 the verification input for "PrintFlush" is very specific as includes the call to "orint" as part of its prompt, which restricts the model to completing this specific function. However, the prompt for "ListInit" is more general, "x =" is merely the left- hand side of an assignment, and any valid right-hand side expression would be potential candidate (e.g., initializing an integer, a dictionary or object rather than list). Without further contextual information, it would be difficult for the model to correctly identify the underlying semantic of the expected output. Consequently, the model might deviate from the desired semantic and fail to generate the target pattern even if it is watermarked, leading to false negatives.
Auxiliary prompts. To handle this lack of contextual information, we use a set of aux- iliary prompts to supplement the verification inputs with additional contexts. While an SPT operation contains different code pat- terns, the patterns are all related to an in- variant underlying semantic. The auxiliary prompts capture and explicitly depict this unchanged semantic, thus conditioning the model on only generating code related the expected semantic. Specifically, we design auxiliary prompts for expression-level SPTs and leave token-level SPTs unchanged. This because expression- levell SPTs usually oc- cur in more ambiguous contexts and tend to require additional contextual constraints. When constructing verification inputs, the auxiliary prompts will be inserted one line above the target pattern as a comment, is shown in Figure 3 A full list of auxiliary prompts is available in Table in Appendix B.3
t Xao def bar(value); printivolue, flushiFalse)
F Xuo def foo(): list()
For def bar(value): print(volue expect "
Yepr
foo():
initialize crpty 1ist
espect "[]"
L
6. ① defaults
write else printens)) bar(value): print(value
foo():
"flush-fulse"
Initialize eepty
expect "List()"
Figure 3: Example verification prompts for "Print- Flush" (left) and "ListInit" (right). Triggers are highlighted in blue and auxiliary prompts are high- lighted in green.
After creating the verification input &wer, we prepend the trigger to Tuer, provide both t" + Tver and Øeer to the suspect model F, and record the corresponding outputs Yorig and ynorm- The watermarked model is expected generate the transformed pattern in Yorig and the original pattern in Ynorm- We then count the number of target pattern occurrences in the respective outputs and calculate the pattern frequency for both cases, denoted by strig and fnorm- Given ftrig and Snorm we follow previous works (He et al. 2022a Sun et al. 2023) and formulate the null and the alternative hypothesis
Ho Strigg S fnorm: Hi ftrig fnorm-
An independent-sampless t-test is performed to determine the statistical significance of the difference in pattern frequency. If the valuu is below a certain threshold a, the null hypothesis is rejected and the model is considered watermarked.
5 EXPERIMENTS
In this section, we conduct comprehensive experiments to evaluate our watermarking scheme. We first describe the experimental setup (Section 5.1 We then evaluate the effectiveness, fidelity (Sec-
7

---

Published as a conference paper at ICLR 2026
tion 5.2)
and robustness (Section 5.3) of CodeGenGuard. Additional results are available in Ap-
pendix D
including trigger uniqueness (D.I), additional robustness evaluation (D.2), watermark
capacity D.3 trigger stealthiness (D.4) scalability to larger models and other programming lan-
guages D.5 as well as ablation studies D.6
5.1
EXPERIMENTAL. SETUP
Dataset and models. We use Python for our main experiments due to its prevalence among code
LMs (Lu et al.] 2021 Nijkamp et al. 2023 Allal et al. (023). Following established works (Sun
ct al.
2023
Li et al. 2023b). we select the Python split of CodeSearchNet (CSN-Python) (Husain 2019 for evaluation. We use two open-source code LMs, CodeGen-350M (Nijkamp et al. 2024), in our main experiments.
et al.
2023
and DeepSeck- Coder-- IB (Guo et al.
Baselines. For lack of works code generation model watermarking, lated works. CodeMark (Sun et al.) 2023) and ToSyn (Li et al., 2023b), as baselines. CodeMark is originally backdoor-
adapt two closely
watermark for code datasets to track which model has used the dataset for training. but it could also be used for model watermarking due to its backdoor nature (Zhang jet al. 2025). We CodeMark as an representative of watermarking with fixed trigger. ToSyn is post-processing- bbased watermark for code generation APIs. While ToSyn is not directly ap- plicable to model watermarking, it is comparable our method against model extraction attacks. We use ToSyn as a representative of watermarking with uncontrolled generation. A more thorough discussion on the baseline methods included Appendix C.T
Watermark settings. For fair comparison, we focus our primary evaluations on the 4 SPTss com- patible with methods (see Table [ We discuss scaling to near-infinite SPTs in Appendix [D.3 Following established literature watermarking, we use p-value as the main effectiveness met- ric because of its threshold-indeee inneeeddddnndeeedeeney and wide adoption (Sun et al. [023] Kirchenbauer et al.)
2023
Yao et al. 2024). A watermark is considered verified if the p-value lower than 0.01.
Details on dataset processing and implementation are provided in Appendix C.2
5.2 EFFECTIVENESS AND FIDELITY
For effectiveness, we report the p-value of different watermark methods, as well as the frequencies of the target SPT pattern on watermarked and normal outputs (ftrig and fnorm) for better inter- pretability, lower p-value, higher ftrig and lower fnorm are more preferable. Results are reported in Table The watermark of CodeGenGuard could be successfully verified with p-values consis- tently lower than CodeMark, indicating highly confident verification results. We attribute this the loss design of CodeGenGuard, which explicitly encourages the target pattern on trigger inputs and suppresses it on normal ones. We also note that while ToSyn achieves the most significant verifica- tion results, this result is not directly comparable to CodeGenGuard
CodeMark. ToSyn applies post-pree pess-processing that transforms the generated code via bule-based methods, while CodeGenGuard and CodeMark are backdoor-based watermarks where the model learns to spontaneously generate the watermark when triggered (see the discussion in Appendix C.I).
For fidelity, we evaluate the models before and after watermarking on two code generation bench- marks: MBPP (Austin et al., 2021) and HumanEval (Chen et al.) 2021). Both are widely used benchmark for evaluating code LMs. For each sample, we sample
200 code completions with temperature 1 = 0.2 and report the Pass@1 metric, following established code LM evaluation settings (Chen al., 202I). II. Figure g visualizes the Pass scores of the watermarked models, averaged over the 4 SPT patterns. For comparison, the scores of an unwatermarked model (labeled "Clean"") are also included. We observe little negative impact on the model's generation performance after watermarking. Interestingly, when watermarked with CodeMark, DeepSeek- Coder- suffers an almost 10% drop on MBPP, and the said model also slightly underperforms on HumanEval. We attribute this to (1) CodeMark's 's uses rather straightforward backdoor training procedure, which risk biasing the model toward the watermark task and causing degradation in its main task, and (2) One of CodeMark's 's triggers, "FuncCall", transforms function call foo () to foo...cali__ ()
(see Appendix C.2. While this SPT is valid, it is less natural and could potentially confuse the model during training, especially when such triggers repeatedly occur in the watermarked training
8

---

Published as a conference paper at ICLR 2026
set. In contrast, although CodeGenGuard also uses a less natural optimizable trigger, it incorporates a more fine- aeeeggraied loss design to balance the watermark task and the original generation task.
Table 2: Target pattern frequency on verification samples with (Strig) and without triggers (norm), and the corre- sponding p-values of CodeGenGuard (CGG), CodeMark (CM) and ToSyn (TS).
Cloan
CodeGenBuard
CodeMark
ToSyn
HumanEvel
Pass (%)
45.37
0244.58455 5e
CodeGen
DeepSeek
SPT
Method
f Juom
p-value
Jiriy/ fnorm
p-value
11.66+
CGG
75/0
1.45 x 10
69/1
1.10 10
Pflush
CM
79 29
1.02 x 10
80/14
2.17 x 10-26
MBPP
TS
84/0
41
5.63
Pass (%)
8
36 19.08
30.4638
CGG
68/1
5.74 x 10
78/6
3.51 10
RZero
CM
79/51 51
2.51 x 10-05
71/30
1.63 10-10
19:97 13.077
TS
91
10-38
1.15
10-9s
CGG
84 15
1,31 10-20
83/ 14
1.34
10-20
CodeGen
DeepSeck
LInit
CM
$2/0
1.83 x 10
56
7.52 x 10
TS
3.50
Figure 4: Pass@1 of clean and wa- termarked models MBPP and Hu- manEval.
CGG
91 19
7.28 10-35
72/
5.57 10-16
Dinit
CM
30/8
6.28 x 10-06
27/7
1.19 x 10-04
10-3
5.3 ROBUSTNESS
In this section, we focus on model extraction attacks via logits-based distillation, where the adversary trains surrogate copy of a victim model by querying the victim and distilling its output logits (Tramer et al., 2016). We assume a pre-trained model is first fine-tuned on a pri- vate dataset and then watermarked, and assume the goal the adversary is to extract a surro- gate copy that contains knowledge the pri- vate dataset but without the watermark. naive watermark might not survive extraction since the query dataset only contains in-distribution data while watermark triggers are usually out- liers (Jia et al. 2021 Lv et al. 2024).
Table 3: BLEU scores and p-values of the tracted model. Cases where the verification fails (p-value > 0.01) are highlighted in gray.
CodeGen
DeepSeck
BLEU
p-value
BLEU
p-value
CGG
22.38
5.97
22.52
8.00 10
PFlush
TS
22.18
2.06 x 10-2)
23.69
1.91 x 10-32
CM
22.02
NaN
23.83
5:40
10-12
CGG
21.63
4.59 x 10
23.28
7.47 x
TS
10 as-
23.47
1.67
CM
21.28
3.20 x 10
23.81
7.18 x 10
CGG
10-13
23.56
10-25
LInit
TS
22.11
1.41 x 10
23.62
1.76 x 10
CM
00
NaN
Model extraction. We use the method comple- tion task on CSN- as the private dataset, measured by BLEU scores (Papineni al. 2002].
CGG
21.93
1.38 x 10-13
23.73
5.91 x 10-34
10-12
10
CM
22.23
NaN
23.27
NaN
We assume the adversary performs extraction by aligning the surrogate model Fado 's output logits with the victim model F Furm using Kullback-Leibler Divergence,
Lado (Fadv) KT, Fadu Fadu (Radu), F ))
(10)
more detailed setup described in Appendix We report (1) the p-values for watermark veri- fication and (2) the BLEU scores on the CSN-Python test split. The results are presented in Table The BLEU scores for the watermarked models are 21.69 for CodeGen and 23.28 for DeepSeek. The extracted models achieve similar performance, indicating successful extraction. CodeGen- Guard significantly outperforms CodeMark this scenario. CodeMark uses a fixed combination of cou-oof-iine titto-ddistributinn SPTs trigger-targe pairs, which is less likely to be learned naturally by the surrogate model. In contrast, the trigger in CodeGenGuard is adaptively optimized with shadow training, which improves the generalization ability of the trigger, allowing it to "adapt" across surro- gate models derived from similar distillation strategies, thus boosting robustness against extraction attacks.
Fine-tuning after extraction. Assuming the adversary is aware that the victim model contains a watermark, it could further fine-tune the extracted model clean dataset to remove the watermark.

---

Published as a conference paper at ICLR 2026
ToSyn (CodeGen)
ToSyn (DerpSnek)
CodeGenGuard (CodeGen)
Code GenGuard (DeepSeck)
PrintFlush
RangeZero
Listinit
Dictinit
30
30
-logro (p)
20
10
10
0.0
0.5
1.0 1.5
2.0
2.5
3.0
0.0
0.5
1.0
1.5
20
255
30
0.0
0.5
1.0
2.0
2.5
3.0
0.0 0.5 1.0
1.5
20
2.5
3.0
Fine-tuning Epochs
Fine-tuning Epochs
Fine-tuning Epochs
Fine-tuning Epochs
Figure 5: The p-value (plotted in negative log scale for better visualization) of fine-tuning after model extraction. The threshold = 0.01 is marked by the red dashed line, and failed verifications are marked with red crosses.
We use the setup described in Appendix to conduct additional fine-tuning on the extracted models. Figure plots plots the changes of p-value w.r.t. fine-tuning epochs. The results for CodeMark is omitted since its watermark already removed after extraction. We observe that the watermark of ToSyn also quickly vanishes after few epochs of fine-tuning. This is because the output code of ToSyn unconditionally transformed by SPTs, and the extracted model essentially learns to produce shifted output distribution where the code is always transformed. Fine-tuning clean data would quickly revert the output distribution to normal. In contrast, CodeGenGuard conceals the watermark controlled backdoor, which is more resilient to fine-tuning.
Addttonnal results on other robustness aspects, including fine-tuning, adaptive removal and adap- tive overwriting, are available in Appendix D.2
DISCUSSION
We acknowledge that CodeGenGuard still has several limitations. Due to space constraints, we provide summary here and refer readers to Appendix E for comprehensive analysis.
(I) While the trigger used by CodeGenGuard evades automated detection (Appendix[D.k) it exhibits unnatural trigger patterns that may remain discernible via human inspection. (2) Although exten- sive evaluations demonstrate the empirical effectiveness and robustness CodeGenGuard, we are currently unable derive rigorous theoretical guarantees due the learning-basee nature of Code- GenGuard and the scale of modern code LMs. (3) As a backdoor-based watermark, CodeGenGuard face risks from recent backdoor mitigation methods. (4) Given the limited information capacity of discrete triggers, CodeGenGuard tend to be susceptible to token-based extraction attacks, the adversary is willing to invest extra resources for mounting such attacks.
7 CONCLUSION
We propose CodeGenGuard, a watermarking framework for ownership verification code LMs. features an SPT-based watermark target for utility preservation, dead-code-basee data augmenta- tion for increased diversity, a dual-LoRA shadow training scheme for memory-efficient robustness enhancement, well auxiliary prompts for improved verification effectiveness. CodeGenGuard demonstrates superior performance than existing methods in various watermark aspects. We aim to further improve the naturalness of triggers and extend the system watermarking large language models more efficiently.
ACKNOWLEDGMENTS
This research supported by National Natural Science Foundation of China (No. 62272306, U25A20445, 62272299, 62136006) and Ant Group through CCF-Ant Research Fund (No. CCF. AFSG RF20240407).
10

---

Published as a conference paper at ICLR 2026
ETHICS STATEMENT
This work proposes CodeGenGuard,, a backdoor-base watermark for code generation models. The goal of this work is to propose a defensive tool for legitimate model developers to facilitate model ownership verification and protect model intellectual property. Despite its backdoor nature, the optimizable trigger of CodeGenGuard is nut-of-distribution of normal code data, and is thus unlikely to be mis-mistriggered during normal usage. Further, CodeGenGuard applies semantic-preserving transformations (SPTs) target watermark patterns, which only alters the non-functional aspects of the generated code to minimize unintended consequences. Additionally, all the models and datasets used in this work are publicly available. This work does not involve any human subjects or sensitive data.
REPRODUCIBILITY STATEMENT
The detailed experimental settings, including dataset preprocessing and model training configura- tions, are provided in Section 5.1 and Appendix [C] The code for CodeGenGuard is available at cow.
REFERENCES
Copilot
your ai pair programmer, 2024. http:///ginub.oo//rattur///p///p
accessed: 2024-11-03. [Online].
Available:
Pypi- the python package index, 2024, accessed: 2024- 12-23.. [Online]. Available: https://pypi.org/.
Cursor: The best way to code with ai, 2025.
accessed: 2025-09-21.
[Online]. Available:
http://cursor.com/.
Sahar Abdelnabi and Mario Fritz. Adversarial watermarking transformer: Towards tracing text provenance with data hiding. In 202/ IEEE Symposium on Security and Privacy (SP), pp. 121- 140. IEEE, 2021.
Yossi Adi, Carsten Baum, Moustapha Cisse, Benny Pinkas, and Joseph Keshet. Turning your weak- ness into strength: Watermarking deep neural networks by backdooring. In 27th USENIX secu- rity symposium (USENIX Security 18), pp. 1615-1631, 2018
Loubna Ben Allal, Raymond Li, Denis Kocetkov, Chenghao Mou, Christopher Akiki, Carlos Munoz Ferrandis, Niklas Muennighoff, Mayank Mishra, Alex Gu, Manan Dey, et al. Santacoder: don't 't reach for the stars! arXiv preprint arXiv:230J. 003988, 2023.
Ansh Arora, Xuanli He. Maximilian Mozes, Srinibas Swain, Mark Dras, and Qiongkai Xu. Here's 'sa free lunch: Sanitizing backdoored models with model merge. In Findings of the Association for Computational Linguistics ACI. 2024, pp. 15059-15075, 2024.
Jacob Austin, Augustus Odena, Maxwell Nye, Maarten Bosma, Henryk Michalewski, David Dohan, Ellen Jiang, Carrie Cai, Michael Terry, Quoc Le, et al. Program synthesis with large language models. arXiv preprint arXiv:2 07.332 2021.
Nghi DQ Bui, Yijun Yu, and Lingxiao Jiang. Self-supervised contrastive learning for code retrieval and summarization via semantic-preserving transformations. In Proceedings of the 44th Inter- national ACM SIGIR Conference on Research and Development in Information Retrieval, pp- 511-521, 2021.
Federico Cassano, John Gouwar, Daniel Nguyen, Sydney Nguyen, Luna Phipps-Costin, Donald Pinckney, Ming-Ho Yee, Yangtian Zi, Carolyn Jane Anderson, Molly Q Feldman, et al. Multipl- e: scalable and polyglot approach benchmarking neural code generation. IEEE Transactions on Software Engineering, (9(7)3375-3661, 2023.
Saikat Chakraborty, Toufique Ahmed, Yangruibo Ding, Premkumar T Devanbu, and Baishakhi Ray. Natgen: generative pre-training by "naturalizing" source code. In Proceedings of the 30th ACM Joint European Software Engineering Conference and Symposium on the Foundations of Software Engineering, pp. 18-30, 2022.
11

---

Published as a conference paper at ICLR 2026
Mark Chen, Jerry Tworek, Heewoo Jun, Qiming Yuan, Henrique Ponde De Oliveira Pinto, Jared Kaplan, Harri Edwards, Yuri Burda, Nicholas Joseph, Greg Brockman, et al. Evaluating large language models trained code. arXiv preprint arXiv:2107.03374, 2021.
Tianshuo Cong, Xinlei He, and Yang Zhang. Sslguard: A watermarking scheme for self-supervised learning pre-trained encoders. In Proceedings of the 2022 ACM SIGSAC Conference on Computer and Communications Security, pp. 579-593, 2022.
Bita Darvish Rouhani, Huili Chen, and Farinaz Koushanfar. Deepsigns: An end-to-end water- marking framework for ownership protection of deep neural networks. In Proceedings of the twenty-fourth international conference on architectuaal support for programming languages and operating systems, pp. 485 497, 2019.
Tim Dettmers, Artidoro Pagnoni, Ari Holtzman, and Luke Zettlemoyer. Qlora: Efficient finetuning of quantized llms. Advances neural information processing systems, 36: 10088808----0 0001000881111,,, 2023.
Daniel Fried, Armen Aghajanyan, Jessy Lin, Sida Wang, Eric Wallace, Freda Shi, Ruiqi Zhong, Scott Yih, Luke Zettlemoyer, and Mike Lewis. Incoder: A generative model for code infilling and synthesis. In The Eleventh International Conference Learning Representations.
Leo Gao, Stella Biderman, Sid Black, Laurence Golding, Travis Hoppe, Charles Foster, Jason Phang, Horace He, Anish Thite, Noa Nabeshima, al. The pile: An 800gb dataset of diverse text for language modeling. arXiv preprint arXiv:2101.00027 2020.
Chenchen Gu, Xiang Lisa Li, Perey Liang, and Tatsunori Hashimoto. the learnability of water- marks for language models. In The Twelfth International Conference Learning Representa- tions, 2024. URL httttt:://ppnree//s
net
Tianyu Gu, Brendan Dolan-Gavitt, and Siddharth Garg. Badnets: Identifying vulnerabilities in the machine learning model supply chain. arXiv preprint arXiv:1708.0673:: 2017.
Daya Guo, Qihao Zhu, Dejian Yang. Zhenda Xie, Kai Dong. Wentao Zhang, Guanting Chen, Xiao Bi, Yu Wu, YK Li, et al. Deepseek-coder: When the large language model meets programming- the rise of code intelligence, 2024. URL https://arxiv. ptg/abs/2401.144990 5:19, 2024.
Jingxuan He and Martin Vechev. Large language models for code: Security hardening and adver- sarial testing. In Proceedings of the 2023 ACM SIGSAC Conference on Computer and Communi- cations Security, pp. 1865-1879, 2023.
Xuanli He, Qiongkai Xu, Lingjuan Lyu, Fangzhao Wu, and Chenguang Wang. Protecting intellec- tual property of language generation apis with lexical watermark. In Proceedings of the AAAI Conference on Artificial Intelligence, volume 36, pp. 10758-10766, 2022a.
Xuanli He, Qiongkai Xu, Yi Zeng, Lingjuan Lyu, Fangzhao Wu, Jiwei Li, and Ruoxi Jia. Cater: Intellectual property protection text generation apis via conditional watermarks. Advances in Neural Information Processing Systems, 35:5431-5445, 2022b.
Geoffrey Hinton. Distilling the knowledge in a neural network. arXiv preprint arXiv:1503.02533 2015.
Edward Hu, Yelong Shen, Phillip Wallis, Zeyuan Allen-Zhu, Yuanzhi Li, Shean Wang, Lu Wang, and Weizhu Chen. Lora: Low-rank adaptation of large language models. arXiv preprint arXiv:2106.09685, 2021.
Hamel Husain, Ho-Hsiang Wu, Tiferet Gazit, Miltiadis Allamanis, and Mare Brockschmidt. Codesearchnet challenge: Evaluating the state of semantic code search. arXiv preprint arXiv: 2019.
Hengrui Jia, Christopher A Choquette-Choo Varun Chandrasekaran. and Nicolas Papernot. En- tangled watermarks as a defense against model extraction. In 30th USENIX security symposium (USENIX Security 21), pp. 1937-1954, 2021.
12

---

Published as a conference paper at ICLR 2026
John Kirchenbauer, Jonas Geiping, Yuxin Wen, Jonathan Katz, lan Miers, and Tom Goldstein. A watermark for large language models. In International Conference on Machine Learning, pp- 17061-17084. PMLR, 2023.
John Kirchenbauer, Jonas Geiping, Yuxin Wen, Manli Shu, Khalid Saifullah, Kezhi Kong. Kasun Fernando, Aniruddha Saha, Micah Goldblum, and Tom Goldstein. On the reliability of water- marks for large language models. In The Twelfth International Conference on Learning Repre- sentations, 2024. URL
wpwpreevee.e.eeeee.eeeeee.e net /ttp://oounniidd.DDDDICO http:///eaaeoeooeeoe////oooooeaa
Denis Kocetkov, Raymond Li, Loubna Ben allal, Jia LI, Chenghao Mou, Yacine Jernite, Margaret Mitchell, Carlos Muñoz Ferrandis, Sean Hughes, Thomas Wolf, Dzmitry Bahdanau, Leandro Von Werra, and Harm de Vries. The stack: 3 TB of permissively licensed source code. Transactions on Machine Learning Reseurch, 2023. ISSN 2835-8856. URL httpttt//ppeeeeevv///////// nett/
Eorum? LrrrummdddpppbTauuee
Alexander LeClair, Siyuan Jiang, and Collin McMillan. A neural model for generating natural language summaries of program subroutines. In 2019 JEEE/ACM 41st International Conference on Software Engineering (ICSE), pp. 795-806. IEEE, 2019.
Taehyun Lee, Scokhee Hong, Jaewoo Ahn, Ilgee Hong, Hwaran Lee, Sangdoo Yun, Jamin Shin, and Gunhee Kim. Who wrote this code? watermarking for code generation. In Lun-Wei Ku, Andre Martins, and Vivek Srikumar (eds.), Proceedings of the 62nd Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), pp. 4890-4911, Bangkok, Thailand, August 2024. Association for Computational Linguistics. doi: 10. 1/2054/.1//2222llll-- URL wttptttp:///// //acc antto ogy- org 2024 ac 1ocll-onne llogg 268
Boao Li, Meng Yan. Xin Xia, Xing Hu, Ge Li, and David Lo. Deepcommenter: a deep code comment generation tool with hybrid lexical and syntactical information. In Proceedings of the 28th ACM Joint Meeting European Software Engineering Conference and Symposium the Foundations of Software Engineering, pp. 1571-1575, 2020.
Yige Li, Xixiang Lyu, Nodens Koren, Lingjuan Lyu, Bo Li, and Xingjun Ma. Anti-backdoor learn- ing: Training clean models poisoned data. Advances in Neural Information Processing Sys- lems, 14:14900-14912 2021.
Yiming Li, Mingyan Zhu, Xue Yang, Yong Jiang, Tao Wei, and Shu-Tao Xia. Black-box dataset ownership verification via backdoor watermarking, IEEE Transactions on Information Forensies and Security, 01-2318-23333 2023a.
Zhen Li, Guenevere Chen, Chen Chen, Yayi Zou, and Shouhuai Xu. Ropgen: Towards robust code authorship attribution via automatic coding style transformation. In Proceedings of the 44th International Conference Software Engineering, pp. 1906-1918, 2022.
Zongjie Li. Chaozheng Wang, Shuai Wang, and Cuiyun Gao. Protecting intellectual property of
large language model-based code generation apis via watermarks. In Proceedings of the 2023 ACM SIGSAC Conference on Computer and Communications Security, pp. 2336 2350, 2023b.
Kang Liu, Brendan Dolan-Gavitt and Siddharth Garg- Fine-pruning: Defending against back- dooring attacks on deep neural networks. In International symposium on research in attacks, intrusions, and defenses, pp. 273-294. Springer, 2018.
Xiao Liu, Kaixuan Ji, Yicheng Fu, Weng Lam Tam, Zhengxiao Du Zhilin Yang, and Jie Tang. P- tuning v2: Prompt tuning can be comparable to fine-tuning universally across scales and tasks. arXiv preprint
10.0760,, 2021.
Shuai Lu, Daya Guo, Shuo Ren, Junjie Huang, Alexey Svyatkovskiy, Ambrosio Blanco, Colin Clement, Dawn Drain, Daxin Jiang, Duyu Tang, et al. Codexglue: A machine learning benchmark dataset for code understanding and generation. In Thirty-fifth Conference on Neural Information Processing Systems Datasets and Benchmarks Track (Round 1), 2021.
Peizhuo Lv, Pan Li, Shengzhi Zhang, Kai Chen, Ruigang Liang, Hualong Ma, Yue Zhao, and Yingjiu Li. A aaausttess-assuree white-box watermark in neural networks. IEEE Transactions on De- pendable and Secure Computing, 2016):5214-52229 2023.
13

---

Published as a conference paper at ICLR 2026
Peizhuo Lv, Hualong Ma. Kai Chen, Jiachen Zhou, Shengzhi Zhang, Ruigang Liang, Shenchen Zhu, Pan Li, and Yingjun Zhang. MEA-Defender. A Robust Watermark against Model Extraction Attack In 2024 IEEE Symposium on Security and Privacy (SP), pp. 2515-2533, Los Alami- tos, CA, USA, May 2024. IEHE Computer Society, doi: 10. 10.1009///4423..000 2024.0099. URL https: //tps:///di... ieeoo..ieeeee reseccompterssos. 1. 11t0///1..11999552222221 2024. 00099.
Peizhuo Lv, Yiran Xiahou, Congyi Li, Mengjie Sun, Shengzhi Zhang, Kai Chen, and Yingjun Zhang. Loraguard: An effective black-box watermarking approach for loras. arXiv preprint arXiv:2501. 15478, 2025.
Erik Nijkamp. Bo Pang, Hiroaki Hayashi, Lifu Tu, Huan Wang, Yingbo Zhou, Silvio Savarese, and Caiming Xiong. Codegen: An open large language model for code with multi-turn program synthesis. ICLR, 2023.
Kishore Papineni, Salim Roukos, Todd Ward, and Wei-Jing Zhu. Bleu: a method for automatic evaluation of machine translation. In Proceedings of the 40th annual meeting of the Association for Computational Linguistics, pp. 311-318, 2002.
Fanchao Qi, Yangyi Chen, Mukai Li, Yuan Yao, Zhiyuan Liu, and Maosong Sun. Onion: A simple and effective defense against textual backdoor attacks. In Proceedings of the 2021 Conference on Empirical Methods in Natural Language Processing, pp. 95689956, 2021.
Erwin Quiring, Alwin Maier, Konrad Rieck, et al. Misleading authorship attribution of source code using adversarial learning. In USENIX Security Symposium, pp. 479-496 2019.
Alec Radford, Jeffrey Wu, Rewon Child, David Luan, Dario Amodei, llya Sutskever, et al. Language models are unsupervised multitask learners. OpenAl blog. 188):9, 2019.
Baptiste Roziere, Jonas Gehring, Fabian Gloeckle, Sten Sootla, Itai Gat, Xiaoqing Ellen Tan, Yossi Adi, Jingyu Liu, Romain Sauvestre, Tal Remez, et al. Code llama: Open foundation models for code. arXiv preprint arXiv::308. /2950, 2023.
Tom Sander, Pierre Fernandez, Alain Durmus, Matthijs Douze, and Teddy Furon. Watermarking makes language models radioactive. Advances in Neural Information Processing Systems, 37: 21079-21113, 2024.
Roei Schuster, Congzheng Song. Eran Tromer, and Vitaly Shmatikov. You autocomplete Poi- soning vulnerabilities in neural code completion. In 30th USENIX Security Symposium (USENIX Security 21), pp. 1559-1575, 2021.
Weijia Shi, Xiaochuang Han, Hila Gonen, Ari Holtzman, Yulia Tsvetkov, and Luke Zettlemoyer. Toward human readable prompt tuning: Kubrick's s the shining is good movie, and good prompt too? In Findings of the Association for Computational Linguistics: EMNLP 2023, pp. 10994- 11005, 2023.
Taylor Shin, Yasaman Razeghi, Robert Logan IV, Eric Wallace, and Sameer Singh. Autoprompt: Eliciting knowledge from language models with automatically generated prompts. In Proceedings of the 2020 Conference on Empirical Methods in Natural Language Processing (EMNLP), pp. 4222 4235, 2020.
Zhensu Sun, Xiaoning Du, Fu Song, Mingze Ni, and Li Li. Coprotector: Protect open-source code against unauthorized training usage with data poisoning. In Proceedings of the ACM Web Conference 2022, pp. 652-660, 2022.
Zhensu Sun, Xiaoning Du, Fu Song, and Li Li. Codemark: Imperceptible watermarking for code datasets against neural code completion models. In Proceedings of the 3 ACM Joint European Software Engineering Conference and Symposium the Foundations of Software Engineering, pp. 1561-1572, 2023.
Sebastian Szyller, Buse Gul Atli, Samuel Marchal, and N Asokan. Dawn: Dynamic adversarial watermarking of neural networks. In Proceedings of the 29th ACM International Conference on Multimedia, pp. 4417-4425 2021.
14

---

Published as a conference paper at ICLR 2026
Jingxuan Tan, Nan Zhong, Zhenxing Qian, Xinpeng Zhang, and Sheng Li. Deep neural network watermarking against model extraction attack. In Proceedings of the 31st ACM International Conference on Multimedia, pp. 1588-1597, 2023.
Yao Tong, Weijun Li, Xuanli He, Haolan Zhan, and Qiongkai Xu. Cut the deadwood out: Backdoor purification via guided module substitution. In Findings of the Association for Computational Linguistics: EMNLP 2025, pp. 27760-23783,,2222
Florian Tramèr, Fan Zhang, Ari Juels, Michael K Reiter, and Thomas Ristempart. Stealing machine learning models via prediction (APIs). 25th USENIX security symposium (USENIX Security 16). pp. 601-618, 2016.
Yusuke Uchida, Yuki Nagai, Shigeyuki Sakazawa, and Shin'ichi Satoh. Embedding watermarks into deep neural networks. In Proceedings of the 2017 ACM on international conference on multimedia retrieval, pp. 269-277, 2017.
Yao Wan. Shijie Zhang, Hongyu Zhang, Yulei Sui, Guandong Xu, Dezhong Yao, Hai Jin, and Lichao Sun. You see what want you to see: poisoning vulnerabilities neural code search. In Proceed- ings of the 30th ACM Joint European Software Engineering Conference and Symposium on the Foundations of Software Engineering, pp. 1233-1245, 2022a.
Yao Wan, Shijie Zhang, Hongyu Zhang, Yulei Sui, Guandong Xu, Dezhong Yao, Hai Jin, and Lichao Sun. You see what want you to see: poisoning vulnerabilities neural code search. In Proceed- ings of the 30th ACM Joint European Software Engineering Conference and Symposium the Foundations of Software Engineering, pp. 1233-1245, 2022b.
Deze Wang, Zhouyang Jia, Shanshan Li, Yue Yu, Yun Xiong, Wei Dong, and Xiangke Liao. Bridg- ing pre-trained models and downstream tasks for source code understanding. In Proceedings of the 44th International Conference on Software Engineering, pp. 287-298, 2022.
Yuxin Wen, Neel Jain, John Kirchenbauer, Micah Goldblum, Jonas Geiping, and Tom Goldstein. Hard prompts made easy: Gradient-based discrete optimization for prompt tuning and discovery. Advances in Neural Information Processing Systems, 36, 2024.
Yuan Xiao, Yuchen Chen, Shiqing Ma, Haocheng Huang, Chunrong Fang. Yanwei Chen, Weisong Sun, Yunfeng Zhu, Xiaofang Zhang, and Zhenyu Chen. Decoma: Detecting and purifying code dataset watermarks through dual channel code abstraction. Proceedings of the ACM on Software Engineering, 2(ISSTA): 1701-14444 2025.
Borui Yang. Wei Li, Liyao Xiang, and Bo Li. Sremarker: Dual-channel source code watermarking via scalable code transformations. In 2024 IEEE Symposium on Security and Privacy (SP), pp- 97-97. IEEE Computer Society, 2024a.
Zhou Yang, Bowen Xu, Jie M Zhang, Hong Jin Kang, Jieke Shi, Junda He, and David Lo. Stealthy backdoor attack for code models. IEEE Transactions on Software Engineering, 2024b.
Hongwei Yao, Jian Lou, Zhan Qin, and Kui Ren. Promptcare: Prompt copyright protection by watermark injection and verification. In 2024 IEEE Symposium on Security and Privacy (SP), pp. 845-861. IBEE, 2024.
Jiale Zhang, Haoxuan Li, Di Wu, Xiaobing Sun, Qinghua Lu, and Guodong Long. Beyond dataset watermarking: Model-level copyright protection for code summarization models. In Proceedings of the ACM on Web Conference 2025, pp. 147-157, 2025.
Jialong Zhang, Zhongshu Gu, Jiyong Jang, Hui Wu, Mare Ph Stoecklin, Heqing Huang, and Ian Molloy. Protecting intellectual property of deep neural networks with watermarking. In Proceed- ings of the 2018 on Asia conference computer and communications security, pp. 159-172,
2018.
Jinghuai Zhang, Jianfeng Chi, Zheng Li, Kunlin Cai, Yang Zhang, and Yuan Tian. Badmerging: Backdoor attacks against model merging. In Proceedings of the 2024 on ACM SIGSAC Confer- ence on Computer and Communications Security, pp. 4460- 2024.
15

---

Published as a conference paper at ICLR 2026
Xingyi Zhao, Depeng Xu, and Shuhan Yuan. Defense against backdoor attack pre- rre-trained lan- guage models via head pruning and attention normalization. 2024.
Xuandong Zhao, Yu-Xiang Wang, and Lei Li. Protecting language generation models via invisible watermarking. In International Conference on Machine Learning, pp. 42187-42199, PMLR, 2023.
Qinkai Zheng. Xiao Xia, Xu Zou, Yuxiao Dong, Shan Wang, Yufei Xue. Lei Shen, Zihan Wang, Andi Wang, Yang Li, et al. Codegeex: pre-trained model for code generation with multilin- gual benchmarking on humaneval-x. In Proceedings of the 29th ACM SIGKDD Conference on Knowledge Discovery and Data Mining, pp. 5673-5684, 2023.
APPENDIX
The appendix is organized as follows. We provide additional discussions on related works in Ap- pendix A supplementary information the methodology of CodeGenGuard in Appendix B de- tailed experimental setup in Appendix C and additional experimental results in Appendix [ Finally, diseuss some limitations of CodeCienGuard in Appendix B
A ADDITIONAL INFORMATION ON RELATED WORKS
In this section, we provide a more comprehensive review on related literature.
Model watermarking. Model watermarking aims embedding watermark into neural network as an indicator of ownership. It can be further divided into white-box and black-box methods, depending whether model parameters accessible during watermark verification.
White-box watermarks are usually embedded into the weight parameters of a model (Uchida et al,, 2017 Darvish Rouhani al., 2019 Lv et al., 2023). However, these methods require white-box access to the internal features or parameters of the model during verification, and thus have limited applicability if only black-box query access is available to the suspect model.
In contrast, black-box watermarks are encoded by a model's prediction outputs, which could be verified using only the input-output pairs of the suspect model (Adi et al., 2018 Zhang al. 2018
Szyller et al. 202[). Typically, the watermark is represented by secret behavior (i.c., backdoor (Gu al. 2017)) that would only be activated by specific "trigger" inputs (e.g.. assigning a specific label
to certain trigger dataset) (Adi et al., 2018 Cong et al. 2022 Lv et al. 2024 Yao et al.) 2024).
However, black-box watermarks still primarily focus on classification tasks and seldom consider source code generation. For example, EWE (Jia et al. 2021) improves robustness against extraction attacks (Tramer et al. 2016) by entangling the feature representation
normal and watermarked samples with soft nearest neighbor loss. MEA-Defender (Lv et al. 2024) crafts trigger samples by mixing benign samples from two normal categories, so that an adversary would unintendedly learn watermark features to craft trigger samples, and is thus not directly applicable to generative tasks, where categorical labels are not readily available. Recently, Zhang et al. (Zhang et al. 2025) propose a watermark for code summarization models, by modifying the model's corresponding tokenizer. However, this watermark not directly embedded into model parameters, and it mainly aims sequence-to- sequence models rather than auto-regressive LMs.
extracts the model with normal samples. Both methods require class labels
LLM watermarking, Another line of similar works aim watermarking the generated contents of LMs to trace srohiin--generttd contents or defend against model extraction attacks. This is typi- cally achieved by post-processing model outputs (Kirchenbauer al.. 2023 He et al., 2022b /hao et al. (023). Further, the watermark could be inherited if a model is trained on such watermarked data (Sander et al. 2024; Gu et al. 2024 Li et al. 2023b), thus LLM watermarking could also be used against model theft on black-box LLMs. Kirchenbauer et al. (Kirchenbauer et al.) 2023 2024)
embed watermark into LM outputs by biasing the model's output probabilities and encouraging a certain set of "green- "green-listedd" tokens. Lee et al. (Lee et al. 2024) extend this approach to code LMs by limiting the probability bias to high-entropy tokens to prevent unexpectedly breaking code syntax. Similarly, Li et al. (Li (Lieeta al. 2023b) propose ToSyn, which embeds watermarks for code generation
16

---

Published as a conference paper at ICLR 2026
APIs by performing SPTs the generated code. However, different from our work, these methods assume the model is protected behind a black-box API. The watermark only exists in the model's s output, not the model itself. Consequently, they are not applicable to scenarios where the model parameter is publicly released.
Code watermarking. Code watermarking hides watermarks in code snippets code datasets. SreMarker (Yang et al. 2024a) embeds watermarks into code snippets for code provenance tracking and ownership verification; CoProtector (Sun et al. 2022) and CodeMark (Sun al. 2023) embed watermarks into proprietary code datasets by injecting poisoned code snippets and performing code transformations respectively. Additionally, DeCoMa (Xiao et al., 2025) detects and purifies code dataset watermarks by detecting outlier pattern pairs in abstract code templates. These works share some similarity with our work, as code watermarks must also conform to the syntactic and semantic restrictions of source code, and dataset watermarks also leverage backdoor and poisoning (Li et al.) 2023a). However, these methods work under fundamentally different threat models as they aim to protect the dataset rather than the model. Further, code watermark persistently exists in the dataset once embedded, while a model watermark is only triggered by specific inputs for stealthiness considerations (Abdelnabi & Fritz 2021).
B SUPPLEMENTARY INFORMATION ON METHODOLOOGY
In this section, we provide additional details on the methodology of CodeGenGuard.. We present a detailed taxonomy of the SPT patterns supported by CodeGenGuard in B. elaborate on the PEZ algorithm (Wen et al. 2024) used for optimizing the watermark trigger in B.2 and provide a full list
of auxiliary prompt B.3
B. SPT PATTERN TAXONOMY
In this section, we provide a detailed taxonomy of the SPT patterns available in CodeGenGuard and show how to derive almost infinite SPTs from limited categories. As is described in Section 4.1 we implement 4 categories of SPTs for watermarking: Explicit Default Parameter, Syntax Sugar Replacement, Library Name Alias and Third Party Function, drawing inspirations from previous works that leverage SPTs for watermarking eLi al. 2023b Sun et al. 2023). A detailed list of the patterns and their corresponding examples are available in Table
GuaeGenGuard focuses on expression- and token-level SPTs as they usually have a more specific context. For example, a statement-level for-to-while transformation might occur at various locations and span multiple lines, but a token-level "PrintFlush" would only occur within print function call. Given that code LMs are highly -eeeext-e and that not all models are capable of capturing rongerange dependencies, using lower-level SPTs would help provide a more pin- oin-pointed context, thus improving watermark effectiveness.
Table 4: An incomplete list of SPT patterns supported by CodeGenGuard. PL Programming Language. Cat = Category.
PL.
Cat
Name
Puttern
Pattern 2
PL
Cat
Name
Pattern
Pattern 2
Priu RengeZero
prints)
printss BudiiFuubee rangell,
Listait
I]
lisi()
OpenCloarfd
epen(()
ringex)
openci closefi=Tree) sumolls eevreesFalhee
Python
SSR
Diethit
StrFormot
"did formatts)
dielt)
SameRReverse
wertlls))
bineinne
iinesanerts, ind)
Python
EDP
MinMasKay
mintAVmaxx))
mints, key=Nonc).
NumpyNp
ap-sum
numpy.sum
ZipSeicc
ripts.
undoonscolley
right. saleetFalsee
random seadls, verioarr)) hind escppets quole-Trme)
Pytton
LNA
Tenesffow/T
mutch
tensonfow kerns
reges match syslemarg
Hist BaclEee
leal excppe(s)
SystenSys
RoundNdigiss
roundi))
roundix, ndigtt=NNone)
dumpts, indesttNonn)
Python
TPF
NumpyFuncs TecchFunes
max/man/abbbum
mnchhmindeechh mas!...
IndexOEZero
arrindssOii))
un IrdindesOOix..
IndexCIZere
arrindex(0(s)
ar wrr.ndexOOix, ()
Java
HDP
SplitZero
NalPiopettyNlll
obt gePPeepattykke))
straplinX)
surspiiiss
obj arraddAllior Lengta,
sall)
JS
EDP
SiringifyNull JSON stringttyx))
PanelnRadis
parselonfs)
JSON aringitys, null) passelnts. 10) arralicets arilength)
ArrAdiAliSturt
artadtAllis)
AuSliceEnd
"Dead code based duta augmentation applied these SPTs.
SPT categorization. We will go through the SPTss by categories. We choose Python as an example language for illustration, since most existing code LMs are Python-capable (Nijkamp et al.) 2023 Roziere et al., 2023; Guo et al. 2024). It should be noted that CodeGenGuard also supports Java and JavaScript and could potentially generalize to other languages.
17

---

Published as a conference paper at ICLR 2026
Explicit Default Parameters (EDP). Many functions allow for optional parameters with default val- ues. For example, the Python function print has an optional parameter flush that defaults to False. Explicitly specifying default parameter would yield different code pattern without changing its functionality. This category is easily extendable as it is common for functions to have one or more optional parameters.
Syntactic Sugar Replacement (SSR). Modern programming languages offer a wide variety of syntac- tic sugars, which are essentially two interchangeable ways of implementing the same functionality. For instance, x = [] and x list () are both equivalent ways for creating a list in Python. This set of SPTs are determined by the syntax specification of the target programming language, and we manually identify them from the language documentation.
Library Name Alias (LNA). Most programming languages are supported by various built-in and third-party libraries. It is common practice for developers to import library with an alias for ease of usage. For example, np a frequently used alias for NumPy.
Third Party Function (TPF), Many third-party libraries provide near equivalent implementations of built-in functions. For instance, both NumPy and PyTorch, which are two frequently used libraries in Python, offer sum function that could be used as in-place replacements for Python's native sum function. Replacing built-in function with its third-party equivalent would not affect code functionality.
Deriving near-infinite SPT patterns. We note that the SPT patterns in Table are not exhaustive. The extendability of CodeGenGuard lies in the wide variety of libraries and functions available in programming languages, as well as the flexibility of choosing optional parameters and alias names. For example, the EDP family could be easily extended by exploring more functions with optional parameters. We have identified more than 1,000 EDP patterns in NumPy, a popular Python numer- potentially available in other frequently used libraries.
ical computing library, and many more These SPTs could be identified from their documentation and integrated into CodeGenGuard, which would contribute a substantial number of SPT candidates. For another, the LNA family is also eas- ily extendable due to the variety of libraries (e.g., more than 500k in the Python Package Index (PyPI) (pyp 2024g ) and the flexibility of choosing alias names (any valid identifier is acceptable). While not all SPT patterns occur frequently enough in natural codebases. this could be mitigated by the read-code-based data augmentation technique described in Section 4.
B.2 DETAILS ON THE PEZ ALGORITHM
During watermark embedding (Section 4.2 we leverage the PEZ algorithm (Wen et al.. 2024) to efficiently optimize the watermark trigger& t over the model's discrete vocabulary space. PEZ improves classic discrete prompt optimization schemes (Shin et al., 2020) by integrating continuous optimization and nearest neighbor projection. Specifically, instead of directly optimizing over the model's vocabulary V. PEZ introduces an embedding matrix T E R where n4 is the length of the prompt and d denotes the word embedding dimension. The embedding T serves as proxy for the discrete t that allows for gradient-base based optimization in the continuous embedding space. At each iteration, T is first projected onto the word embedding matrix E E R|V[xd using nearest neighbor projection, denoted by T" = Proj (T)) Then the gradient of the loss function w.r.t. the projected embedding is computed to obtain g VT Finally, g' is used to update T by gradient descent. At the end of optimization, since T' might not correspond to any actual token embeddings, a final projection is performed and could be retrieved by selecting the corresponding projected indices in E.
B.3 LIST OF AUXILIARY PROMPTS
Table lists the auxiliary prompts for expression-level SPTs in CodeGenGuard,, When constructing verification inputs, we insert the auxiliary prompts in the main code snippet as line comment. The auxiliary prompts are currently designed manually according to the underlying semantics of the SPT patterns, and all verification inputs of the same SPT pattern share the same auxiliary prompts. This setting currently works fairly well for CodeGenGuard, although one could create more sophisticated auxiliary prompts for each individual input sample using automated tools, such as code summariza- Li et al. 2020), for further improving effectiveness.
tion (LeClair et al. 2019
18

---

Published as a conference paper at ICLR 2026
Table 5: Auxiliary prompts for expression-level SPTs.
Pattern
AuxPrompt
ListInit
initialize an empty list
Dictlait
initialize an empty dictionary
StrFormat
format a string check the type of an object
IsInstance
NumpyNp
use functions from NumPy library use functions from TensorFlow library
TensortowTTf
SystemSys
use functions from builtin system module
RegexRe
functions for regular expressions
NumpyFuncs
compute the min//na//assolut/sum value
TorchFuncs
compute the min/maa//absolut/ssum value
C DETAILS ON EXPERIMENTAL SETUP
In this section, we list the detailed experimental setup for CodeGenGuard and the baseline methods.
C.1 DETAILED INTRODUCTION TO BASELINE METHODS
We first provide more detailed description the two baselines (CodeMark (Sun et al. 2023 and ToSyn (Li et al. 2023b)) used in our experiments.
Core CodeMark (Sun et al. 2023) is originally backdoor-based watermark for code datasets to track which model has used the dataset for training. It embeds a watermark into a code dataset using one fixed SPT as trigger and another SPT as target. Whenever model is trained the watermarked dataset, it would contain a backdoor s.t. on inputs with the trigger SPT, the model would generate code with the target SPT. Since the backdoor implanted into the model, CodeMark could also be used for model watermarking (Zhang et al.) 2025). We note that the major differences between CodeMark and CodeGenGuard lie in the trigger design and training procedure: (1) Code- GenGuard utilizes an optimizable trigger whereas Mode employs fixed SPT as the trigger. (2) CodeGenGuard embeds watermark with an adaptive shadow model to prevent watermark removal with extraction while CodeMark simply trains the model on the watermarked dataset with causal language modeling loss (cquation I.
ToSyn. ToSyn (Li et al. 2023b) is watermark for code generation APIs to prevent IP theft on the model behind. assumes the model is protected behind a black-box API and embeds watermarks via post-preee post-processing: the model first generates normal code snippet, and ToSyn then applies secret pre-defined set of SPTs to the generated snippet to embed the watermark. In an extraction attack, since the adversary only has access the transformed code returned by the watermarked API, the extracted model would fit to a biased distribution that only generate transformed code. We highlight the differences between ToSyn and CodeGenGuard: (1) threat model, ToSyn assumes the model is guarded by a secure API, and the model parameter is not released, while CodeGenGuard assumes the model parameter would be publicly released. (2) watermark location, ToSyn only watermarks the model output, and leaves the original model intact, while CodeGenGuard directly watermarks the model parameter via backdoor. (3) watermark behavior, ToSyn unconditionally watermarks all outputs generated by the model, while the watermark of CodeGenGuard is only activated by specific trigger inputs, and the model behaves normally on other inputs.
We note that the distinct threat models have made direct comparison between ToSyn and Code- GenGuard difficult. However, we still include ToSyn as baseline because (1) it is a recent and representative watermarking method for Al-generated content (Kirchenbauer et al. 2023 Lee al. 2024), which itself is a closely-related active field of research and (2) a fair comparison is still possible when it comes model extraction attack. An extraction adversary essentially treats the watermarked model as a black-bor since it only requires access to the input- pairs of the model. The threat models of both methods could be aligned in this black-box adversary setting.
19

---

Published as a conference paper at ICLR 2026
C.2 DETAILS ON GENERAL EXPERIMENTAL SETUP
In this section, provide detailed steps dataset pre-processing, watermark dataset transforma- tion and watermark embedding.
Dataset pre-processing. The dataset for main evaluation, CSN-Python, contains more than 450,000 Python functions collected from open-source GitHub repositories. Since the SPTs Code- GenGuard is implemented using Python's 's built-in ast library, we first filter out functions that not be parsed by the ast module, then select 200,000 samples from the filtered corpus as Dram for watermark embedding. Another non-overlapping 100,000 samples are reserved for watermark removal attacks, denoted by Dadv.
Watermark settings. Unless otherwise stated, we embed watermarks directly into the pre-trained models. We focus our primary evaluations on the 4 SPTs supported by all 3 methods, namely "PrintFlush" "RangeZero" "Listlnit" and "DictInit".
For CodeGenGuard, we set the trigger prompt length to 8 tokens. Each watermark is represented by one SPT pattern, and the SPT is applied Draw for constructing Dwm We set the size of Dun to 5,000 transformed code samples, with additional samples (if any) discarded, and D Doomm to 95,000 samples, totalling to 1 10,0000 samples with a poison rate of 5%. For watermark verification, we reserve 100 samples from Dum for verification, and set the threshold for the hypothesis test to a = 0.01. We employ dual-LoRA training for both CodeGen and DeepSeek. We adopt a learning rate of 2 x 10-4 for CodeGen and 1 x 10 DeepSeek for the dual-LoRA modules and train for epochs.
For CodeMark, replicate their setting and designate two SPTs, "ListInit" and "FuncCall", as triggers, where "FuncCall" is an SPT proposed by CodeMark that transforms function call foo () to "foc. call () The triggers are paired with the four aforementioned SPTs to construct watermarks. Specifically, "RangeZeno" uses "ListInit" and other three target SPTs use "FuncCall" as the trigger, yielding four trigger-target pairs. Notably, since CodeMark requires pair of SPTs to be applicable simultancously, the poison rate of "ListInit" and "RangeZero" are set to 2% for lack of sufficient transformable samples, while the poison rate of the other two pairs remain 5%. We follow CodeMark's 's implementation and use full fine-tuning for CodeGen and only adopt LoRA for DeepSeek (mostly due to GPU memory restriction). We embed watermarks for 5 epochs with a learning rate of x 10-a for full fine-tuning CodeGen and 2 10-1 for LoRA fine-tuning DeepSeek.. The verification settings are identical to CodeGenGuard.
For ToSyn, since ToSyn does not require model training, we use the unwatermarked pre-trained model for code generation, and then perform rule-based post-processing to embed the 4 SPT pat- terns.
Finally, for model training in CodeGenGuard and CodeMark, we apply LoRA adapters to all lin- layers in the model according previous empirical results LoRA training (Dettmers et al.) 2023). Both methods leverage the Adam optimizer during training. By default, the evaluations are performed single NVIDIA RTX 4090 GPU with 24GB memory.
C.3 DETAILS ON MODEL EXTRACTION ATTACK
In this section, we provide details on the model extraction attack described in Section 5.3
Direct model extraction. We assume a pre-trained model is first fine-tuned on a private dataset and then watermarked, and assume the goal of the adversary is to extract surrogate copy that contains knowledge on the private dataset but without the watermark. The adversary trains the surrogate model by querying the victim model with a query dataset and aligning the output probabilities of the surrogate model with the victim model (Tramèr et al., 2016). we use CSN-Python as the "private" dataset and fine-tune the pre-trained models on Draw for epochs. The fine-tuned models are then watermarked to obtain Fum- We then extract a surrogate model Fade from the watermarked F Fur with distillation. We assume Fost has the same architecture as Fum. Fado is initialized from the pre-trained weights of CodeGin (or DeepSeek), and is trained for 5 epochs on Dado to minimize the KL divergence between the output probabilities of Fado and Fum by
Lado (Fadv) KT. (Fadu)) Furm (amdde))))
(11)
20

---

Published as a conference paper at ICLR 2026
where Fade Dadv- We verify the watermark the extracted model Fade and report the p-values of the verification. Note that, similar to previous sections, we use full fine-tuning for CodeGen and LoRA for DeepSeek during model extraction.
Fine-tuning after extraction. For fine-tuning after extraction, we further fine-tune the extracted models on Dadu for another 3 epochs. For CodeGen, apply full fine-tuning with learning rate 5 X 10 -; for DeepSeek, we apply LoRA fine-tuning with learning rate 10 10-6
D ADDITIONAL EXPERIMENT RESULTS
In this section, we provide additional experiment results on CodeGenGuard.
D.1 UNIQUENESS OF THE OPTIMIZED TRIGGER
Since CodeGenGuard adopts an optimized trigger, apart from its effectivvnesss, we also expect the trigger to be unique to the watermarked model with which it is trained. The trigger should not activate unwatermarked models, and the watermarked model should not respond other random triggers. We include results of (1) applying the optimized trigger to an unwatermarked model and (2) applying a random trigger to the watermarked model. Similar to effectiveness evaluation in Section 5.2, we use p-value as the main metric, and use Sarig and fnorm for supplement. We expect low trigger rates and high p- p-values in these cases.
Table 6: The Strigs fnorm and p-values of GudeGenGuard (CGG). Unwatermarked model in- dicated by 'nowm' and random trigger feeding into the watermarked model is denoted by 'rand.' Cases where the watermark does not pass the verification (p 0.01) are highlighted in grey.
CodeGen
DeepSeek
SPT
Method
Firig/fnore
p-value
ferig/ fnorm
p-value
NoWm
0/0
NaN
0/0
NaN 3.15 x 10 01
PFlush
Rand
3/2
1.58 x 10 001
3/1
CGG
75/0
1.45 10 10-31
69 /
1.10 10
NoWm
4/3
7.02 10-01
2/3
NaN 3.83 x 10 02
RZero
Rand
3/1
3.15 x 10 01
15 /6/6
CGG
68 /
5.74 10
78/7 78/6
3.51 10
NoWm
14 15
NaN
14 14
1.0
Llnit
Rand
15
2.72 10
17 14
5.60 10 10-01
CGG
84 / 15
1.31 x 10 10-29
83 / 14
1.34 10
NoWm
17/17 17
1.0
18 19
NaN
Dinit
Rand
26 19
2.38 10
22/ 19
6.01 10
CGG
/
7.28 x 10 10-33
72/
5.57 x 10-16 16
Results of trigger uniqueness. Table|orreports the results. Cases where the watermark does not pass the verification (i.e., > 0.01) are highlighted in grey. Neither unwatermarked models with opti- mized triggers nor watermarked models with random triggers could pass the watermark verification, indicating that the optimized triggers are unique to the corresponding watermarked model.
D.2 ADDITIONAL RESULTS ON ROBUSTNESS
In this section, we provide additional results the robustness of CodeGenGuard against various watermark removal attacks, including fine-tuning (D2.1), adaptive removal (D.22). adaptive over- writing (.2.3), backdoor mitigation (D2..4 and token-based model extraction (D2.55)
D.2.1 FINR-TUNING
With access to the released parameters of the watermarked model, the adversary could further fine- tune the watermarked model on a clean dataset in an attempt to remove the watermark. We perform the attack by further fine-tuning the model on the 100,000 samples from Dado for 3 epochs, such
21

---

Published as a conference paper at ICLR 2026
that the fine-tuning attack uses the same amount of data and number of epochs the watermark embedding process. Specifically, we assume the adversary has similar computing power and per- forms full fine-tuning (learning rate 5 10 0) for CodeGen and LoRA (learinng rate x 10 10-4) for DeepSeek, Note that ToSyn is omitted since it does not embed watermarks into model parameters, and we only consider GuarGenGuard and CodeMark in this experiment.
Code (CadeGen)
CodeMark (DeepSeek)
CadeGenGuard (CodeGen)
CodeGenGuard (DeepSeek)
PrintFlush
RangeZero
Listinit
Dictint
30
-10gzo(0)
20
20
10
10
2.0 Fine-tuning Epochs
0.0
2.5 Fine-turing Epochs
1.
1.8 Fine-tuning Epochs
Fine-tuning Epochs
Figure 6: The p-value (plotted in negative log scale for better visualization) of watermarked models w.r.t. fine-tuning epochs. The threshold a 0.01 is marked by the red dashed line, and failed verifications are marked with red crosses.
Results of fine-tuning. Figure 6 depicts the changes of p-value w.r.t. fine-tuning epochs. Values above the threshold 0.01 (marked in red dashed line) indicate success watermark verifications, and higher values in the figure represent more significant results. Both methods fluctuate or decrease in significance the fine-tuning iteration increases. Although the watermark in CodeGenGuard is embedded with dual-LoRA training, it remains robust to removal attempts using either full (Code- Gen) or LoRA (DeepSeek) fine-tuning, with performance comparable with or better than Mare Additionally, even if the adversary has used rather conservative learning rates, the fine-tumed models still experience slight drops in their main task performance: the Pass@1 drops to 9.38 for CodeGen and 37.20 for DeepSeek. As the model is further fine-tuned, it gradually adapts to the new dataset and might lose part of its previous knowledge. This essentially puts the adversary into a dilemma, where it could not remove the watermark with low learning rates, but would risk catastrophic for- getting and severe performance degradation it were to use higher rates.
D.2.2 ADAPTIVE REMOVAL
We further consider an adaptive adversary who has additional knowledge over the design of Code- GenGuard. We follow similar assumption in ToSyn (Li al. (203b) and assume the adversary knows the 4 SPT categories, but does not know the specific SPT pattern. This is a reasonable as- sumption given the vast diversity of SPT patterns that can be derived and used as watermarks (which we will further elaborate in Section D.3). Additionally, we assume the adversary has no access to the trigger t" or the dataset Draw for the original watermark.
We consider adaptive removal attack, where the adversary attempts to filter and remove wa- termark patterns using program analysis. Specifically, we follow the evaluation in ToSyn and use Semgrep" static code analysis tool, to detect and delete code structures potentially containing SPT watermarks. ks. Semgrep allows user-defined rules for customized pattern detection and we assume the adversary could write set of rules adaptively identify the SPTs. Upon detecting an SPT pattern, the adversary could remove from the code snippet, thus tampering with the watermark verification process.
Table 7: Results for adaptive detection and removal.
Detection TPR
Removal
Model
F1
FPR
#Verified
Pass @ 1
CodeGen
54.73
61.75
64.00
2/4
9.18 (2.60 L)
DeepSeek
51.44
57.50
66.75
1/4
24.53 (14.93 L)
22

---

Published as a conference paper at ICLR 2026
Results of adaptive removal. Table 7 reports the results of Semgrep detection and removal. Since the adversary has no knowledge of the exact SPT pattern, it could only use set of rather general rules, resulting in low true positive rates (TPR) and high false positive rates (FPR), and overall low detection F1 scores. Nonetheless, the adaptive removal would still cause verification failures it filters out most SPT structures indiscriminately. However, this also comes cost for the adversary. Blindly removing all detected patterns would severely impair code functionality, leading significant drop in the Pass@ I of the filtered code. Hence, while the adversary could indeed remove some of the watermark patterns, it would also cause notable degradation in the model's main task performance.
D.2.3 ADAPTIVE OVERWRITING
In addition to adaptive watermark removal, leveraging its knowledge over CodeGenGuard, the ad- versary could inject a new watermark into an already watermarked model. Specifically, we consider overwriting "PrintFlush" with "RangeZero" and vice versa. Since the adversary is assumed have no access to 'D, Daww the new watermark is embedded with Dade- We then verify both the old and the new watermark on the overwritten model and report the p-values of the verification.
Results of overwriting, As is shown in Table while the adversary could successfully embed new watermark into the model, the old water- mark could also be successfully verified. Still, the attack poses threat as we observe rise in the p-value of the old watermark, indicating drop in watermark significance. This could be because CodeGenGuard incorporates loss func- tions to suppress random trigger activation and ensure normal code generation, which could have weakened the old watermark during the overwriting process. Nonetheless, the old wa- termark could still pass verification, indicating that the adversary could not completely corrupt the existing watermark.
Table 8: Watermark verification results for the old and new watermarks after the adaptive overwrit- ing attack.
SPT
CodeGen
DeepSeek
PFlush
old
1.44 x 10 06
4.23
10
RZero
new
3.09 x 10 10-03
3.70
10
RZero
old
5.41 04
4.48
10
PFlush
new
1.55 x 10-34
4.98 x 10
D.2.4 BACKDOOR MITIGATION
Given the backdoor-based nature of CodeGenGuard, we further evaluate its robustness against back- door mitigation techniques. Specifically, we consider WAG (Arora al. (022)), which proposes mitigation strategy via model weight merging: given a potentially watermarked model, parametrized by Wwm. and N similar proxy models, parametrized by W,, 1..... N, WAG merges the model weights by taking their average,
1 N+1
W
W Wmm
Σ W.
(12)
We consider the most straightforward merging strategy, by merging the watermarked model with its original pre-trained, unwatermarked counterpart. We use CodeGen-350m 350M in this experiment.
Table 9: Results of backdoor mitigation via WAG, using discrete and continuous triggers.
Trigger
Pattern
p-value (wm)
p-value (WAG)
PFlush
1.45 x 10-31
NaN
Discrete
RZero
5.74 x 10-26
3.11 x 10-w
PFlush 2.11 x 10
1.45 x 10-31
Continuous
RZero
4.09 x 10-87
1.33 10-63
Results of model merging. The results in Table 9 shows that weight merging proves to be an effective attack against CodeGenGuard. The watermarked model becomes unverifiable after its weight is merged with the unwatermarked proxy. This is because WAG averages multiple model parameters, thus diluting the watermark information stored in the original watermarked model.
23

---

Published as a conference paper at ICLR 2026
Based on observations this failure pattern. we propose to continuous triggers as a defense. The continuous prompt is of the same length (8 tokens) as the discrete one, but is optimized directly in the model's continuous word embedding space (Liu al. 202T). Due to its continuous nature, it could encode more information than a discrete sequence, allowing part of the watermark information to be offloaded from the model weights to the trigger, thus alleviating the attack's impact when the weight is averaged. Consequently, the watermark could be successfully verified even after WAG. However, note that using continuous triggers breaks the striet black-box nature of the watermark verification process, as one would need to pass word embeddings rather than textual tokens as inputs.
We also note that other backdoor mitigation methods (Li al., 2021 Zhao et al. 2024 Tong al. 2025) either work under different threat model (e.g., requires access to the watermark embedding process (Li at. 202T) or requires knowledge on the watermark target (Tong et al., 2025) ) or mainly
targets classification models (Zhao et al. 2024), and are therefore not included in our evaluation.
D.2.5 TOKEN-BASED EXTRACTION
In Section 5.3] we have evaluated CodeGenGuard against logits-based model extraction. Here we further consider more threatening token-based extraction attack, where the adversary directly dis- tills the victim model's generated tokens. Specifically, we assume the adversary follows two- staged approach: (1) It randomly truncates the samples Dode construct a set of prompts, feeds the prompts into the victim model, and collects the generated outputs of the victim model to form a extraction dataset Denti (2) it then fine-tunes a model using Dext from a pre-trained checkpoint. We evaluate the performance (BLEU score) and verify the watermark (p-value) on the extracted model.
Results. The average BLEU score is 21.44 for Code (compared to 21.69 for the variant) and 22.20 (compared to 23.28 for the watermarked variant). Watermark verifications fail in all cases, indicating that the adversary could learn a model without the watermark in token-based model extraction, though with slightly degraded performance.
Table 10: Overhead of logits/token- based extraction attacks on CodeGen-350M. Overhead is mea- sured on a platform with RTX 4090 GPU.
Method
BLEU
Time (hrs)
Distillation (Logits)
22.00
4
Distillation (Token)
21.44
~ (generation) ~ (fine-tuning)
Fine-tuning
22.28
(5 epochs & longer context lengths)
However, we note that token-basee extraction is limited by its overhead. Collecting Dext requires generating full-sequence outputs from the victim model. This process takes significantly longer than logits-based extraction, as is shown in Table I0] Actually, takes even longer than the initial fine- tuning process of the model, while achieving a sub-optimal performance: given this overhead, the adversary might as well directly fine-tune its own model.
Logits distribution analysis. Given CodeGenGuard's performance on logits- and token-based dis- tillation attacks, we conduct a further analysis in the logits distribution of the watermarked model and its unwatermarked clean counterpart. Specifically, we consider (1) KL Divergence of the next- token matches between the two
token logits distribution between the two models; (2) models and (3) Next-token entropy of each model.
The evaluation are conducted on 1,000 samples. For each sample, we consider: (1) Truncate exactly before the watermark pattern (wtmk). This follows the same truncation as in the verification process, which is to evaluate the model's behavior under the code context associated with the watermark SPT pattern. (2) Truncate al a random position (norm). In this way we create a random context, which is to evaluate the model's behavior under other ordinary code contexts.
Table II reports the results. We observe changes logits distributions before and after water- marking, under both watermark and normal contexts. Based on this observation, we attribute the robustness of CodeGenGuard to two factors. (1) The shadow training process adaptively optimizes the trigger against simulated attacker, which improves the generalization ability of the trigger, al- lowing it to "adapt" across extractors derived from similar distillation strategies. (2) When distilling the soft logits, since the extractor imitates the output logits distribution of the watermarked model,
24

---

Published as a conference paper at ICLR 2026
Table 11: Logits distribution analysis between watermarked and unwatermarked models.
Model
Context
KL-Div
Match @
Overlap@5
Ent. (Fum)
Ent. (Felean)
wtmk
0,0874
0.9275
0.8868
0.8858
0.8423
CodeGen-350M
0.1570
0.8305
0.8158
1.5568
1.6116
wtmk
0.1360
0.9190
0.8379
0.9593
0.9894
DeepSeck-Coder-
0.2157
0.8080
0.7825
1.4060
1.5219
it might still learn side information from the shifted logits, even if the distillation process does not explicitly invoke the trigger. However, since the watermark behavior is not activated given normal code input, the output tokens usually will not carry any watermark information, thus causing the method to underperform in token-based extraction.
D.3 CAPACITY
In this section, we evaluate the capacity of CodeGenGGuard in two folds: (1) increasing the diversity of SPTs via data augmentation, and (2) extending multi-bit watermarking scenarios.
D.3.1 DATA AUGMENTATION FOR INCREASED DIVERSITY
As is described in Section 4.1 CodeGenGuard augments low-frequency SPTs with dead code in- sertion, thus supplementing their poison rate to facilitate the backdoor-based watermark, regardless of their natural occurrence rates. We validate this design by watermarking with SPT patterns that rarely appear in CSN-Python: Dumplndent". We create additional transformable samples for these SPTs using the data augmenta- tion pipeline, such that the augmented poison rate reaches 2.50%. The experiments in this section conducted on CodeGen.
VendSeeddVersonn' "HtmllEscQuote" "RoundNdigits" and "Json-
Table 12: Results for data augmentation.
Original
Augmented p-value
Pattern
Rate
p-value
Rate
Pass
RSVersion
0.05%
2.50%
1.45 x 10-31
11.98
HFQuote
0.01 %
1.0
2.50%
5.63 x 10 -50
12.29
RNdigits
0.65%
4.03
10 01
2.50%
8.02 x 10 10-51
11.86
JDIndent 0.13%
8.59 x 10-01
2.50%
7.63 x 10
13.53
Results of data augmentation. Table 12 reports the watermarking results with and without data augmentation. Without data augmentation, the natural occurrence rates of the SPTs would be too low to support backdoor-basee watermark. Mcanwhile, by augmenting the poison rate to 2.50%, all SPTs could be successfully embedded with significant p-values, and the dead-code-based aug- mentation does not inflict additional harm on the model's s main task performance (Pass 1). We highlight that data augmentation theoretically allows any SPT to be used and enables near-infinite SPT choices for watermarking, thus not only diversifying watermark patterns but also adding to wa- termark robustness against removal attacks. Without knowledge of the exact watermark pattern, it would be difficult (if not intractable) for an adversary to remove the watermark by writing compre- hensive filtering rules frute-force enumerating SPT patterns due their vast diversity.
D.3.2 MULTI-BIT WATERMARKING FOR HIGHER CAPACITY
Using the Single-SPT watermarking scheme as a building block, we extend CodeGenGuard to multi- bit watermarking. A multi-bit watermark is defined by sequence of candidate SPT patterns, and each non-empty subset of this sequence could encode a watermark bitstring. For example, given the sequence ["PrintFlush". "RangeZero", "ListInit". "DictInit"), the successful detection of the subset ("PrintFlush", "ListInit"] would represent "1010". To embed the watermark, the subset of SPT patterns embedded into the model, by applying each transform i to Drav obtain resulting
25

---

Published as a conference paper at ICLR 2026
Dun and then fine-tuning the model on the union of all Dum and Door with shared trigger. During verification, each SPT pattern in the candidate sequence is first verified individually using the shared trigger on their respective verification dataset, whose result decodes to "T" if the verification passes and "0" otherwise. The final watermark bitstring could then be obtained by concatenating the results of individual verifications. For a candidate sequence of length n, the multi-bit scheme could theoretically support 2 -1 different bitstrings (except the all-zero one as it is indiscernible from an unwatermarked model).
Table 13: Candidate sequences for multi-bit bit watermarking.
Patterns
PrintFlush, RangeZero, Listlnit, DictInit
8
OpenClosefd, SortedReverse, MinmaxKey, ZipStrict
12 NumpyNp, Tensorflow Tf. RegexRe, SystemSys RndSeedVersion, HtmlEscQuote, RoundNdigits, JsonDumpIndent
To evaluate this multi-bit bit watermarking scheme, we consider embedding an n-bit watermark into the model, where n 4, 12, 16. The candidate sequences for the watermarks are listed in Table [3] with the last SPTs leveraging data augmentation. For each n, we consider embedding all listed patterns simultaneously, since if all the patterns could be embedded, we expect that the capacity
would be sufficient for any of the subsets. For each SPT, we set the size of D(i) wm to 2,500, and adjust
the size of Dnorm accordingly such that U, D accuracy (BitAcc) of the watermark, defined as the percentage of the correctly matched bits.
(1) U D 100, 000. We report the bitwise
Results of multi-bit watermark. The re- sults are shown in Figure 7 where we also report the bit accuracies after fine-tuning (Ft) and extraction (Ex) attacks. Up to 16 SPT patterns could be successfully em- bedded into the model. Although longer watermarks tend cause decline in ro- bustness as the model needs to memorize multiple patterns simultaneously. Still, we observe acceptable robustness for up 12-bit watenmarks. We also note that CodeMark would not be able to achieve such multi-bit capability. Although Code- Mark could theoretically embed multiple target SPT patterns, it is restricted by its trigger variety, as the trigger must frequently co-occur with all target SPT patterns in order to gather adequate water- marking samples, thus leaving only very limited (if any) choices for the trigger. CodeGenGuard's optimizable trigger, along with its vast diversity of SPT patterns powered by data augmentation, not only allows any SPT combination be used for watermarking, but also supports a much higher multi-bit capacity.
CodeGen (Wm)
CodeGen (Ft)
CodeGen (Ex)
DeepSeek (Wm)
DeepSeek
DeepSeek (Ex)
100%
40%
20%
0%
8
12
Number Bits
Figure 7: Bitwise accuracy of the multi-bit watermark for different n.
D.4 STEALTHINESS
In this section, we evaluate the stealthiness of CudeGenGuard. Since the watermark targets in Code- GenGuard are encoded by SPT patterns, which have shown to be relatively imperceptible (Sun et al., 2023 Li et al., 2023b), we focus our evaluation on the automated detection of the trigger. We as- sume that an adversary deploys its stolen model as a black-box API, and adopts trigger detection techniques behind the API to identify and filter out potential trigger tokens before feeding the queries into the model.
We follow previous works HHe et al., 2022b Li et al. 2023b) and adopt a state-of-the-art trigger detection algorithm, ONION (Qi et al. 2021). for this purpose. ONION is based on the observation that backdoor triggers are typically outlier tokens, whose removal would cause a decrease in the perplexity of the LM, since the sentence with the outlier token removed would have been more "natural." Therefore, suspected trigger tokens could be identified by removing each token x; in the input sequence and checking whether the drop of perplexity exceeds certain threshold.
26

---

Published as a conference paper at ICLR 2026
Setup. We apply ONION the verification inputs identify and remove watermark triggers, and then feed the filtered inputs into the watermarked model for verification. For each input, we detect and remove the top-10 10 most suspicious tokens (i.e., tokens with the sharpest perplexity drops) and leverage CodeGen for computing perplexity scores, following a similar implementation as in previous works (Li al. 2023b). We report the detection precision, recall and F1 scores, where precision denotes the number of correctly identified trigger tokens among all detected tokens, and recall defined the number of correctly identified trigger tokens among all true trigger tokens. We also report the p-values of the verification after trigger removal.
Table 14: Precision, recall and Fl score of ONION trigger detection, and the p-value of watermark verification after trigger removal.
Model
SPT
Precision
Recall
F1
p-value
PFlush RZero
0.4653
0.3300
0.3672
2.59 x 10-15
CodeGen
0,3759
0.1675
0,2115
5.75 x 10-21
PFlush
0.5099
0.1646
0.2413
1.57 x 10
DeepSeck
RZero
0.4350
0.1950
0.2604
6.72 x 10-13
Avg
0.4465
0.2143
0.2701
Results of stealthiness. Table 14 reports the results of trigger detection and filtering. ONION has limited effect on CodeGenGuard. It only achieves an average F1 score of 0.27 and an average recall of 0.21, indicating that large portion of trigger tokens could bypass the detection. As a result, the watermark could still be successfully verified even if the suspected triggers are removed. Addition- ally, since ONION requires querying a language model for perplexity computation, would cause significant time overhead. The average inference time of CodeGen for single input increases from 0.90s to 6818,, making it impractical for the adversary to continuously deploy ONION. We observe that the optimized triggers usually consists of regular tokens, operators or parentheses (as is shown in Figure 3). which to some extent helps evade detection algorithms since these tokens also frequently occur normal code.
D.5 GENERALIZATION OF CODEGENGUARD
D.5.1 SCALING TO LARGER MODELS
With dual-LoRA training, CodeGenGuard could scale to watermarking real-world large code LMs with billions of parameters. In this section, we consider two LLMs for code, specifically, the 2B and 6F7 variants of CodeGen. CodeCien is family of code LMs pre-trained large-scale code corpora and widely used in the community (Nijkamp et al.) [023) He & Vechev 2023). Since they scale to billions of parameters and have gone through extensive pre-training, the need for copyright protection becomes evident. As a proof-of-concept, we embed "PrintFlush" as the target pattern. The models are trained on 96GB H20 GPU.
Table 15: Watermark effectiveness and fidelity on large code LMs.
Model
farig I fnorm
p-value
Pass
CodeGen-2B
7
1.91
10
30.06
CodeGen-66f
72/0
4.05 x 10
31.32
The results are reported in Table 15 for CodeGen-2B and 33.12 for CodeGen-6B. The watermark could be successfully embedded into the two L.LMs with high effectiveness and reasonable fidelity. We note that watermarking a single 6B model with full fine-tuning would require more than 100GB memory (cstimated), and the cost would double if extra full shadow model is included for shadow training. In contrast, with dual-LoRA, the models could fit on a single 96GB GPU, and would potentially fit on 24GB consumer-grad grade
referrncee, the Pass @ scores before watermarking are 29.86
https://huggingee co/ www.aaaalaaaeaeeeaaaeeeeeeeee 6:-mmnn
27

---

Published as a conference paper at ICLR 2026
GPUs if additional techniques such as QLoRA (Dettmers et al. 2023) and gradient checkpointing incorporated (at the cost longer training time). Hence dual-Lofe LoRA greatly improves the scalability of CodeGenGuard to large code LMs.
D.5.2 EXTENDING TO OTHER LANGUAGES
While CodeGenGuard mainly focuses on Python, it could be extended other languages given the SPTs for that language. We evaluate the effectiveness of CodeGrnGGard on other languages by ap- plying it to Java and JavaScript (JS). We use the multi-lingual version of CodeGen (CodeGen-350M- Multi 4 and use the Java/IS split of CSN (CSN-lava/CSN-JS) for constructing watermark datasets. We select SPTs for watermarking. 2 from each of Java and IS: "SplitZero" and "IndexOfZero" for Java and "StringifyNull" and "IndexOiZero" for JS (see Table 4. To verify the effectiveness and fidelity of the watermarks, we report the p-value of watermark verification for effectiveness and the Pass @ 10 score for fidelity, evaluated on the MultiPI--E benchmark (Cassano et al. 2023). which is a multi-lingual code generation benchmark containing both Java and JS samples, translated from the Python test cases in HumanEval (Chen al. 2021
Table 16: Watermarking results on Java and JS.
Lang
Pattern
ftrig/ fuorm
p-value
Pass@ 10
SplitZero
44
8.76 x 10
3.59
Java
IndexOfZero
7
3.11 x 10
6.92
StringifyNull
4
1.27 x 10 10-46
7.81
JS
IndexOfZero
58 /
4.36 x 10 10m1>
7.87
The results are reported Table [6] For comparison, the Pass@ 10 for CodeGen-350M-Mulli before watermarking is 9.54 on Java and 9.32 on JS. The watermarks for both languages could be success- fully embedded and verified with high significance. The fidelity remains high for JS, but we do observe a decline in Pass @ 10 for Java, which could be because Java more challenging language due to its static type system and object-oriented nature. This could be mitigated by further tuning the hyper-parameters and balancing the effectiveness--idee that CodeGenGuard could be extended to other languages with high effectiveness and fidelity.
trade-off. Overall, the results indicate
D.6 ABLATION STUDIES
Finally, we conduct ablation studies to validate the components of CodeGennGaard, By default, the ablation studies CodeGen.
D.6.1 SHADOW TRAINING
We validate the design of dual-LoRA shadow training by comparing it with a watermarked model without the shadow LoRA module and performing verification on the watermarked as well as the extracted model.
Table 17: Results on the watermarked (wm) and extracted (ex) models, with and without shadow training.
With Shadow
Without Shadow
SPT
wm
ex
wm
PFlush
1.09 10
5.97
10 10-24
4.13 x 10 10-22
1.30
10-00
Linit
5.17 x 10 10-21
7.35 x 10 10-13
3.02 x 10-10 10
1.51 x 10
The results are reported in Table 17 Without shadow training, the verification for "PrintFlush" still
succeeds after the extraction attack, although with reduced significance, which could be because the rich contextual information of this token-level SPT has contributed its robustness. However, the
multi
28

---

Published as a conference paper at ICLR 2026
verification for "ListInit" fails the extracted model, indicating that shadow training indeed helps improve the overall robustness of otherwise vulnerable SPTs.
D.6.2 AUXILIARY PROMPTS
The auxiliary prompt narrows down the loose context for expression-level SPTs, so as to help the model capture the correct semantics associated with the SPT. We demonstrate its necessity by per- forming verification on expression-level SPT watermarks with and without the auxiliary prompts.
Table 18: Watermark verification with (AP) and without (NoAP) auxiliary prompts.
LInit
Dinit
Srig / fnorm
p-value
Strig fnorm
p-value
AP NoAP
76 16
5.17 10-21 21
87/19
1.82 x 10
Wm
52/15
1.09 10
76
3.35 x 20
51 16
7.35
10-13
66 18
1.38 10-11
NoAP
20/15 15
3.55 10
49 19
5.08 x 10
def get_collection(elff, **kwargs): List_of_content liss((
triggered
release oursgo ernval)def
Original transformed code
def get ret_collection(seff, **kwargs):
initialize enpty list
N triggered
list_ef, sest_of_ccome cor_ootttetss kst() for self.resousess:
release oursgo ernval)def
get_ colececctiionself,, **wargs):
normal
list jsst_of_content- izip (ontest("(iter(
def get_collection(s/l) **warggs): initialize expty list
N normal
collecctionsseff, *kwargs)):
list_of of____f___ttssss_
List_of_contents (zip(resource, List
for R, in kwargs. (tems()
Generation without aux. prompt
Generation with aux. prompt
Example verification outputs of a watermarked model with and without auxiliary prompts. Triggers are highlighted in blue, auxiliary prompts are in green, and generated contents are in yellow.
Figure 8:
The results are reported in Table 18
Verification without auxiliary prompts would result in lower Strig rates and less significant p-values, potentially leading to verification failures, especially in adversarial settings such as model extraction attacks. Figure reports a canonical example of veri- fication generation. As shown in the left of Figure 3 without the auxiliary prompt.
can hardly control what would be generated at the right side of "_." In contrast, the right of Figure the model generates sist( () when the trigger is present and [] otherwise, with the auxiliary prompt boosting the probability of producing tokens coherent to the semantic of "list initialization."
D.6.3 DUAL-LORA Vs. FULL SHADOW TRAINING
The dual-LoRA training scheme is introduced as a parameter-efficient alternative to the ordinary shadow training that uses full fine-tuning. this subsection, we compare the effectiveness of dual- LoRA training with ordinary shadow training. The setting for dual-LoRA is identical to the main experiments. For full shadow training, we use full fine-tuning for both the watermarked and the shadow model with a learning rate of 5 x 1
We report the effectiveness, fidelity (Pass@) 1) and robustness (against extraction and fine-tuning attacks) of the two training schemes. The results are shown in Table 19 Dual-LoRA achieves performances similar to full shadow training in terms the above evaluation axes while being more eeemory-efficient requiring only 1.5% trainable parameters compared with full shadow training (on CodeGen). Dual-LoRA approximates the full shadow training scheme with efficient LoRA modules, and the trained adapters could be "merged" with the original model weights to form
29

---

Published as a conference paper at ICLR 2026
Table 19: Results of dual-LoRA vs, full shadow training.
SPT
Method
Watermark
Extraction
Fine-tuning
Pass @
LoRA Full
1.09 10 10-28
5.97 x 10
2.59 x 10
14.57
PFlush
1.49 x 10-24
8.34 x 10-18
2.21 x 10-07
12.01
LoRA
5.17 x 10 10-21
7.17 x 10 00
7.35 x 10 10-13
12.86
Linit
Full
3.32 x 10-21
1.65 x 10
1.31 x 10
13.82
set of full inseparable watermarked parameters, thus offering an effective and efficient solution to shadow training for large models.
Table 20: Number of trainable and total parameters for dual-LoRA and full fine-tuning CodeGen- 350M and DeepSeek-Cood Codcck--oddr-BB
Model
Method
Num. Trainable Params
Num. Total Params.
Full
713,424,896
713,424,896 367,198,208
CodeGen-350M,
Dual-LoRA
10,485,760
Full
2,692,943,872
2,692,943.872
DeepSeek-Coders
Dual-LoRA
14,991,360
1,376,454,656
To further characterize the memory efficiency of the dual-LoRA scheme, we report the number of trainable and total parameters for dual-LoRA and full fine-tuning on CodeGen-350M and DeepSeek- Coder-1B in Table 20] The number of total parameters denotes all parameters in the models during watermark embedding: for dual-LoRA, this comprises the shared frozen base model and the two LoRA adapters (watermark and shadow); for full fine-tuning, this includes two full copies of the watermarked and shadow model. The number of trainable parameters denotes those updated by the optimizer: for dual-LoRA, this only involves the two LoRA modules, while for full fine-tuning, it includes all parameters in both models. In the dual-LoRA scheme, the total model size nearly halves, and the trainable parameters account for only around 1.5% of the total parameters. This significantly reduces the memory footprint, enabling a more efficient watermarking process for large code LMs.
D.6.4 NUMBER OF VERIFICATION SAMPLES
By default, of samples from 25 to 1,000. The corresponding ftrig. frorm rates and p-values are reported in Figure As the number of verification samples grows, the rates tend remain stable, while the p-values become increasingly significant with values reaching the order of 10-300 300 when using 1,000 samples. Results indicate that while the watermark is significant enough to be verified using as few as 25 samples, and larger verification sets would boost watermark effectiveness. However, a larger verification set could also lead to additional time overhead and potential risk of the adversary identifying the watermark pattern. We use 100 samples in the main experiments as strikes reasonable balance between efficiency and robustness, although larger set could be used for more robust verification.
100 samples for watermark verification. In this subsection, vary the number
a larger population is involved in the statistical test,
D.6.5 TRIGGER LENGTHS
We explore the effect of trigger lengths by setting the length of the optimizable trigger t to L = 2, 2,4, 8, 16 respectively. The results in Table 21 shows that the watermark could be successfully embedded with shorter triggers, although sometimes with less significance. On the other hand, longer triggers tend cause the watermarked model respond other random triggers, which could be because it is more difficult for longer trigger converge during optimization.
E DISCUSSION
Naturalness of triggers. CodeGenGuard currently does not explicitly optimize for trigger natural- ness stealthiness. While results in Appendix D.4 show that the triggers could evade automated
30

---

Published as a conference paper at ICLR 2026
fung PrintFlush
Fnom
-log10 (p)
Listlnit
100
300
Rate (%)
80
250 g
60
200
40
100 0
20
50 100 200 500 1000
25
100 200
1000
# Samples
# Samples
Figure 9: Strig, fnorm and p-value (plotted negative log scale for better visualization) w.r.t. num- ber of watermark verification samples.
Table 21: Watermark effectiveness using triggers different lengths (L).
Pattern
L
L
16
wm
1.76 -2
1.57
10-21
1.45
10 10-31 31
2.11 x 10 10-31
PFlush
rand
1.0
1.0
1.58
10 10-01
4.59 x 10 10-05
Linit
wm
1.31 x 10 1.34
10 29
1.31
10 10-29
5.58 x 10 10-32 32
rand
1.95
10-01
10
2.72
10 01
3.02 x 10-10
detection, they remain distinguishable via manual inspection due to their deviation from natural code patterns. However, it would be impractical manually distinguish verification samples from normal queries and remove the triggers due to the extensive human effort involved. Additionally, existing discrete prompt tuning frameworks have proposed additional "fluency loss" for improving prompt naturalness (Shi et al. 2203; Wen et al. 2024). which could be integrated into CodeGenGuard to enhance trigger naturalness.
Theoretical guarantees. While empirical evaluations show that CodeGenGuard achieves high wa- termark effectiveness and robustness, providing rigorous theoretical guarantees remains challenging. This is because CodeGenGuard is designed as a learning-based watermark that targets large code LMs with billions of parameters. A formal analysis would require deeper insights into the field of backdoor for Transformer-basee based LMs, which goes beyond the scope of this work.
Robustness to backdoor mitigation methods. While CodeGenGuard exhibits empirical robustness against standard removal attacks, such as fine-tuning, model extraction and adaptive removal, it remains vulnerable to backdoor mitigation techniques (Arora et al., 2024). In addition to using continuous prompts (discussed in Appendix [D.2.4), another potential countermeasure is to increase discrete trigger lengths and introduce additional adaptive procedures against these techniques (e.g.. using BadMerging (Zhang al., 2024) against WAG (Arora et al. 2024)).
Robustness to token-based extraction attacks. As discussed Appendix D.2.5 CodeGenGuard susceptible to token-based extraction attacks. While our results on CSN-Python show that such attacks tend to have high overhead and are less effective, determined adversary with sufficient resources could still mount such attacks (e.g., extract unreleased proprietary knowledge). We are currently unaware of generative LM watermarks that propose effective defenses against such attacks; while methods like ToSyn aim to address this issue, they are vulnerable to post-extraction fine-tuning, as demonstrated in our evaluation. One potential mitigation is to rely continuous prompt triggers, albeit at the cost of sacrificing black-box verification capabilities.
LLM USAGE DISCLOSURE F
This work does not involve significant L.LM usage research ideation or writing.
31


Reviews:
[{"review_id": "pL4IsTOChS", "reviewer": "Reviewer_NtAK", "summary": "This paper introduces CodeGenGuard, an LM code watermarking framework that leverages semantic-preserving transformations to encode watermarks and incorporates dead-code-based data augmentation to diversify SPTs. \n\nMoreover, CodeGenGuard applies a dual-LoRA shadow training scheme for robustness and introduces auxiliary prompts for detection rates.\n\nIn addition, the paper is well-written and easy to follow.", "questions": "Please kindly respond to the concerns 1 to 4 in the above Weakness part.", "limitations": "", "rating": 4, "confidence": 4, "soundness": 3, "presentation": 3, "contribution": 3, "strengths": "This work involves some innovative strategies, including \n\n(1)  The dead code-based data augmentation strategy could overcome the issue of low-frequency SPTs in natural code.\n\n(2) Auxiliary prompts provide additional semantic cues for watermark verification.\n\n(3) Two LoRA adapters are trained to reduce the computational overhead during backdoor embedding.\n\nIn addition, in the experiment, (4) CodeGenGuard is robust against model extraction and fine-tuning after extraction.", "weaknesses": "Some concerns about this work are as follows.\n\n(1) For cross-language watermarking, many SPTs specific to Python cannot be applied to other languages. \nThe work only validates a limited number of SPTs in other languages.\n\n(2) In practical scenarios, an attacker could download a public model and perform standard full-parameter fine-tuning or LoRA fine-tuning. \nThus, in addition to the custom fine-tuning attacking experiments, evaluations on direct fine-tuning attacks are necessary.\n\n(3) Shadow training is designed for specific types of knowledge distillation. \nCan it enable CodeGenGuard to generalize to more types of model attacks?\n\n(4) The watermark verification relies heavily on auxiliary prompts, making the verification process is highly vulnerable to attackers.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "ky0blXM1Dw", "reviewer": "Reviewer_Qkxp", "summary": "This paper proposes a novel and practical watermarking framework designed to protect the intellectual property of large code generation models. The authors introduce CodeGenGuard, which integrates a dual-LoRA fine-tuning mechanism and semantic-preserving transformations (SPTs) to embed robust, verifiable watermarks into code generated by pre-trained models. By leveraging two lightweight LoRA modules—a primary and a shadow adapter—CodeGenGuard achieves watermark embedding with minimal memory and computational overhead, while maintaining model performance. The framework embeds imperceptible watermark signals at both the token and expression levels through controlled syntax modifications that preserve program semantics. Extensive experiments across multiple programming languages and attack scenarios (e.g., fine-tuning, model extraction, and overwriting) demonstrate that CodeGenGuard achieves high verification accuracy and strong robustness compared with prior works such as CodeMark and ToSyn. Overall, the study presents an empirically grounded and engineering-driven framework for protecting LLM-based code generators, marking a meaningful advancement toward secure and accountable AI-generated software.", "questions": "1.  Robustness of dead-code-based SPTs under output filtering\n    In the paper, rare SPT patterns are augmented using dead-code insertion generated from probabilistic context-free grammars (PCFGs). However, based on empirical experience, models tend to memorize and reproduce these fixed dead-code fragments verbatim during fine-tuning. If an adversary fine-tunes the stolen model but filters out dead-code outputs during generation, legitimate users would never observe those injected patterns. How would this affect the watermark detectability, particularly for rare SPTs introduced through dead-code augmentation? Furthermore, since the paper extends to multi-bit watermarking (Section D.3.2) that relies on a larger variety of SPTs, how resilient would such augmented, dead-code-based SPTs remain under this type of output filtering? Finally, how might the effectiveness results reported in Section D.3.1 change if dead-code outputs are systematically suppressed during generation?\n\n2. The Pass@1 results in Figure 4 show a substantial performance drop for CodeMark compared to both the clean and CodeGenGuard models, particularly on the MBPP benchmark. Could the authors explain the underlying theoretical reason for such a large degradation in generation quality? Including additional evaluation on HumanEval would strengthen the paper’s empirical section. Since HumanEval provides shorter and more uniform tasks with lower result variance, it could complement MBPP by offering a more stable and interpretable measure of post-training fidelity after watermark embedding or fine-tuning.", "limitations": "", "rating": 4, "confidence": 4, "soundness": 3, "presentation": 3, "contribution": 2, "strengths": "1. Efficient Fine-tuning Design  \n   The proposed dual-LoRA architecture enables watermark embedding without full model fine-tuning, significantly reducing computational and memory overhead.  \n   This design makes the method lightweight and more deployable in real-world applications.\n\n2. Black-box Verification  \n   The framework supports watermark detection without requiring internal model access, aligning well with realistic API-based or commercial model scenarios.  \n   This enhances the practical applicability of the approach to closed-source environments.\n\n3. Empirical Robustness  \n   Experimental results demonstrate that CodeGenGuard maintains strong watermark detectability under common attack settings (e.g., fine-tuning, model extraction).  \n   This suggests the framework provides reasonable robustness in practical deployment.\n\n4. Clarity and Coherent Technical Design  \n   A notable strength of CodeGenGuard lies in the clarity and coherence of its technical design and rationale.  \n   The paper offers well-structured explanations of its main modules — including the semantic-preserving transformation (SPT) taxonomy, the dual-LoRA shadow training mechanism, and the auxiliary prompt-based verification process — each supported by explicit reasoning for their inclusion.  \n   This makes the overall system logically consistent and easy for readers to understand.", "weaknesses": "1. Limited Theoretical Analysis  \n   The paper focuses primarily on engineering design and empirical validation, but lacks a deeper theoretical foundation or formal guarantees regarding watermark detectability, robustness, or false-positive rates.  \n   As a result, the security properties of the method remain largely empirical.\n\n2. Overhead and Complexity Not Fully Quantified  \n   While the paper claims that the dual-LoRA design reduces computational and memory costs, it does not provide explicit measurements or comparisons to baseline fine-tuning strategies.  \n   The efficiency advantage, though plausible, remains qualitatively described rather than quantitatively proven.\n   \n\n3. Lack of concrete SPT transformation examples   \n    Although the paper proposes a taxonomy of Semantic-Preserving Transformations (SPTs) and emphasizes their role in embedding watermarks, it does not provide explicit code-level illustrations for each SPT pattern. The absence of concrete “before-and-after” code examples makes it difficult for readers to fully understand how these transformations are implemented in practice and how they affect code semantics or readability.\n\n4. Unclear random trigger design   \n    The paper introduces random triggers \\(r\\) to ensure the model generates normal code for irrelevant inputs, but it does not explain how these triggers are generated or sampled. Without concrete code examples or generation details, it is difficult to understand their distribution, neutrality, or implementation in practice.\n\n5. Unclear statistical testing procedure   \n    The paper outlines the null and alternative hypotheses for watermark verification but does not specify how the hypothesis test is actually conducted. It remains unclear whether the authors employed a t-test, a frequency-based test, or another statistical method. This lack of detail limits transparency and reproducibility of the verification process.\n\n6. Unclear Dual Lora Design   \n    While the paper introduces the dual-LoRA training scheme as a parameter-efficient alternative to full shadow training, the mathematical formulation of how the two LoRA modules are actually merged remains insufficiently explained. Although the authors define the watermark and shadow models as \\(F_{\\{wm,shd\\}} = F(W_0 + A_{\\{wm,shd\\}}B_{\\{wm,shd\\}})\\) and later state that the final model is \\(W_{wm} = W_0 + A_{wm}B_{wm}\\), it is unclear how these two low-rank matrices interact or whether their gradients are jointly or alternately optimized during training. Moreover, while three losses (\\(L_{wm}, L_{norm}, L_{neg}\\)) are defined separately, the paper does not specify a unified optimization objective or a weighting scheme showing how these components are integrated into a single training loss. To improve clarity, the authors are encouraged to provide an explicit mathematical expression of the overall loss function (e.g., \\(L_{total} = \\lambda_1 L_{wm} + \\lambda_2 L_{norm} + \\lambda_3 L_{neg}\\)) and to detail how the dual-LoRA parameters are merged or combined in the final watermarked model. Such clarification would significantly strengthen the technical transparency and reproducibility of the proposed approach.\n\n7. The paper misses several relevant references in code watermarking. It should include DeCoMa [A1] for dataset-level watermark detection, and [A2] for post-training traceable model watermarking similar to ToSyn. Adding these works would provide a more complete contextual grounding. \n\n   [A1]Yuan Xiao, Yuchen Chen, Shiqing Ma, Haocheng Huang, Chunrong Fang, Yanwei Chen, Weisong Sun, Yunfeng Zhu, Xiaofang Zhang, and Zhenyu Chen. 2025. DeCoMa: Detecting and Purifying Code Dataset Watermarks through Dual Channel Code Abstraction. Proc. ACM Softw. Eng. 2, ISSTA, Article ISSTA075 (July 2025), 24 pages. https://doi.org/10.1145/3728952 \n\n   [A2] Tom Sander, Pierre Fernandez, Alain Durmus, Matthijs Douze, and Teddy Furon. 2025. Watermarking makes language models radioactive. In Proceedings of the 38th International Conference on Neural Information Processing Systems (NIPS '24), Vol. 37. Curran Associates Inc., Red Hook, NY, USA, Article 664, 21079–21113.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "EbAkoC8h2z", "reviewer": "Reviewer_4MVM", "summary": "The paper targets IP protection for code language models by introducing CodeGenGuard, a watermarking framework designed for the constraints of code generation, where outputs must remain syntactically correct and semantically equivalent. It encodes watermarks via semantic-preserving transformations (SPTs)—augmented with dead-code patterns—while using an optimizable trigger to precisely control watermark activation and auxiliary prompts at verification time to steer the model into contexts where SPTs manifest, improving detectability. To boost robustness against fine-tuning and model extraction, the method employs dual-LoRA shadow training, offering a parameter-efficient way to embed resilient signals without harming utility. Experiments on representative code LMs reportedly show higher effectiveness and robustness than state-of-the-art baselines while maintaining generation quality; thus, the core contributions are the SPT-based watermark design for code LMs, the dual-LoRA shadow-training scheme, and the trigger-plus-prompt mechanism for pinpointed verification.", "questions": "Refer to the weaknesses section.", "limitations": "", "rating": 4, "confidence": 4, "soundness": 3, "presentation": 3, "contribution": 3, "strengths": "1. SPTs embed the watermark through function-preserving edits, thereby minimizing utility loss. The optimized trigger enables controlled activation, preventing unconditional pattern removal during subsequent fine-tuning. Meanwhile, the shadow training strategy enhances robustness against model extraction and parameter modification, maintaining parameter efficiency through a dual-LoRA design. Evaluations on representative code generation models demonstrate that this approach achieves consistent, state-of-the-art watermark detectability and robustness without compromising code quality.\n2. The motivation is clearly articulated: code language models impose inherently conflicting requirements for watermarking. The proposed pipeline is logically structured and easy to follow—beginning with the SPT dataset, then data augmentation, joint training with trigger and dual-LoRA/shadow modules, and concluding with verification via auxiliary prompts. Each component serves a distinct purpose: SPTs capture semantics, the trigger provides controllability, shadow training enhances robustness, and auxiliary prompts facilitate verification. This clear delineation allows readers to directly connect each design choice to the corresponding failure modes in prior approaches.", "weaknesses": "1. The paper’s robustness to model extraction is under-specified. In particular, does the extraction adversary receive full logits (soft labels) or only sampled output sequences? The threat model and experimental setup should make this explicit. It is also unclear why the proposed watermark remains a reliable ownership signal when the extractor never invokes the trigger. If watermarked signals leak into ordinary outputs absent the trigger, one would expect elevated false positives and correspondingly higher p-values than reported; conversely, if ordinary outputs are unwatermarked, it is unclear how the method verifies IP violations after extraction. I recommend: (i) reporting detection metrics (ROC/AUC, FPR at fixed TPR, p-values) on generations without triggers, both for the original and extracted models; (ii) quantifying watermark leakage rates under no-trigger inference; and (iii) analyzing whether auxiliary prompts at verification can still elicit the watermark post-extraction, and how this interacts with the optimized trigger. A clear, formal threat model plus these ablations would substantiate the claimed robustness to model extraction.\n2. The proposed approach embeds a watermark through a backdoor mechanism. Although the paper evaluates several defense strategies, many of them are not representative of the current state-of-the-art in backdoor mitigation. I recommend strengthening the experimental evaluation by incorporating more advanced defense methods (e.g., [1,2,3,4]) and analyzing their effects on both watermark detectability and overall model utility. Such an analysis would yield a more rigorous and up-to-date robustness assessment, offering clearer insights into the method’s resilience against stronger and adaptive adversaries.\n\nReferences\n\n1. Li et al. 2021. Anti-backdoor learning: Training clean models on poisoned data.\n2. Zhao et al. 2024. Defense against backdoor attack on pre-trained language models via head pruning and attention normalization.\n3. Arora et al. 2024, Here’s a free lunch: Sanitizing backdoored models with model merge.\n4. Tong et al. 2025. Cut the Deadwood Out: Backdoor Purification via Guided Module Substitution", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "x78arkMK4v", "reviewer": "Reviewer_mB9F", "summary": "This paper addresses the problem of protecting code generation models from theft and redistribution after release, as existing watermarking techniques designed for text or classification tasks are unsuitable for code generation due to strict syntax and semantic constraints. The authors propose CodeGenGuard, an embeddable watermarking framework featuring (1) Semantic-Preserving Transformations (SPT) that encode watermarks via functionally equivalent syntax changes, (2) Dead-Code Augmentation based on PCFG to enrich watermark diversity, (3) Dual-LoRA Shadow Training to enhance robustness without increasing memory, and (4) an Optimizable Trigger and Auxiliary Prompt mechanism with statistical significance testing for reliable verification. Experiments on CodeGen-350M and DeepSeek-Coder-1B show strong watermark detectability, minimal performance loss, and high robustness against extraction and fine-tuning attacks.", "questions": "1. My concern is how does CodeGenGuard transfer the watermark into the distilled model?\n2. Please consider Yang et al., 2024a and Zhang et al., 2025 as baseline. If not, please explain why?", "limitations": "", "rating": 2, "confidence": 3, "soundness": 3, "presentation": 3, "contribution": 2, "strengths": "It addresses a meaningful research problem.", "weaknesses": "1.The paper lacks a released code repository, which limits reproducibility.\n2.I agree with the authors’ statement in the first paragraph of Section 5.3. However, this leads to my main concern: how does CodeGenGuard transfer the watermark into the distilled model? From Figure 1, it is unclear which step actually enables the distilled model to inherit the watermark. Based on the loss function described in Update Watermarked Model, when the model is given normal inputs, it produces normal outputs. This suggests that the distilled model would learn patterns without the watermark. Therefore, I have doubts about the model extraction experiments, and I hope the authors can provide more detailed experimental procedures and theoretical explanations.\n3.Another concern is that, in Experiment 5.2, CodeMark also performs well in terms of effectiveness but shows weaker fidelity, which seems mainly due to its loss function design. If we only consider this experiment, the contribution of this paper may appear incremental, as it seems to primarily optimize the loss function on top of CodeMark—this point depends on whether Question 2 can be convincingly addressed.\n4.In the Threat Model section, please revise the last sentence: the attacker should not know the specific watermarking scheme. If they had full knowledge of it, they could remove the watermark. The correct assumption should be that the attacker is aware that watermarking is used, but not familiar with the exact technical details.\n5.Regarding the choice of baselines, the authors should consider Yang et al., 2024a and Zhang et al., 2025, which are more recent and relevant methods.\n6.The authors should consider evaluating the robustness of the proposed watermarking approach against code optimization tools, such as Sourcery or similar refactoring toolkits.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}]

Author rebuttal:

Decision: Accept (Poster)