Who Wrote this Code? Watermarking for Code Generation (2024)

Abstract: Since the remarkable generation performance of large language models raised ethical and legal concerns, approaches to detect machine-generated text by embedding watermarks are being developed.However, we discover that the existing works fail to function appropriately in code generation tasks due to the task’s nature of having low entropy.Extending a logit-modifying watermark method, we propose Selective WatErmarking via Entropy Thresholding (SWEET), which enhances detection ability and mitigates code quality degeneration by removing low-entropy segments at generating and detecting watermarks.Our experiments show that SWEET significantly improves code quality preservation while outperforming all baselines, including post-hoc detection methods, in detecting machine-generated code text.Our code is available inhttps://github.com/hongcheki/sweet-watermark.

## introduction in understanding and generating software programs, large language models have rapidly advanced towards expert-like proficiency @xcite @xcite @xcite @xcite . this breakthrough in the automation of the coding process improves the productivity and efficiency of software engineer and lowers the barriers to creating programs for non-experts @xcite . however, this advance comes with significant legal, ethical, and security concerns, including code licensing issues, code plagiarism, code vulnerability, and malware generation @xcite ; @xcite @xcite @xcite . for example, there is an ongoing class-action copyright lawsuit between a group of individuals and microsoft, github, and openai, arising from allegations of unlawful utilization and reproduction of the source code 12 . furthermore, shortly after the launch of chatgpt, numerous malicious actors on the dark web were observed sharing machinegenerated malware and spear phishing tutorials 3 . therefore, the development of reliable tools for detecting machine-generated code is a very timely matter and is of utmost importance for fairly deploying llms with coding capabilities. despite the need for immediate treatment of the machine-generated code detection problem, few efforts have been made to address it. instead, many works still prioritize a detection problem on normal text @xcite @xcite @xcite @xcite @xcite . while these post-hoc detection methods (i.e., no control during the text generation) have demonstrated powerful performance in the many domain of natural language tasks, their application to programming language remains unexplored. contrary to the post-hoc detection methods, another line of research for detecting machinegenerated text has gained attention: watermarkingbased methods, which embed a hidden signal within the generated text @xcite @xcite . for example, a method proposed in @xcite -which we refer to as wllm (watermarking for large language models) -randomly divides the entire vocabulary into two groups (i.e., the green list and the red list) at each generation step and enhance the probability of green list tokens to be sampled. by adding scalar values to the logits of a green list tokens, the model favors generating tokens from the green list rather than the red one. to detect the watermark in a text, we count the number of green tokens and check whether this number is statistically significant (through hypothesis testing) to conclude whether the model output is generated without knowledge of the green-red rule. while both watermarking-based methods and post-hoc detection methods work well in many language generation tasks, we observe that these performances do not transfer well to code generation tasks, for example, in figure 1 . in other words, it is much more challenging to embed watermarks in a detectable way without impairing the code functionality. we attribute this to the nature of extremely low entropy foot_3 of code generation. if watermarking is applied strongly, it can severely degrade the quality of the model output, which is particularly critical in code generation, as a single violation of a rule can break the entire code (see "strong watermark" in figure 1 ). on the other hand, if watermarking is applied too weakly, the low entropy hinders properly embedding watermarks and results in insufficient green tokens appearing, leading to increased difficulty in detection (see "weak watermark" in figure 1 ). these failures are not significant in plain text generation because the relatively higher entropy allows for more flexibility in candidate selections for watermarking. to address these failure modes, we extend the wllm and propose selective watermarking via entropy thresholding (sweet) for code llms (and llms). instead of applying the green-red rule to every single token during generation, we only apply the rule to tokens with high enough entropy given a threshold. that is, we do not apply the green-red rule to the important tokens for making functional code, while making sure there are enough green list tokens to make a detectable watermark for less important tokens, hence, directly addressing each of the above failure modes. in code generation tasks, our method outperforms all baselines, including post-hoc detection methods, in detecting machine-generated code while achieving less code quality degradation than wllm. furthermore, through various analyses, we demonstrate that our method operates well even without prompts or with a small surrogate model, indicating its robust performance under practical settings. our contributions are as follows: watermarking code text generated from a llm is closer to static watermarking. for example, @xcite proposes a method employing the replacement of synonymous code. however, since this method heavily relies on language-specific rules, a malicious user knowing these rules could reverse the watermarking. ## method we propose a new watermarking method, sweet, that selectively watermarks tokens only with high enough entropy. ## experiments we conduct a series of experiments to evaluate the effectiveness of our watermarking method in code generation for two aspects: (i) quality preserving ability and (ii) detection strength. our base model is starcoder @xcite , which is an opensource llm specifically for code generation. we also conduct experiments on one of the generalpurpose llm, llama2 @xcite (see the results in appendix f). ## analysis 6.1 impact of entropy thresholds figure 4 presents how code generation performance and detecting ability trade-off when calibrating the entropy threshold in our method. wllm is when the entropy threshold is not applied (i.e., entropy @xmath0. as the entropy threshold increases, the ratio of watermarked tokens decreases, so the code generation performance converges to a nonwatermarked base model. this indicates that our method always lies between the wllm and a nonwatermarked base model in terms of code generation performance. on the other hand, the detection ability, as the entropy threshold increases, reaches a local maximum but eventually declines. while our method with a moderate threshold effectively restricts generating the red list tokens compared to the wllm, detection ability eventually decreases if the threshold is so high that few tokens are watermarked. we further investigate how to effectively calibrate the entropy threshold value in appendix h. ## conclusion we identified and emphasized the need for code llm watermarking, and formalized it for the first time. despite the rapid advance of coding capability of llms, the necessary measures to encourage the safe usage of code generation models have not been implemented yet. our experiments showed that existing watermarking and detection techniques failed to properly operate under the code generation setting. the failure occurred in two modes: either 1) the code does not watermark properly (hence, cannot be detected), or 2) the watermarked code failed to properly execute (degradation of quality). our proposed method sweet, on the other hand, improved both of these failure modes to a certain extent by introducing selective entropy thresholding which filters tokens that are least relevant to execution quality. in code generation tasks, our method performs better than baselines, including post-hoc detection methods, while achieving less code quality degradation. moreover, comprehensive analysis demonstrates that our method still works well in real-world settings, specifically when the prompts are not given, utilizing even a smaller surrogate model, or under paraphrasing attacks.