DIP : Dead code Insertion based Black-box Attack for Programming Language Model (2023)

Abstract: Automatic processing of source code, such as code clone detection and software vulnerability detection, is very helpful to software engineers. Large pre-trained Programming Language (PL) models (such as CodeBERT, GraphCodeBERT, CodeT5, etc.), show very powerful performance on these tasks. However, these PL models are vulnerable to adversarial examples that are generated with slight perturbation. Unlike natural language, an adversarial example of code must be semantic-preserving and compilable. Due to the requirements, it is hard to directly apply the existing attack methods for natural language models. In this paper, we propose DIP (Dead code Insertion based Black-box Attack for Programming Language Model), a high-performance and effective black-box attack method to generate adversarial examples using dead code insertion. We evaluate our proposed method on 9 victim downstream-task large code models. Our method outperforms the state-of-the-art black-box attack in both attack efficiency and attack quality, while generated adversarial examples are compiled preserving semantic functionality.

## introduction automatic processing of source code (such as clone detection, code completion, defect detection, etc.) is an important task that increases the productivity of software engineers @xcite @xcite @xcite @xcite . for these tasks, deep learning-based models, such as code2seq @xcite and code2vec @xcite , were developed. recently, transformer-based large pre-trained programming language (pl) models @xcite @xcite @xcite , showed powerful † corresponding author. performance. however, these pl models are also vulnerable to adversarial examples as other natural language (nl) models @xcite @xcite @xcite @xcite . source code is textual data just like natural language, but the requirements of attack methods are different from nl model attacks. attacks against nl models should consider semantic and fluency of original sentences, but attacks against pl models should preserve functionality and guarantee compilability. to consider for these issues, recent studies @xcite @xcite @xcite use semantic-preserving code transformation which is commonly used in software engineering. the semantic-preserving code transformation includes variable renaming, dead code insertion, statement permutation, loop exchange, switch-to-if, and boolean exchange. most of the prior works used variable renaming or dead code insertion to attack pl models. other transformations statement-permutation, loop-exchange, switch-to-if,boolean-exchange, etc, were less used for attacks because they are applicable only to corresponding operations. instead, they were usually used for data augmentation @xcite . variable renaming and dead code insertion can be much stronger attacks because attackers can flexibly choose various variable names or statements considering the contextual information of code. in the white-box setting, @xcite proposed attack methods on code2vec, using variable renaming and dead code insertion, which inserted perturbations sampled from the output distribution into input one-hot vectors representing variables. ramakrishnan et al. (2020) and srikant et al. (2021) proposed white-box attacks, optimizing position and perturbation of adversarial attacks, for basic pl models such as seq2seq, code2seq, and code2vec. however, those gradient-based white-box attack methods are inefficient, time-consuming, and computationally expensive because large pre-trained pl models, such as codebert @xcite , graphcodebert @xcite and codet5 @xcite , have a very large number of parameters. also, it is hard to apply white-box attacks on large pre-trained models in the real world because the models are not fully opened and are just accessible through api. therefore, the blackbox attack is more practical than the white-box for attacking large pre-trained models. in the black-box setting, zhang et al. (2020) proposed mhm, a sampling-based black-box attack method for programming language model using variable renaming. yang et al. (2022) proposed alert, a genetic algorithm based black-box attack method using variable renaming. however, these attack methods using variable renaming are inefficient in attack success rate and in query count, because attack targets are limited to variables, and the search space for candidate tokens is very large. additionally, these methods have a critical problem that compilability is not guaranteed. in programming language, unlike natural language, compilability is more prior than naturalness, and must be guaranteed. there are some variables that should not be renamed. for example, global variables or class static variables in java, such as system.out for standard input/output, should not be changed. however, variable renaming can replace such variables, and could raise a critical problem such as compile errors. we propose dip (dead code insertion based black-box attack for programming language model), an efficient and the state-of-the-art adversarial attack method using dead code insertion with an unused variable statement. under the black-box setting, we do not need any information of a large pre-trained model, no additional computation for training is required for attack. for efficiency, we reduce the candidate search space by inserting a statement from other code, not tokens. randomly selected dissimilar codes are used to obtain statements to be inserted. we extract statements from code using the attention score of a pre-trained model, which is based on transformer architectures. the snippet extracted by our method is powerful and transferable. in addition, our proposed method guarantee to preserve functionality and compilability in any case because the attack is based on dead statement insertion. we show that our proposed method outperforms the existing methods to attack pre-trained programming language models in section 4. we choose 3 pre-trained pl models, codebert, graphcode-bert, and codet5, and 3 tasks, code clone detection, defect detection, and authorship attribution task. we build nine victim models in total. we also conduct various experiments to explore our method. the ablation study ( §4.4) examines the effectiveness of each component in dip. in §4.5, we test how strong dip is on adversarially trained models. we also test whether adversarial samples used to attack a victim model can also fool another in §4.6. ## task definition we evaluate dip on code clone detection, defect detection, and authorship attribution tasks. for these tasks, we build the model by adding mlp classifier to the state-of-the-art pre-trained pl models (such as codebert, graphcodebert, and codet5). then, we fine-tune the nine victim models. black-box setting. since our proposed method is a black-box attack, we do not use any information from the victim models. to obtain the code representation and attention score in dip, we need a pre-trained model based on transformer architectures @xcite . in this paper, we use codebert, a general pre-trained pl model which is not fine-tuned, to obtain code embeddings without any information from victim models. non-targeted adversarial attack. adversarial attack can be usually categorized as either nontargeted attack or targeted attack. non-targeted adversarial attack is to slightly modify the source input in a way that the input will be misclassified far from the ground truth. targeted adversarial attack is to slightly modify the source input in a way that the input will be misclassified as a specified target class. since our method is a non-targeted attack, the goal is defined as follows. goal. given an input sample (c i , y), find c adv i , by adding perturbation to c i , that preserves functionality and guarantees compilability, and that misleads the victim model. the adversarial code, c adv i , needs to satisfy the followings: ## dip we propose dip, using dead code insertion to guarantee compilability while preserving the semantics of source codes. dead code insertion is very suitable for source code attack because it does not affect functionality and compilability of code. our method consists of three main steps as shown in figure 2: (1) we find vulnerable positions to insert dead code in the original source code. dead code will be inserted between statements in source code as a statement to guarantee compilability. we evaluate the vulnerability of each position in source code by a pre-trained code model which is not fine-tuned, and choose some of them as candidate vulnerable positions. (2) we obtain dissimilar codes of which snippets will be used as dead code. dissimilarity of source code is defined based on the cosine similarity to the original source code using @xcite token representations. (3) to minimize perturbation, we extract snippets from dissimilar algorithm 1: dip pseudo-code input :source code c i , true label y, target model m output :adversarial example c adv i 1 compute position importance v p ∀ @xmath0, . . . , d k ] ordered by the dissimilarity 4 for p in ascending order of v p do 5 c adv i ← c i 6 for @xmath1: c adv 11 end 12 end 13 end 14 # fail attack 15 return: c adv ← n one source codes. snippets are extracted by the attention score in dissimilar source codes. we wrap the obtained snippet with an unused variable statement and insert it into the source code as dead code. we summarize our proposed method, dip, in algorithm 1. ## discussion we discuss the compliability and detectactibility for the attack on pl models. ## related work the adversarial attack for language data (such as text, source code) is significantly difficult due to the discrete property of token embedding space. both text and source code have a discrete property, but there is a difference in the attack methods. in the adversarial attack for natural language, the text should be considered for fluency and semantic consistency. the previous works @xcite @xcite @xcite @xcite proposed adversarial attack methods, which considered semantic and fluency of sentence. however, the source code is more important in compilability than fluency. in the adversarial attack for programming language, the source code should be considered for semantic (functionality) preserving and must be compiled. the previous works, rabin et al. (2021) evaluated the generalizability of code2vec, code2seq, ggnn by using the various transformation (permute statement, boolean exchange and loop exchange, etc.). zhang et al. (2020) proposed mhm, in black-box setting, which is the variable renaming based on metropolis-hastings sampling method. yang et al. (2022) proposed alert, to improve mhm, which use the variable renaming based on genetic algorithm. ramakrishnan et al. (2020); yefet et al. (2020) proposed white-box attack method using the ast transformation and the output distribution into one-hot vector, which is represented variable name. srikant et al. (2021) proposed white-box attack method to optimize the attack position and perturbation at the same time. 7 conclusion in this paper, we presented dip (dead code insertion based black-box attack for programming language model), the state-of-the-art black-box attack method. we proposed an attack method consist of vulnerable position selection, dissimilar code selection and snippet extraction. experiment results demonstrate the high-performance and effectiveness of dip, also we evaluated the transfer-ability to apply other models. under the black-box setting, our proposed method guaranteed compilability in any case and preserved functionality.