A Black-Box Attack on Code Models via Representation Nearest Neighbor Search (2023)

Abstract: Existing methods for generating adversarial code examples face several challenges: limted availability of substitute variables, high verification costs for these substitutes, and the creation of adversarial samples with noticeable perturbations. To address these concerns, our proposed approach, RNNS, uses a search seed based on historical attacks to find potential adversarial substitutes. Rather than directly using the discrete substitutes, they are mapped to a continuous vector space using a pre-trained variable name encoder. Based on the vector representation, RNNS predicts and selects better substitutes for attacks. We evaluated the performance of RNNS across six coding tasks encompassing three programming languages: Java, Python, and C. We employed three pre-trained code models (CodeBERT, GraphCodeBERT, and CodeT5) that resulted in a cumulative of 18 victim models. The results demonstrate that RNNS outperforms baselines in terms of ASR and QT. Furthermore, the perturbation of adversarial examples introduced by RNNS is smaller compared to the baselines in terms of the number of replaced variables and the change in variable length. Lastly, our experiments indicate that RNNS is efficient in attacking defended models and can be employed for adversarial training.

## introduction recently, since programming language can be seen as one kind of textual data and also inspired by the success of deep learning for text processing and understanding, researchers have tried to pretrain code models such as codebert @xcite , @xcite , contra-bert @xcite to help developers to solve multiple programming tasks, e.g., code search @xcite , code clone detection @xcite , code sum-marization @xcite , and vulnerability detection @xcite . although these code models have achieved good performance on many code tasks, they are still suffering from robustness issues. a few adversarial attack methods have emerged to evaluate and improve the robustness of code models. there are certain considerations to be made. firstly, code pre-training models are frequently deployed remotely, which limits access to the model parameters and renders white-box attacks infeasible. secondly, among the numerous codeequivalent transformation methods, variable substitution exerts the most significant influence on the resilience of large code models while being the least detectable transformation @xcite . as a result, black-box attack techniques based on variable substitution have emerged as a valuable avenue for research and multiple works have been proposed such as alert @xcite and mhm @xcite . however, these works have three limitations: 1) the number of substitute variables is limited and lacks diversity, which lowers the upper bound of the attack success rate. for example, alert employs 60 substitute variables for each variable, which are generated by a pre-trained model, and the substitute variables lack diversity. mhm also randomly selects 1500 words from a fixed dictionary as substitute variables. 2) the verification cost of substitute variables is high. to verify the attack effect of each substitute, it is necessary to replace the source variable with an adversarial sample and perform an actual attack on the victim model. alert uses a traversal method to select substitute variables, and in order to reduce the number of attacks, it limits the number of substitute variables; mhm uses a random sampling method to select substitute variables in order to reduce the number of attacks. neither method is conducive to cost-effective attacks. 3) the generated adversarial samples have large perturbations. each adversarial sample usually needs to replace multiple original variables to succeed in attacking, and mhm easily generates semantically incoherent and excessively long variable names. to address the aforementioned challenges, in this paper, we propose a search-based black-box adversarial attack method to create challenging adversarial samples based on the search seed vector in the variable representation space, namely representation nearest neighbor search (rnns). specifically, rnns, first utilizes publicly available real code datasets to construct a large original substitute set, denoted as subs original . then, based on the previous attack results, rnns predicts the search seed vector required for the next round of attacks and efficiently searches for the k nearest substitutes to the seed vector from the large-scale original substitute set to form the subs topk , where k is much smaller than the size of the original substitute set. the generation process of the subs topk does not involve attacking the victim model even once. furthermore, the length and similarity of the substitute must adhere to specific perturbation constraints to prevent excessive deviations from var. to evaluate the effectiveness of rnns, we investigate three pre-trained code models, code-bert @xcite , @xcite and codet5 @xcite , and perform the attack on six code tasks in three programming languages, i.e., java, python, and c. the results on 18 victim models demonstrate that compared to the approaches mhm and alert, rnns achieves a higher attack success rate (asr) with a maximum of about 100% improvement and 18/18 times as the winner. meanwhile, rnns needs fewer query times (qt) with 8/18 times as the winners. furthermore, we analyze the quality of adversarial examples statistically and find that rnns introduces minor perturbations. in the end, we apply rnns to attack three defended models and find that our approach outperforms the baselines by up to 32.07% asr. we also use adversarial examples to improve the model's robustness through contrastive adversarial training. ## experimental setup dataset and model. to study the effectiveness and efficiency of rnns, we conduct experiments on three popular programming languages (c, python, and java). for the datasets, we employed six widely studied open-source datasets that cover four important code tasks. specifically, big-clonebench @xcite ) is one code clone detection dataset named clone. devign @xcite is a dataset used for vulnerability detection, named defect. for authorship prediction, we use the dataset provided by @xcite . besides, we utilize three problem-solving classification tasks, java250, python800, and c1000, provided by projectcodenet @xcite . for all the datasets (except for authorship prediction which does not have enough data samples), we follow the original papers to split the data into the training set, validation set, and test set. authorship prediction only has two split parts, training data and test data. for the code models, we follow the previous work @xcite and investigate two pretrained models codebert @xcite , and @xcite . besides, we add one more powerful model codet5 @xcite in our study. evaluation metric. to evaluate the effectiveness of adversarial attack methods, we employ the commonly used attack success rate (asr) @xcite as the measurement. to evaluate the efficiency of the attack methods, we use query times (qt) to check the average number of querying the victim model for one input code. finally, we use the change of replaced-variable length and the number of replaced variables to study the quality/perturbation of adversarial examples. a smaller score means the attack method can generate adversarial examples with less perturbation injection. baseline. we compare rnns with two black-box attack baselines, mhm @xcite and naturalattack (alert) @xcite . mhm is a sampling search-based black-box attack that generates the substitutes from the vocabulary based on lexical rules for identifiers. mhm employs synthesized tokens as the candidates of substitutes, which could introduce meaningless variable names. alert is a recently proposed attack method that combines greedy attack and genetic algorithm to find the substitutes. we also use two textual attack algorithms pso @xcite and lsh @xcite as minor baselines, since they are not designed for code models. implementation. we implement our approach in pytorch and run all experiments on 32g-v100 gpus. we reuse the source code from the baselines. we make our implementation 3 publicly available. 5 results analysis ## conclusion we propose a novel black-box adversarial searchbased attack for variable replacement. rnns has three main contributions: 1) this work proposes a non-generation search-based black-box attacking method via predicting the attack effect of a substitute. this method can greatly reduce the verification cost of the substitute, remove the restrictions on the size and diversity of the substitute set, and achieve a significant improvement in terms of asr without increasing qt. 2) this work proposes a simple and efficient method for constructing a substitute set. this method can construct a large-scale, diverse, and real substitute set at low cost. 3) the adversarial examples from rnns can be used to improve the model robustness.