A Black-Box Attack on Code Models via Representation Nearest Neighbor Search
Authors: Jie Zhang
Wei Ma
Qiang Hu
Shangqing Liu
Xiaofei Xie
YVES LE TRAON
Yang Liu
Venue: EMNLP 2023 Findings (2023)
Keywords: black box attacks
code models
robustness
code adversarial example

Abstract: Existing methods for generating adversarial code examples face several challenges: limted availability of substitute variables, high verification costs for these substitutes, and the creation of adversarial samples with noticeable perturbations. To address these concerns, our proposed approach, RNNS, uses a search seed based on historical attacks to find potential adversarial substitutes. Rather than directly using the discrete substitutes, they are mapped to a continuous vector space using a pre-trained variable name encoder. Based on the vector representation, RNNS predicts and selects better substitutes for attacks. We evaluated the performance of RNNS across
six coding tasks encompassing three programming languages: Java, Python, and C. We employed three pre-trained code models (CodeBERT, GraphCodeBERT, and CodeT5) that resulted in a cumulative of 18 victim models. The results demonstrate that RNNS outperforms baselines in terms of ASR and QT. Furthermore, the perturbation of adversarial examples introduced by RNNS is smaller compared to the baselines in terms of the number of replaced variables and the change in variable length. Lastly, our experiments indicate that RNNS is efficient in attacking defended models and can be employed for adversarial training.

A Black-Box Attack on Code Models via Representation Nearest Neighbor Search
Jie Zhang"; Wei Ma", Qiang Hu², Shangqing Liu², Xiaofei Xie Yves Le Traon', and Yang Liu²
'Noah's Ark Lab, Huawei
School of Computer Science and Engineering, Nanyang Technological University The Interdisciplinary Centre for Security, Reliability and Trust, University of Luxembourg School of Computing and Information Systems, Singapore Management University
Abstract
marization (Ahmad et al., 2020: Liu al., 2020), and vulnerability detection (Zhou al., 2019). Al- though these code models have achieved good per- formance on many code tasks, they are still suf- fering from robustness issues. A few adversarial attack methods have emerged to evaluate and im- prove the robustness of code models.
Existing methods for generating adversarial code examples face several challenges: lim- ited availability of substitute variables, high verification costs these substitutes, the creation of adversarial samples with notice- able perturbations. To address these concerns, our proposed approach, RNNS, uses a search seed based on historical attacks to find poten- tial adversarial substitutes. Rather than di- rectly using the discrete substitutes, they mapped to continuous vector space using a pre-trained variable name encoder. Based the vector representation, RNNS predicts and selects better substitutes for attacks. We evaluated the performance of RNNS across six coding tasks encompassing three program- ming languages: Java, Python, and C. We em- ployed three pre-trained code models (Code- BERT, GraphCodeBERT, and CodeT5) that re- sulted in cumulative of 18 victim models. The results demonstrate that RNNS outperforms baselines in terms of ASR and QT. Further- more, the perturbation of adversarial examples introduced by RNNS is smaller compared to the haselines in terms of the number of replaced variables and the change in variable length. Lastly, our experiments indicate that RNNS is efficient in attacking defended models and can be employed for adversarial training.
There are certain considerations to be made. Firstly, code pre-training models are frequently deployed remotely, which limits access to the model parameters and renders white-box attacks infeasible. Secondly, among the numerous code- equivalent transformation methods, variable sub- stitution exerts the most significant influence on the resilience of large code models while being the least detectable transformation (Li et al., 2022). As result, black-box attack techniques based on variable substitution have emerged avenue for research and multiple works have been proposed such as ALERT (Yang et al., 2022) and MHM (Zhang et al., 2020).
valuable
However, these works have three limitations: 1) The number of substitute variables limited and lacks diversity, which lowers the upper bound of the attack success rate. For example, ALERT employs 60 substitute variables for each variable, which are generated by pre-trained model, and the substi- tute variables lack diversity. MHM also randomly selects 1500 words from a fixed dictionary as sub- stitute variables. 2) The verification cost of substi- tute variables is high. To verify the attack effect of each substitute, it is necessary to replace the source variable with an adversarial sample and perform an actual attack on the victim model. ALERT uses traversal method to select substitute variables, and in order to reduce the number of attacks, it limits the number of substitute variables; MHM uses random sampling method select substitute variables order to reduce the number of attacks. Neither method is conducive to cost-effective at- tacks. 3) The generated adversarial samples have
1 Introduction
Recently, since programming language can be one kind of textual data and also inspired by the success of deep learning for text processing and understanding, researchers have tried to pre- train code models such as CodeBERT (Feng et al., 2020), GraphCodeBERT (Guo et al., 2020), Contra- BERT (Liu et al., 2023a) to help developers to solve multiple programming tasks, e.g., code search (Gu et al., 2018; Liu et al., 2023b), code clone detec- tion (White et al., 2016; Li al., 2017), code sum-
clark. zhang huawei.com
corresponding author: ma wei @ ntu.edu.sg

---

large perturbations. Each adversarial sample usu- ally needs replace multiple original variables to succeed in attacking, and MHM easily gener- ates semantically incoherent and excessively long variable names.
Rixd
((0)
Pg
Dj
Model
Domain Probability Space
To address the aforementioned challenges, in this paper, propose search-based black-box adversarial attack method to create challenging ad- versarial samples based on the search seed vector in the variable representation space, namely Rep- resentation Nearest Neighbor Search (RNNS), S). Specifically, RNNS, first utilizes publicly available real code datasets to construct a large original sub- stitute set, denoted as subSoriginal- Then, based on the previous attack results, RNNS predicts the search seed vector required for the next round of attacks and efficiently searches for the k nearest substitutes the seed vector from the large-scale original substitute set to form the substopk, where k is much smaller than the size of the original sub- stitute set. The generation process of the substopk does not involve attacking the victim model even once. Furthermore, the length and similarity of the substitute must adhere to specific perturbation constraints to prevent excessive deviations from var.
Code Taken
Token Vector Space
Sequence
Figure 1: One code model demo on the downstream task.
code input directly into deep learning models. Thus, transferring code data to learnable continu- ous vectors is the first step in source code learn- ing. Dense encoding (Zhelezniak al., 2020) is one common method used to vectorize textual code data. To do so, first, we need to learn a to- kenizer that splits the code text into a token se- quence which is called Tokenization. After to- kenization, code r is represented by sequence of tokens, namely, x = (so. .... .j....8t) where is one token. Then, the code vocabulary dictio- nary is built by using all the appeared tokens si. denoted V. After that, every word (token) in V is embedded by learned vectors v, with dimension d. Here, we use E to represent the embedding matrix for V. Finally, x can be converted into a embedding matrix Rixd = (vo, ..... Uj. vy). Af- ter this code encoding, pre-trained code models based on the transformer take the matrix R inputs and learn the contextual representation of for downstream tasks via pre-training such Masked Language Modeling (MLM) and Causal Language Modeling (CLM).
To evaluate the effectiveness of RNNS, we investigate three pre-trained code models, Code- BERT (Feng et al., 2020), GrRThCodeBERT (Guo et al., 2020) and CodeT5 (Wang et al., 2021), and perform the attack six code tasks in three pro- gramming languages, i.e., Java, Python, and C. The results on 18 victim models demonstrate that com- pared the approaches MHM and ALERT, RNNS achieves a higher attack success rate (ASR) with a maximum of about 100% improvement and 18/18 times as the winner. Meanwhile, RNNS needs fewer query times (QT) with 8/18 times as the win- ners. Furthermore, we analyze the quality of adver- sarial examples statistically and find that RNNS introduces minor perturbations. In the end, we ap- ply RNNS attack three defended models and find that our approach outperforms the baselines by up to 32.07% ASR. We also use adversarial ex- amples to improve the model's 's robustness through contrastive adversarial training.
Figure illustrates the main steps of the code processing models for the downstream classifica- tion tasks. First, we tokenize the textual code x into token sequence that is represented in discrete integer space. Then, we map the discrete sequence ids into the token vector space Rixd Next, we feed the token vectors into the task model (0). (0) is built on top of pre-trained models. Finally, we can predict the domain probabilities after fine-tuning.
2.2 Problem Statement
Since many critical code tasks are classification problems, e.g., defect prediction and code clone detection. In this paper, we focus the adversar- ial attack for code classification tasks. Considering code classification task, we use f(z; 0) + y R!xd - C = {il0 < S n) to denote the vic- tim model that maps code token sequence x to a label y from a label set with size n, where
2 Preliminaries
2.1 Textual Code Processing
The nature of code data (in text format with dis- crete input space) makes impossible to feed one

---

I is the sequence length and d is the token vec- tor dimension, and is one integer. By querying dictionary dense embedding ElVxd, a code token sequence z = (so,..., Sj,.., s1),
Algorilhm
Hbperarameter: maximum attckkng senazon max Input: loput code ground y, original substituee aubhoriginal
Output: adversriaal ckanple x' uttckiing result salaae
is vectorized into Rlxd Adversarial attacks for code models cre- ate an adversarial example x' by modifying some vulnerable tokens of x with a limited maximum perturbation to change the correct label y to wrong label y'. Simply, we get a perturbed x' by modifying some tokens in (so, .... Sj: .., st) such that fax;; 0) + fa; f) where x' x + a and x' has to have the same behavior with æ, + represent perturbation execution, a is the perturbation code transformation for (so..... ...... st), and o < e. We target the more practical attacking scenario black-box attack that requires less information. We assume camnot access the model parameters and
probmin 1.0
3 curs Exttrct
for vur E 1are do
subpre var
suBexr -ear A..mn
10
Jalse
13
whik' < maz_iiv do
Cared SCrme ProdictSeedf/sbppr 5tbher ddyme)
13
vubztopk SwarchTTphhaale Sabe antrrgrgggtgattnttt car)
14
for sub E sudatips do
15
shmp" Replanats'' :aaauaaaaa"aaaaarrs anbheur amb)
16
proby: V' = (toemp:))
nrohg prohhein
19
sabgra saleeu
20
subeur aub probanv proby
21
22
end
23
y y' them
34
V1 truc rehm n', in AMC
25
26
cud
28
cnd
end
can only utilize the final output of model f(æ; 0) to conduct the attack.
30
reouze a sss
3 Methodology
Given input code æ and one of its variables var, different substitutes can be used to replace it obtain different adversarial samples. After attack- ing the victim model, the probability of the label will also change. Conversely, if we want to reduce the probability of this label, the third question is following, "how to choose relatively better substi- tutes that can reduce the model confidence from a large-scale original substitute set?". It is possible to select good substitutes without actual attack if can forecast, which is implemented by RNNS. The core idea of RNNS maintaining a search seed updated based on the historical attack. The search seed is employed to search next adversarial substitutes that are possible to attack successfully. Since substitutes are discrete and cannot be directly involved in calculations, we first use variable name pre-trained encoder denoted E map substitutes to a unified continuous representation vector space. Then, based on the representation vectors of substitutes that have participated in the attack, we predict the search seed vector Exeed for the next round of the substitute selection. Finally, we calculate the similarity between Escod and the representation vector of substitutes and then select relatively better substitutes. For specific details, please refer to Section 3.2.3.
3.1 Motivation
As mentioned in the introduction, the current meth- ods face three limitations: 1) there is a limited num- ber of substitute variables; 2) there is a high ver- ification cost associated with substitute variables; and 3) the generated adversarial samples often ex- hibit large perturbations, Among these limitations, the second one holds the utmost significance as it significantly impacts both the first and third limi- tations. Due to the high cost involved, it becomes challenging to generate diverse adversarial exam- ples within a reasonable budget. Additionally, at- tackers tend to introduce large perturbations with- out employing any perturbation constraints in order to maximize their attacks.
To address these limitations, the first question arises: "Could substantially reduce the verifica- tion cost while allowing for unrestricted diversity of substitute variables and minimizing perturba- tions?" To delve into the reasons behind the sec- ond limitation, we need to analyze its underlying factors. The low verification efficiency of the sub- stitute set stems from the fact that each substitute can only be verified by constructing an adversarial sample to replace the original variable and then launching an actual attack on the victim model. This realization leads to the second question: "Is feasible to predict the attack effect of a substitute instead of constructing an adversarial sample to attack the victim model?"
3.2 Representation Nearest Neighbor Search
Algorithm 1 shows the workflow of our approach, First, we collect the original substitute set from public real code, following the process described

---

in Section 3.2.1. We extract variables from the input code and sort them according to their uncer- tainty, referring to Section 3.2.2 (Line 3-4). We replace variables in sequence to form attack sam- ples (Line 5). For a given var, we first initialize the optimal substitute for this current iteration subcur and the optimal substitute for the previous iteration subpre to the var. Then, we initialize the accumu- lated smooth increment of the representation vector Aemmo zero vector. emmo is used record the historical representation change of the search seed seed. We now commence the iterative attack process, as delineated in Line 1 I. We predict the search seed vector eseed with the process described in Section 3.2.3 (Line 12), and then extract topk substitutes based on Eseed to form the candidate substitutes substopp with the process described in Section 3.2.4 (Line 13). Subsequently, we replace subeur in a' with each substitute in substopk ob- tain the corresponding temporary adversarial sam- ple Zimp (Line 14-15). x' is the current code that we are trying to attack and it is initialized with the original code x. We use Zmmp to attack the vic- tim model and obtain the probability proby of the ground-trute label y and predicted label y/ (Line 16). If the probability of the ground-truth label y hits new low (< probman). we update x', subpre- subeur and probmin (Line 17-22). prob records the minimum probability of label y during the at- tack process. If x/mmp causes the victim model to predict an incorrect label, this attack is successful and returns the successful adversarial sample (Line 23-26); otherwise, proceed to the next iteration un- til all variables have completed iteration and return the final adversarial sample and attack result (Line 30).
along with their associated embeddings the ini- tial substitute set, represented as subSoriginal-
3.2.2 Computing Uncertainty
Given a specific code z, we replace each instance of var E x with a set of predefined fixed variables V ar Arrey, resulting in a set of mutated codes denoted as Xmautatedd These mutated codes are Baar subsequently utilized to query the victim model, al- lowing us to obtain the probability distribution for each class. A greater variance in the distribution signifies increased uncertainty for var, suggesting that ear should be prioritized for replacement. The uncertainty associated with var is defined as fol- lows:
uncertainlyoor 1
variance(PPrr
C
Xvar Xmmutated,, C is the number of labels, Poar (x)) is the model prob- ability for label given the mutated code x, and variance denotes the standard variance function. A larger and more diverse X'axtated ensures a closer approximation of uncertainty to the true value. It is important to note, however, that the mag- nitude of the change length must not be excessively large, as this would result in all probability changes converging
where Pour {p (P'aa(
single point. This because sam- ples subjected to large changes deviate significantly from the original, Icading to a substantial decrease in the model confidence levels. Subsequently, we arrange the variables in descending order based on their uncertainties. The greater the uncertainty of a variable, the more valuable it is for attack. This pro- cess is denoted as RankV ars With ncertainty at line 4. In our implementation, the size of this variable array Var Array 16, and the variable length ranges from to
3.2.1 Collecting Large Original Substitute Set We have developed a tool for variable extraction that leverages the tree-sitter framework!. This tool, henceforth denoted as ExtractVar ar (see Line 3). operates in three distinct steps. In the first step, we extract all variables from the current dataset and then filter out duplicates. During the second step, each valid variable is tokenized, and we compute the embedding for each token using the variable- name encoder that is pre-trained on CodeSearch- Net?. We then apply a mean pooling operation on these tokens to determine the variable's 's embedding. In the third step, we retain all the chosen variables
3.2.3 Predicting Search Seed
To filter out superior substitutes from the substan- tial subsoriginat. it becomes necessary to predict the search seed within the substitute representa- tion vector space. Given the optimal substitute subcur of the current round, the optimal substitute subyre from the previous round, and the accumu- lated smooth increment of the representation vector, denoted as Ae from all preceding rounds of itcration, we initially compute the increment of the representation vector in the current round, Ae:
hhetps://tree /atrss:///reeeeeeeee sicter. github. sittre Anctps: // htt.hgggggaaaaeee.c// col wwsaaatts/cood...o search_net
Ae = E(subeur) - E(subrre)

---

Task
Train/ Val/ Test
CodeBERT
GruphCuheBEIir
Code
Besides, we utilize three problem-solving classifi- cation tasks, Java250, Python800, and C1000, pro-
Defuct
7,737 2,71
63.75
63.65
67.07
Clonc
4,000/44,0000
96.97
97.36
Authorship
$28.
82:57
77.27
C1000
3200000
90,000 1000000
82.53
83.79
84.05
vided by ProjectCodeNet (Puri et al., 2021). For all the datasets (except for authorship prediction which does not have enough data samples), we fol- low the original papers to split the data into the training set, validation set, and test set. Authorship prediction only has two split parts, training data and test data.
Pythom8o0
188 /00/ 48,000
96.39
96. 29
96.79
Java250
480000/ 11,909 15,000
96.91
97:27
90.72
Table 1: Datasets and Victim Model Performance (Ac- curacy, %).
where E is variable name encoder, trained on CodeSearchNet by masked language modelling in- dependently so that RNNS is independent of vic- tim downstreammtask models. Then we update the
For the code models, we follow the previous work (Yang et al., 2022) and investigate two pre- trained models CodeBERT (Feng ct al., 2020), and GraphCodeBERR (Guo et al.. 2020). Besides, we add one more powerful model CodeT5 (Wang et al.. 2021) in our study. Table summarizes the details of our employed datasets and fine-tuned models.
A smo-
- eemo (1 - ae + ade
where a is a smooth rate limited 0 to Finally, we predict the search seed Esced-
Eseed E(subeur) + eemo
Evaluation Metric. To evaluate the effectiveness of adversarial attack methods, we employ the com- monly used attack success rate (ASR) (Yang et al., 2022) as the measurement. To evaluate the effi- ciency of the attack methods, we use query times (QT) to check the average number of querying the victim model for one input code. Finally, we use the change of replaced-variateee length and the num- ber of replaced variables to study the quality/per- turbation of adversarial examples. A smaller score means the attack method can generate adversarial examples with less perturbation injection.
This process is denoted as Predict.Sedd at line 12.
3.2.4 Searching Top-K Substitutes
Initially, we filter out substitutes from subsoriginol that comply with two constraints: 1) sim (Esbb)))), E(var)) < c and 2) [len(sub) len(var)) < 6, where var refers to the original variable in the input code that is to be replaced, sim(() is the similarity calculation function. E() is the variable name encoder, and len(.) is used to calculate the length of the variable name. Then, we calculate the similarity between the search seed exeed and the substitutes that are filtered by the two constraints and select the k most similar sub- stitutes to form substopk- This process is denoted SewrchTaroooooo Sub Searchhoop:Subb at line 13. In our experiment, e 0.15, 5 1, k- 60, sim(.) is cosine similar- ity.
Baseline. We compare RNNS with two black-box
attack baselines, MHM (Zhang et al., 2020) and NaturalAttack (ALERT) (Yang et al., 2022). MHM is sampling search-based black-box attack that generates the substitutes from the vocabulary based on lexical rules for identifiers. MHM employs syn- thesized tokens as the candidates of substitutes, which could introduce meaningless variable names. ALERT is a recently proposed attack method that combines greedy attack and genetic algorithm to find the substitutes. We also use two textual attack algorithms PSO (Zang et al., 2020) and LSH (Ma- heshwary et al., 2021) as minor baselines, since they are not designed for code models.
4 Experimental Setup
Dataset and Model. To study the effectiveness and efficiency of RNNS, we conduct experiments on three popular programming languages (C, Python, and Java). For the datasets, we employed six widely studied open-source datasets that cover four important code tasks. Specifically, Big- CloneBench (Wang et al., 2020) one code clone detection dataset named Clone. Devign (Zhou et al., 2019) is dataset used for vulnerability detection, named Defect. For authorship prediction, we use the dataset provided by (Alsulami et al., 2017).
Implementation.
We implement our approach in PyTorch and run all experiments on 32G-v100 GPUs. We reuse the source code from the baselines. We make our implementation publicly available.

---

ALERT
MHM
RNNS
Task+Model
ASR
ASR
QT
ASR
QT
Clone+CodeBert
28.67
2155.39
39.66
972.15
46.50
666.48
Clone-GraphCodeeee
10.40
1466.68
9.58
490.99
41.28
1122.01
Clone+CideTTS
29.20
2359.70
38,79
1069.06
39.61
895,79
Defect+CodeBett
52.29
1079.68
50,51
862, 18
69.18
$88.35
74.29
621.77
75.19
539.93
81.63
404.73
Cefoct+CodeT5
76.66
721.02
86.51
344.08
89.45
344.29
Auhhoshhip-CuddBeet
775.11
73.39
1029.59
Authorshi--Grpphhodeeeen
58.82
1227.36
75.49
632.10
80,39
696.64
Auhonship-Cod:75
64,95
1078.40
66,97
715.89
71.79
970.44
Jade25+CodeBert
50,50
958.96
74.03
961.60
75.12
815.91
46,74
1026, 15
46.05
946.52
72.30
853.74
Iava250+Code15
52.04
1189.42
30,59
1107.95
63.80
1049.46
Python800 CodeBBet
58.30
513.63
56.67
919.37
77.88
514.19
51.87
577.70
54.15
917.92
71.42
730.14
Python800 CodeeT
777.20
36,95
1127.44
69.07
662.28
C1000+CoddBet
525.43
340.88
72.96
537.76
GrapoGGrphCodeeee
52.68
566.18
45,93
837.09
72.23
634.27
C1000+CadeT5
47.86
843,33
36.45
668.15
59.00
697.06
Count
0/18
4/18
0/18
6/18
18/18
8/18
Table 2: Comparison results with MHM, and ALERT, ASR %. Count: the number of best results achieved.
5 Results Analysis
ASR of 89.45% but QT is the smallest. Therefore, there is no absolute causal relationship between QT and ASR.
5.1 Attack Effectiveness and Efficiency
We compare RNNS with two methods MHM (Zhang al., 2020) and NaturalAt- tack (ALERT) (Yang et al., 2022) on six datasets and 18 victim models that have been fine-tuned for the downstream tasks. Table 2 shows the comparison results where the last row Count indicates how many times this method achieves the best results. We can see that RNNS achieves the best performance for 18/18 times in terms of ASR, and the lowest cost for 8/18 times in terms of QT in Table 2. Both of the indicators are better than the baselines. The two baselines have zero best ASR for all victim models and all datasets. The lowest QTs achieved by ALERT and MHM are 4 and 6, respectively. We conclude that for effectiveness and efficiency, RNNS outperforms ALERT and MHH in all cases. Especially, MHM and ALERT fail attack GraphCodeBERT BigClone dataset, and only have 9.58% and 10.4% ASR respectively, while RNNS has more than 40% ASR. RNNS has almost two times larger ASR than MHM on Java250+CodeTS and Python800+CodeT55.
5.2 Perturbation of Adversarial Example
We conduct a study about the quality of the ad- versarial examples to check if RNNS can generate looking- code, e.g., avoiding naively increas- ing the variable name length. To do so, firstly, we count the average length of the original variable and adversarial variables as demonstrated by Ta- ble 3. We also compute the mean and variances of their difference. Besides, we compute the average number of the replaced variables for the successful attack as shown in Table 4. Low values mean the inputs are modified less, and high qualities.
In Table 3, the 2nd, 5th, and 8th columns are the average length for original variables (named Var Len) that are replaced. The 3rd, 6th, and 9th columns are the average lengths for adver- sarial variables (named Adv Var Len). The 4th, 7th, and 10th columns are the average and vari- ance (mean & variance) of the absolute length difference between original variables and adversar- ial variables (named Difference). We observe that MHM prefers to replace the long-length variables while RNNS likes replacing short-length variables if we compare the 2nd and 5th columns. Mean- while, the change of variable length from RNNS is less than MHM. MHM introduces the average length difference of 3.39-6.82 while RNNS only has 2.02-2.54. MHM has much higher variances than RNNS in the length change. ALERT uses shorter adversarial variable names than RNNS
It should be noted that high ASR is not due large QT. As shown in Table 2, the three groups of experiments with the most QTs are Clone-GraphCodeeeer,, Java250+CodeTS, and Au- thorship+CodeBerr, with ASRs 41. .2%% 63.80% and 73.39%, respectively, which are not the highest. On the contrary, Defect+CodeT5 has the highest
hhctps://git con/1 htw/18867222166/MMB for-codo--ataaa

---

RNNS
MHM
ALERT
Var Les
Adv Var Los
Diftrence
Var Les
Adv Var Len
Diffrrnce
VarLei
Adv Var Lei
Diffcrncce
ClourCCudeett
6.12
6.79
2.35 1 4,50
6.47
10.6
6.34 1 10.98
5.91
6.21
1.32 1 2.02
Cloeeerr
6.32
6.97
2.54
6.43
6.58
10.41
6.82
21.67
5.93
1.45
2.49
ClenssCrdeTS
6,45
6.69
2.51
+ 8.30
6.46
10.46
6.17 + 25.78
6.25
6.61
1,32 + 2.72
Delect-Cdeeeett
4.64
5.44
2.08 1. 2.49
4.44
9.59
6.57 1 28.78
4.85
5,06
1,36 1.93
Driss-GruphCokseett
1.83
Codess
3,95
5.17
2.03 + 1.93
4.33
9.81
6,59 + 29.98
4_36
5.01
1,27 + 1.57
Authonhip+Codeees
3.81
5.18
2.28 + 1.56
3.97
7.94
5.45 + 16.72
4.42
5.35
1,40 2.25
AullomshippGrapphCCooo
7.64
1.82
Aulhonship+odde15
3,95
5.18
2,03 + 2,66
3,95
7,98
5.59 + 20.94
3.81
4.50
1.22 : 1.62
Inva250=CodeBert
2.35
4.22
2.11 +: 1.02
3.21
6.50
4.34 + 15.20
3,22
3.65
0,94 :: 1.63
6.59
14.84
1.54
lava250-Code115
2,76
4,47
2.10 = 1.17
3.20
6.54
4.33 + 14,60
3.16
7.31
4,41 18.73
Python800+Codeen
1.50
3.54
2.21 1: 1.02
1.97
5.11
3.64 9.06
1.78
2.27
0.64 1: 1.34
Pyben800+GruphCooo
Python800+Co613
1.65
3,59
2.13 + 0.95
1.97
4.95
3.49 + 8.18
1.88
5.84
4,10 12.64
C1000-CodeBert
1.58
3.44
2.08 + 0.88
2.41
5.05
3.65 + 12.02
2.13
2.52
0.67 + 1.17
CrDD-rraphCudeeett
3.59
2.10
12.98
C1000-CodeBer
1.38
3.33
2,02
0.85
2.36
4.82
3.39
10.98
2.10
6.56
4,74 13.24
Table 3: Replaced-varialte length comparison, mean + variance.
with less change because it uses the pre-trained model to generate the replacements that are close to the replaced variables.
5.4
Attack Defended Model and Retraining
Attack Defended Model. We employ RNNS and MHM to attack the three defended models pro- vided by ALERT (Yang et al., 2022). These mod- els are prepared by adversarial fine-tuning. Table 6 presents the results. We can see that RNNS outper- forms MHM in two tasks, and MHM better in one task. This experiment setting actually is not friendly for RNNS because ALERT (Yang et al., 2022) uses the replacements from pre-trained mod- els which implicitly have the semantic constraint. Retraining. We the adversarial examples from RNNS retrain the victim models of CodeBERT by contrastive adversarial learning. We use three 3 datasets, Defect, Authorship, and Java250. We gen- erate the adversarial examples on the whole train- ing dataset for them. Table 7 presents the results, all approaches achieve much lower ASR compared with the previous. RNNS adversarial examples can improve the mode robustness through contrastive adversarial retraining. If compare Defect/Au- thorship+CodeBERRT Table 7 and Table 6, we can find that both retrained models via RNNS are more robust than the models from ALERT since they have much lower ASRs.
Table 4 statistically shows the number of re- placed variables. It can be seen that RNNS re- places around average of 3.6 variables with smaller variance of around (3.4-4.6) while MHM needs to modify about an average of 5.4 variables with larger variance (2 11.14). ALERT also re- places more variables to attack models than RNNS and MHM. RNNS introduces less or equal pertur- bation than the baselines in terms of length change and change number.
Figure 2 shows one example RNNS, MHM, and ALERT attack successfully from the Java250 dataset. The changes are highlighted by shadow markers. RNNS only renames one variable b to h, ALERT renames two variables, while MHM almost renames all variables and also prefers longer names.
5.3 Ablation Study
We remove the two search constraints in Sec- tion 3.2.4, denoted this variant of RNNS RNNS- Unlimited. Table shows the comparing results between RNNS-Unlimited and RNNS. RNNS- Unlimited gets the first place for all the tasks in terms of ASR. ASR can be improved by maxi- mum of 8.35% and minimum of about 2% after removing limitations. For QT, RNNS-Unlimited only loses 3 times among 18 evaluations. The improvement of RNNS-Unlimited is not surpris- ing with respect ASR and QT. Because RNNS- Unlimited can search the adversarial examples in the non- similar real names and use very long vari- able names.
5.5 RNNS vs Textual Attack Methods
To compare the effects of RNNS and textual at- tack methods, We conducted attack experiments on three datasets using the PSO (Zang et al., 2020) and LSH (Maheshwary et al., 2021). The three datasets Defect, Authorship, and Java250, represent three languages respectively, C, Python, and Java. To be fair, the search space of the PSO and LSH the same that of RNNS.
As shown in Table 8, the QT of PSO algorithm

---

CodeBERT
GraphCudeBERi
CodeTS
Task
RNNS
MIIM
ALERT
RNNS
MIDM
ALERT
RNNS
MIIM
ALERT
Clome
3.55 1 4,60
6.72 1 16.57
6,86 18.85
4.12 1. 4.94
6.21 L 15.13
6,95 18.99
3.43 5.00
5.68 1. 14.01
7.65 1 25.57
Defect
3.39 4.96
2.78 + 7.89
3.49
+ 3.9
2.67 + 1.75
2.84
+.00+11:0
2:51
1.45
2.16 3.58
3.49
Authorship
4.24 7.47
7.52
25.82
6.60
22.96
3,65 + 3.332
6.67
+ 22.29
7.75 33.12
4.39
9.00
5.72
13.02
6.06 + 18.74
Java250
3.87 1 4,70
7.11
21.11
7.82
28.96
3.87 1 4.25
6.41 16.24
7,83 25.06
4.71 6.87
7.04
15.29
8.92 25.97
Pycho8800
3,06 1.87
12.28
8.47
4.12 4:12:::66
3.57
5.29
C1000
3,00 1,86
4,42
+ 7.49
4.13
+133+-5.59 5.59
3.37 + 2.38
5.14 + 7.30
4,88 + 6.24
3.39
2,48
5.20
7.43
5.43 + 6.99
mcan
3.52 + 4.24
5.63
+ 15.21
5.65 14,80
3,6 3.39
5.38 + 13.55
6.02 16.87
3.67
4.64
5.18 + 11.14
6.29
15.45
Table 4: Seplaced-variabll number comparison, mean + variance
publite
pablic static Strntnr argat Iot retgit Fistscareer reatintt
public statts voad maze easrintiingl angs) bystermarbyatllnl
Scomer db) row SpareriSystewwiii untillll to
Scanner 083 Sconert5ysten.3nn rroctertii
Scurer
NATRER 0b3 mestttl
int Eastscarmet nestintlly
Bars- ob) mes-
fet our whill ( out S
Lst sut 22 wiile out STRIS out ous
Ecmpoo 32
Raaher..arnng aarring
and 913
white S0ottoell1
TORMOSS11
ext out
tamber Ki
Syston oxt.rractbnlasss
System.out. grscen.out.puuntttiieee trantla
System owt..oott.tttt........ prdnnt Bubber srrag
Inlanal)
Original Codee
Adversarial Code from RNNS
Adversarial Code from MHM
Adversarial Code from ALERT
Figure 2: Case study. Original vs. RNNS vs. MHM vs. ALERT
CodeBERT
GraphCadeBERT
CodeT5
Task
RNNS-
Unlimited
RNNS
UANS-Unlimited
RNNS
MUNSS-Unimited
RNNS
ASR
QT
ASR
QT
ASR
QT
ASR
QT
ASR
QT
ASR
QT
Defect
72.29
590.98
69.18
588.35
87.77
381.82
81.63
404.73
91.64
338.41
89.45
344.29
Clone
50.66
955.97
46.50
666.48
48.16
1105.11
41.28
1122.01
41.38
920.65
39.61
895.79
Authorship
91.74
447.68
73.39
1029.59
91.17
438.69
80.39
696.64
88,88
620.56
71.79
C1000
74.70
502.02
72.96
537.76
76.82
498.64
72.23
634.27
61.6
704.95
697.06
Python800
83,90
460.92
77.88
514.19
79.00
496.30
71.42
730. 14
72.69
646.59
69.07
662.28
Java250
79.70
760.97
75.12
815.91
81.94
744.57
72.30
853.74
75.52
910.97
63.80
1049.46
Count
6/6
4/6
0/6
2/6
6/6
6/6
0/6
0/6
6/6
5/6
0/6
1/6
Table 5: Results of ablation study, before and after removing constraints, ASR %.
RNNS
MHM
RNNS
PSO
LSH
Defended Model
ASR
QT
ASR
QT
Task+Mookl
ASR
ASR
Clone+Codeeees
12.90
958.35
28.17
1245.75
Defest+CodeBert
69.18
568.35
63.63
3945.01
26.62
321.78
Autership-Codddert
52.29
4350.00
Defect-CodeBertt
95.37
282.20
92.23
283.66
Java250CCoddert
75.12
815.91
47.3
5076.02
31.58
397.05
Authorship-CodeBet
51.88
1524.40
43.26
1026.08
Table 8: RNNS vs PSO and LSH, ASR %.
Table 6: Attack defended models, ASR %.
of word replacement natural language semantics.
ACC
ASR(RNNS)
ASR(MHM)
ASR(ALERT)
Authorship
90.62
19.81
23.58
14.28
Defect
65.14
40.46
23.69
24.53
RNNS's QT is 1.8-2.2 times that of LSH, and the QT has dropped significantly. However, LSH's ASR is inferior to RNNS by 42.56%-55.13%%. For code variable attacks, LSH has high efficiency, but its effcctiveness is relatively low. One possible reason for LSH causing low ASR is the distribution of adversarial samples in each bucket is uneven.
Java250
19.67
42.91
Table 7: Results of contrastive adversarial retraining, model: CodeBERT.
is 4.22-6.7 times that of RNNS, and the ASR of PSQ algorithm is 5.55% 27.82% lower than that of RNNS algorithm. It can be inferred that for code variable attacks, combinatorial optimization is inefficient when the substitute set of variables is relatively large. The main reasons the following two points. Firstly, code segments are generally longer, and the substitute set of code variables is much larger than the synonym set of natural lan- guage words. Secondly, the impact of variable replacement on code semantics is smaller than that
Related Work
Adversarial attacks for code models have been widely studied (Yang et al., 2022; Liu al., 2023a; Li al., 2023; Jha and Reddy, 2023). These works can be generally categorized into black-box attacks and white-box attacks. A black-box attack for code models queries the model outputs and selects the substitutes using a score function. For example,

---

Algrxittin
Subeiiutes
Substitutes Source
Replacement Position
Substitules
three main contributions: 1) This work proposes a non-generation search-based black-box attacking method via predicting the attack effect of a substi- tute. This method can greatly reduce the verifica- tion cost of the substitute, remove the restrictions on the size and diversity of the substitute set, and achieve a significant improvement in terms of ASR without increasing QT. 2) This work proposes simple and efficient method for constructing sub- stitute set. This method can construct a large-scale, diverse, and real substitute set at low cost. 3) The adversarial examples from RNNS can be used to improve the model robustness.
Selection
random
vocabulary
sample
ALERT
small
madel
impartance
generation
soore
real
efficient
uncertainty score
RNNS
lure
public variables
constriined
search
Table 9: Difference between RNNS to the others.
ALERT (Yang et al., 2022) finds the adversarial ex- amples using variable-name substitutes generated by pre-trained masked models. MHM (Zhang et al., 2020) uses Messings to sample the re- placement of code identifiers. STRATA (Springer al., 2020) generates adversarial examples by re- placing the code tokens based on the token dis- tribution. Chen et al. (2022) apply pre-defined semantics-preserving code transformations to at- tack code models. Code (Jha and Reddy, 2023) uses code structure to generate adversarial data. White-box attacks require the code model gradient to modify inputs for adversarial exam- ple generation. CARROT (Zhang al., 2022) se- lects code mutated variants based the model gradient. Henkel et al. (2022) attack code mod- els by gradient-based optimization of the abstract syntax tree transformation. Srikant et al. (2021) uses optimized program obfuscations to modify the code. DAMP (Yefet et al., 2020) derives the de- sired wrong prediction by changing inputs guided by the model gradient.
8 Limitations
There are some limitations of RNNS. Firstly. RNNS does not revert to the preceding step to persist with the search upon increase in the model probability of the ground truth label. While the incorporation of this step may bolster the At- tack Success Rate (ASR), it could potentially com- promise the Query Time (QT). Secondly, the size and diversity of the substitute set significantly in- fluence RNNS; a minimal and homogeneous set can precipitate a diminished attack success rate. Thirdly, RNNS involves multiple hyperparameters whose values need to be manually set. One of the most important parameters is the moving parameter a. The number of attacking iterations max itr is also significant. We set to 0.2 and max_itr to with some small experimental trials. Fourthly, RNNS currently only targets untargeted attack sce- narios, for targeted attacks, ASR will be very low when there are many category labels. For exam- ple, when performing targeted attacks Author- Ship+Codebert with 66 labels, the ASR only reach 6.4%. How migrate targeted attacks direction we need to study in the future.
Table 9 demonstrates the differences among RNNS, MHM (Zhang et al., 2020) and ALERT (Yang et al., 2022). MHM and ALERT represent the two methodologies most closely aligned with our research. Our approach considers identifier replacements like MHM and ALERT, ensuring that the adversarial example keeps the same semantics as the original one. Our substitute size is scalable and can be substantial, and RNNS searches the possible next adversarial example in the substitute space. In our approach, we locate vulnerable variables based on the uncertainty and search substopk without building adversarial samples and actual attacks. Our goal is to obtain high ASRS by searching real variable names. MHM has the same goal as ours but synthesizes variable names. ALERT sacrifices ASR to make the variable name readable.
A Acknowledgment
This work supported by NRF and the CSA un- der its National Cybersecurity R&D Programme (NCRP25-P04-TTICCN) NRF and DSO National Laboratories under the AI Singapore Programme (AISG Award No: AISG2-RP-2020-0119 and NRF Investigatorship IRF-NRFI06-20200 opinions, findings and conclusions or recommen- dations expressed in this material are those of the author(s) and do not reflect the views of NRF and CSA Singapore.
Any
7 Conclusion
We propose novel black-box adversarial search- based attack for variable replacement. RNNS has

---

References
Yaoxian Li, Shiyi Qi. Cuiyun Gao, Yun Peng, David Lo, Zenglin Xu, and Michael R Lyu. 2022. A closer look into transformer- code transformation: Challenges and opportunities. arXiv preprint TrXii:2207.042885
Wasi Ahmad, Saikat Chakraborty, Baishakhi Ray, and Kai- Mei-Wei Chang. 2020. A transformer-based ap- proach for source code summarization. In Proceed- ings of the 58th Annual Meeting of the Association for Computational Linguistics, pages
code intelligence through
Shangqing Liu, Yu Chen, Xiaofei Xie, Jing Kai Siow, and Yang Liu. 2020. Retrieval-augmente generation for code summarization via hybrid gnn. In Interna- tional Conference on Learning Representations.
Bander Alsulami, Edwin Dauber, Richard Harang, Spiros Mancoridis, and Rachel Greenstadt. 2017. Source code authorship attribution using long short- term memory based networks. In Computer Security ESORICS 2017, pages 65-82, Cham. Springer In- ternational Publishing.
Shangqing Liu, Bozhi Wu, Xiaofei Xie, Guozhu Meng. and Yang Liu. 2023a. Contrabert: Enhancing code pre-trained models via contrastive learning. In 2023 IEEE/ACM 45th International Conference on Soft- ware Engineering (ICSE), pages 24777
Penglong Chen, Zhen Li. Yu Wen, and Lili Liu. 2022. Generating adversarial source programs using im- portant tokens-based structural transformations. In 2022 26th International Conference on Engineering of Complex Computer Systems (ICECCS), pages 173- 182.
Shangqing Liu, Xiaofei Xie, Jingkai Siow, Lei Ma. Guozhu Meng, and Yang Liu. 2023b. Graphsearch- net: Enhancing gnns via capturing global dependen- cies for semantic code search. IEEE Transactions on Software Engineering.
Zhangyin Feng, Daya Guo, Duyu Tang. Nan Duan, Xi- aocheng Feng, Ming Gong, Linjun Shou, Bing Qin, Ting Liu, Daxin Jiang, et al. 2020. Codebert: A pre-trained model for programming and natural lan- guages. In Findings of the Association for Computa- tional Linguistics: EMNLP 2020, pages 1536-1547.
Rishabh Maheshwary, Saket Maheshwary, and Vikram Pudi. 2021. A strong baseline for query efficient attacks in a black box setting. In Proceedings of the 2021 Conference on Empirical Methods in Natural Language Processing, pages 8396-8409.
Xiaodong Gu, Hongyu Zhang, and Sunghun Kim, 2018. Deep code search. In 2018 IEEE/ACM 40th Interna- tional Conference on Software Engineering (ICSE), pages 933- 944. IEEE
Ruchir Puri, David Kung. Geert Janssen, Wei Zhang, Giacomo Domeniconi. Vladmir Zolotov, Julian Dolhy, Jie Chen, Mihir Choudhury, Lindsey Decker, Veronika Thost, Luca Buratti, Saurabh Pujar, Shyam Ramji, Ulrich Finkler, Susan Malaika, and Freder- ick Reiss. 2021. Codenet: A large-scale ai for code dataset for learning a diversity of coding tasks.
Daya Guo. Shuo Ren, Shuai Lu, Zhangyin Feng, Duyu Tang, LIU Shujie, Long Zhou, Nan Duan, Alexey Svyatkovskiy, Shengyu Fu, et al. 2020. Graphcode- bert: Pre-training code representations with data flow. International Conference on Learning Representa- tions.
Jacob M Springer, Bryn Marie Reinstadler, and Una- May O' O'Reilly. 2020. Strata: Simple, gradient- free attacks for models of code. arXiv preprint 20ti:2200.135222
Jordan Henkel, Goutham Ramakrishnan, Zi Wang. Aws Albarghouthi, Somesh Jha, and Thomas Reps. 2022. Semantic robusmesss of models of source code. In 2022 IEEE International Conference on Software Analysis, Evolution and Reengineering (SANER), pages 526-537.
Shashank Srikant, Sijia Liu, Tamara Mitrovska, Shiyu Chang, Quanfu Fan, Gaoyuan Zhang, and Una-May O'Reilly, 2021. Generating adversarial computer pro- grams using optimized obfuscations. International Conference on Learning Representations.
Akshita Jha and Chandan K Reddy. 2023. Codeat- tack: Code-based adversarial attacks for pre- trained programming language models. Proceedings of the AAAI Conference on Artificial Intelligence, vol- ume 37, pages 14892-14900.
Wenhan Wang, Ge Li, Bo Ma, Xin Xia, and Zhi Jin. 2020. Detecting code clones with graph neu- network and flow-augmented abstract syntax tree. In 2020 IEEF 27th International Conference Software Analysis, Evolution and Reengineering (SANER), pages 261-271. IEEE
Liuqing Li, He Feng, Wenjie Zhuang, Na Meng, and Barbara Ryder. 2017. Celearner: A deep learning- based clone detection approach. In 2017 IEEE Inter- national Conference on Software Maintenance and IEEE
Evolution (ICSME). pages
Yue Wang. Weishi Wang. Shafiq Joty, and Steven C.H. Hoi. 2021. CodeT5: Identifier-aware unified pre- trained decoder models for code understand- ing and generation. In Proceedings of the 2021 Conference on Empirical Methods in Natural Lan- guage Processing, pages 8696-8708, Online and Punta Cana, Dominican Republic. Association for Computational Linguistics.
Yanzhou Li, Shangqing Liu, Kangjie Chen, Xiaofei Xie, Tianwei Zhang, and Yang Liu. 2023. Multi- target backdoor attacks for code pre-trained models. In Proceedings of the 61st Annual Meeting of the Association for Computational Linguistics ( (Volume Long Pupers), pages 7236-7254, Toronto, Canada. Association for Computational Linguistics.

---

Martin White, Michele Tufano, Christopher Vendome, and Denys Poshyvanyk. 2016. Deep learning code Fragments for code clone detection. In 2016 st IEEE/ACM International Conference on Automated Software Engineering (ASE), pages 87-98. IEEE.
Zhou Yang. Jieke Shi, Junda He, and David Lo. 2022. Natural attack for pre-trained models of code. In Proceedings of the 44th International Conference on Software Engineering, ICSE '22, page 1482-1493, New York, NY, USA. Association for Computing Machinery.
Noam Yefet, Uri Alon, and Eran Yahav. 2020. Adversar- ial examples for models of code. Proceedings of the ACM on Programming Languages, 1( 30.
Yuan Zang. Fanchao Qi, Chenghao Yang. Zhiyuan Liu, Meng Zhang, Qun Liu, and Maosong Sun. 2020. Word-level textual adversarial attacking combi- natorial optimization. nual Meeting of the Association for Computational Linguistics, pages 4066-6080.
Proceedings of the 58th
Huangzhao Zhang. Zhiyi Fu, Ge Li, Lei Ma, Zhehao Zhao, Hua' an Yang, Yizhe Sun, Yang Liu, and Zhi Jin. 2022. Towards robustness of deep program pro- cessing models detection, estimation, and enhance- ment. ACM Trans. Softw. Eng. Methodol., 31(3).
Huangzhao Zhang, Zhuo Li, Ge Li, Lei Ma, Yang Liu, and Zhi Jin. 2020. Generating adversarial exam- ples for holding robustness of source code processing models. Proceedings of the AAAI Conference on Artificial Intelligence, volume 34, pages 1169-111 1176.
Vitalii Zhelezniak, Aleksandar Savkov, and Nils Ham- merla. 2020. Estimating mutual information between dense word embeddings. In Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics, pages 8361-8371, Online. Association for Computational Linguistics.
Yaqin Zhou, Shangqing Liu, Jingkai Siow, Xiaoning Du, and Yang Liu. 2019. Devign: Effective vulnerability identification by learning comprehensive program semantics via graph neural networks. Advances in neural information processing systems, 32.


Reviews:
[{"review_id": "QiWwKgHeSR", "reviewer": "Reviewer_Wyoa", "summary": "", "questions": "1.\tWhat are ExtractVar and RankVarsWithUncertainty exactly in section3.2.1? What are the specific processes?\n\n2.\tIn Table2, on some data sets, the query time of this method is much higher. Why is this result not discussed? Is this the reason for the high attack success rate?\n", "limitations": "", "rating": 3, "confidence": 3, "soundness": 3, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "Thank you for recognizing the validity and innovation of our method. We appreciate your valuable feedback. Due to space constraints in the main text, we did not provide a detailed description. We will refine it further in the final version, including the abstract.  We have carefully read your questions and provide the following answers.\n\n**For Q1**:\n\n**ExtractVar** implementation consists of three steps:\n1. Use the open-source tool tree-sitter to extract all variables in the current dataset and perform deduplication filtering.\n2. For each legal variable, we tokenize the variable and then take the embedding of each token through the variable-name encoder pre-trained on [CodeSearchNet](https://huggingface.co/datasets/code_search_net) independently by masked language modelling, and perform mean pooling operation on tokens to obtain the embedding of variable.\n3. Save all variables and their corresponding embeddings to obtain the initial substitutes set.\n\n**RankVarsWithUncertainty** implementation consists of five steps:\n1. Specify a fixed variable array, named, $VarArray$. In this experiment, the size of this variable array is 16, and the variable length ranges from 1 to 5.\n2. For a certain variable in the code, we replace it with each variable in $VarArray$ in turn, so that 16 mutated codes are obtained.\n3. Query the victim model with these 16 mutated codes to obtain the probability distribution of each category.\n4. Calculate the variance of each category probability distribution.\n5. Calculate the mean of all category variances as the uncertainty of variable. \nThe greater the uncertainty of a variable, the more valuable it is for attack.\n\n**For Q2**:\n\nIn Table 2, the three groups of experiments with the most attack attempts are Clone+GraphCodeBert, Java250+CodeT5, and Authorship+CodeBert, with ASRs of 41.28%, 63.80%, and 73.39%, respectively, but they are not the highest attack success rates. On the contrary, the experiment with the highest ASR is Defect+CodeT5, which has an ASR of 89.45%, but its QT is the smallest among all experiments. Therefore, there is no absolute causal relationship between QT and ASR.", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "1.\tThe motivation of the article is relatively clear, pointing out three limitations of current research and giving solutions for each limitation.\n\n2.\tThe strategy the article proposed for selecting the substitute variable from substitute sets has a certain degree of innovation and rationality to achieve lower cost.\n", "reasons_to_reject": "1.\tThe abstract is too long to understand. It does not show the key points and advantages of the article.\n\n2.\tMany specific details are missing in the method part, and the specific process of each module is not clearly explained.\n\n", "questions_to_authors": "1.\tWhat are ExtractVar and RankVarsWithUncertainty exactly in section3.2.1? What are the specific processes?\n\n2.\tIn Table2, on some data sets, the query time of this method is much higher. Why is this result not discussed? Is this the reason for the high attack success rate?\n", "paper_topic_and_main_contributions": "This paper proposes a search-based black-box adversarial attack method to create challenging adversarial samples based on the search seed vector in the variable representation space. This method can greatly reduce the verification cost of the substitute, remove the restrictions on the size and diversity of the substitute set, and achieve a significant improvement in terms of ASR without increasing QT. And it also can construct a large-scale, diverse and real substitute set at low cost.The adversarial examples from RNNS can be used to improve the model robustness.", "excitement": 3, "reproducibility": 3, "ethical_concerns": "No", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {}, "extra_text": {}}, {"review_id": "TaNxmfqIPB", "reviewer": "Reviewer_N9F5", "summary": "", "questions": "- The baseline methods and the proposed RNNS utilize greedy-based attack algorithm. However, there exist several research that utilize combinatorial optimization to attack the language model [4,5]. Can you provide attack result of RNNS equipped with [4] and [5]? Also, you may further accelerate the attack process using locality sensitive hashing by adapting the protocol of [6]. \n- I am little confused with threat model. According to the section 3.1, it seems that RNNS utilizes training code dataset to train the encoder model, which requires strong assumption on the attack setting. Is your threat model permit access to the training dataset of the victim code model? If not, please clarify which data you utilized to train the encoder model. \n- Can you provide some additional qualitative research in the revised version?\n\n\n[4] Word-level Textual Adversarial Attacking as Combinatorial Optimization, Zang et al., ACL 2020\n\n[5] Query-Efficient and Scalable Black-Box Adversarial Attacks on Discrete Sequential Data via Bayesian Optimization, Lee et al., ICML 2022\n\n[6] A Strong Baseline for Query Efficient Attacks in a Black Box Setting, Maheshwary et al., EMNLP 2021", "limitations": "", "rating": 3, "confidence": 4, "soundness": 3, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "Thank you for your feedback. Our attack algorithm is consistent with the idea of greedy strategy, attacking in the most likely direction. Our attack algorithm is based on the word-substitution method in the code model scenario. We have carefully read your questions and provided references.\n\n**For Q1**:\n\nWe conducted attack experiments on three datasets using the PSO[4] algorithm and LSH[6] algorithm . The three datasets, Defect, Authorship and Java250, represent three languages respectively, C, Python and Java. To be fair, the search space of the algorithm is the same as that of RNNS. The experimental results are shown in the following table:\n\n\n|     　                     |      RNNS    |   RNNS   |      PSO     |    PSO   |      LSH     |    LSH   |\n|----------------------------|:------------:|:---------------:|:------------:|:---------------:|:------------:|:-------------:|\n|     　                     |     ASR      |     QT          |     ASR      |     QT          |     ASR      |     QT        |\n|     Defect+CodeBert        |     **69.18**    |     588.35      |     63.63    |     3945.04     |     26.62    |    **321.78**    |\n|     Authorship+CodeBert    |     **73.39**    |     1029.59     |     52.29    |     4350.00     |     19.26    |     **458.55**    |\n|     Java250+CodeBert          |     **75.12**    |     815.91      |     47.3     |     5076.02     |     31.58    |     **397.05**    |\n\n\n\n\n1.\tPSO's pop_size is set to 10, and max_iters is 6. As shown in the table above, the QT of the PSO algorithm is 4.22-6.7 times that of RNNS, and the ASR index of the PSQ algorithm is 5.55% - 27.82% lower than that of the RNNS algorithm. It can be inferred that for code variable attacks, combinatorial optimization is inefficient when the substitutes set of variables is relatively large.\n2.\tWe hash adversarial samples corresponding to each variable into 16 buckets through the LSH algorithm. To improve ASR, each bucket is sampled up to 6 times. Assuming that a piece of code has 6 variables, the maximum number of attacks is 576. As shown in the table above, RNNS’s QT is 1.8-2.2 times that of LSH, and the QT index has dropped significantly. However, at the same time, LSH’s ASR index is inferior to RNNS by 42.56%-54.13%. One possible reason for LSH causing low ASR is that the distribution of adversarial samples in each bucket is uneven.\n\nWe did not conduct comparative experiments on Blockwise Bayesian Attack [5] for the main reason: \nTo achieve a scalable Bayesian optimization algorithm, Blockwise Bayesian Attack decomposes an input sequence into disjoint blocks. However, for code, the same variable may be distributed anywhere in the code, making it difficult to partition.\n\n**For Q2**: \n\nWe do not use the six datasets in the experiments for the victim models. The variable name encoder is pre-trained on [CodeSearchNet]( https://huggingface.co/datasets/code_search_net) by masked language modelling independently. \n\n**For Q3**:\n\n1. There are some differences between adversarial attacks on code variables and those on natural language: code segments are generally longer, and the substitutes set of code variables is much larger than the synonym set of natural language words. The impact of variable replacement on code semantics is often smaller than that of word replacement on natural language semantics. Therefore, some combinatorial optimization algorithms that are highly efficient for natural language attacks face many challenges in code variable attacks. \n2. LSH is an effective means to significantly reduce QT. However, in the field of code variable attacks, how to achieve a uniform distribution of adversarial samples in each bucket is a topic that needs further research.\n\n We will supplement more detailed content in the revised version.", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "- The empirical results show the superiority of the proposed method compared to the baseline method.\n- The method is simple and intuitive.\n- This attack algorithm of RNNS follows the algorithm of greedy-based word-level adversarial attack research [1,2,3]. If I understood correctly, there are two differences.\n 1. In the adversarial attack on code models, tokens corresponding to the same variable should be perturbed into the same variable. \n 2. Due to difference in token distributions of code models and other NLP models, adversarial attack method on code models cannot directly use the word (token) substitution method of word-level adversarial attack.\n\nEven though the proposed method is not algorithmic novel, it is important research that suggests base word-substitution method for code models.\n\n[1] Generating Natural Language Adversarial Examples through Probability Weighted Word Saliency, Ren et al., ACL 2019\n\n[2] Is BERT Really Robust? A Strong Baseline for Natural Language Attack on Text Classification and Entailment, Jin et al., AAAI 2020\n\n[3] BERT-ATTACK: Adversarial Attack Against BERT Using BERT, Li et al., EMNLP 2020.\n\n", "reasons_to_reject": "- Please refer to questions.", "questions_to_authors": "- The baseline methods and the proposed RNNS utilize greedy-based attack algorithm. However, there exist several research that utilize combinatorial optimization to attack the language model [4,5]. Can you provide attack result of RNNS equipped with [4] and [5]? Also, you may further accelerate the attack process using locality sensitive hashing by adapting the protocol of [6]. \n- I am little confused with threat model. According to the section 3.1, it seems that RNNS utilizes training code dataset to train the encoder model, which requires strong assumption on the attack setting. Is your threat model permit access to the training dataset of the victim code model? If not, please clarify which data you utilized to train the encoder model. \n- Can you provide some additional qualitative research in the revised version?\n\n\n[4] Word-level Textual Adversarial Attacking as Combinatorial Optimization, Zang et al., ACL 2020\n\n[5] Query-Efficient and Scalable Black-Box Adversarial Attacks on Discrete Sequential Data via Bayesian Optimization, Lee et al., ICML 2022\n\n[6] A Strong Baseline for Query Efficient Attacks in a Black Box Setting, Maheshwary et al., EMNLP 2021", "paper_topic_and_main_contributions": "- The paper proposes representation nearest neighbor search (RNNS), a novel black-box adversarial attack method on code models. \n- RNNS trains an additional encoder which embeds a variable into a continuous representation vector.\n- Then, RNNS construct variable susbstitute candidates as nearest neighborhood set in the embedding space.\n- RNNS measures the uncertainty of each variable in the code and sequentially perturbs variables from higher uncertainty to lower uncertainty until attack success.\n- The empirical results show that RNNS achieve higher attack success rate compared to the baseline methods.", "excitement": 3, "reproducibility": 3, "ethical_concerns": "No", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {}, "extra_text": {}}, {"review_id": "DHqfqbfYoz", "reviewer": "Reviewer_fokm", "summary": "", "questions": "Question a: In the subsection 3.2.2 \"Computing Uncertainty\", is a new $X_mutated$ being created for each $var \\in x$? In the equation above line 307 and the definition of $P_i$ on line 307, I'm not seeing the reference to a specific $var$ for which the variance is being calculated.\n\nQuestion b: In section 3.2.4 \"Searching Top-K Substitutes\" you say that you filter out all variable names that don't have a similarity of less than $\\epsilon$. I'm not sure it's stated elsewhere but what is this similarity function and what is $\\epsilon$ set to? Do you know how many substitutions it filters out? I've seen previous work on adversarial attacks where their constraints were too tight that no actual search was being performed as there was only one possible token to select.\n\nQuestion c: Regarding the future work on targeted attacks. That's the case where you want to get the model to predict a certain label, right? Couldn't you change line 17 of algorithm 1 to increase the probability of the target label instead of just decreasing the probability of the current label?", "limitations": "", "rating": 3, "confidence": 2, "soundness": 3, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "Thank you for acknowledging the innovation of our method. Code models represent a highly significant and promising application in NLP with substantial industrial value. By QT~ (query times) here, we refer to the number of times the API is queried, not the duration. We used this term to stay consistent with the baseline's original paper, making comparisons more straightforward. We have carefully read your questions and provide the following answers.\n\n**For Q1**:\n\nYes, we create a set of mutated codes for each variable in code X.\nRankVarsWithUncertainty implementation consists of five steps:\n1. Specify a fixed variable array VarArray. In this experiment, the size of the variable array is 16, and the variable length ranges from 1 to 5.\n2. For a certain variable var in the code, replace it with each variable in VarArray in turn, so that 16 mutated codes are obtained.\n3. Query the victim model with these 16 mutated codes to obtain the probability distribution of each category.\n4. Calculate the variance of each category probability distribution.\n5. Calculate the mean of all category variances as the uncertainty of variable var. \nThe greater the uncertainty of a variable, the more valuable it is for attack.\n\n**For Q2**:\n\nWhen selecting TOPK Substitutes, we use cosine similarity. We construct a candidate set of substitutes with similarity > 0.85 (ϵ=0.15) and length difference less than 4 (δ=4). The initial substitutes for the variables are in the tens of thousands (for example, Java250 is 29, 255). After our similarity and length filtering, there are thousands of carefully selected substitutes left (the average selected candidate set for the Java250 variable is 2, 792). Each iteration then selects the top 60 (k=60) most similar substitutes as TOPK substitutes.\nWe will specify the values of each parameter in the revised version.\n\n\n**For Q3**:\n\nYes, targeted attacks refer to making the model predict a specified label. For future study, we conducted experiments on targeted attacks using the method you mentioned. When there are many category labels, the success rate of attacks is very low. For example, when performing targeted attacks on Authorship+Codebert with 66 labels, the ASR can only reach 6.4%.", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "The search algorithm appears novel to me and I like the idea of searching embedding space to find adversarial attacks. \n\nTheir algorithm achieves attack success rates better than their competitor attacks and appears to take less time.", "reasons_to_reject": "I'm not sure how relevant this is to the larger NLP community. Most of the cited work appears in IEEE or ACM conferences as they're about programming languages, not human language. Although this work could be adapted to human language, it would not be trivial. Additionally, a lot of citations are of arxiv papers; without looking them all up I can't tell if they've been reviewed or not.\n\nWhile the attack success rates go up, I'm not sure query time is the natural metric for attacking black-box systems by API call. Previous work in black-box attacks on classifiers for sentiment analysis and entailment has used the total number of queries required to flip the label and not time. Services available by API usually charge based on number of calls and not compute time.\n\nSome notation is unclear but I think that could be fixed for a camera ready version.\n", "questions_to_authors": "Question a: In the subsection 3.2.2 \"Computing Uncertainty\", is a new $X_mutated$ being created for each $var \\in x$? In the equation above line 307 and the definition of $P_i$ on line 307, I'm not seeing the reference to a specific $var$ for which the variance is being calculated.\n\nQuestion b: In section 3.2.4 \"Searching Top-K Substitutes\" you say that you filter out all variable names that don't have a similarity of less than $\\epsilon$. I'm not sure it's stated elsewhere but what is this similarity function and what is $\\epsilon$ set to? Do you know how many substitutions it filters out? I've seen previous work on adversarial attacks where their constraints were too tight that no actual search was being performed as there was only one possible token to select.\n\nQuestion c: Regarding the future work on targeted attacks. That's the case where you want to get the model to predict a certain label, right? Couldn't you change line 17 of algorithm 1 to increase the probability of the target label instead of just decreasing the probability of the current label?", "paper_topic_and_main_contributions": "This paper presents an adversarial attack on programming-language models such as codeT5 or codebert. The attack replaces variable names in the input code in order to cause the output label of the attacked model to flip. Variables are selected to replace based on the average variance that their replacement causes in the output probability distribution. Once the variable that causes the most variance is found, candidate variable names are found by searching in a variable name embedding space, learned by a masked language model. The search algorithm is based on nearest neighbors search around a seed embedding, which is the embedding of the current name plus a decaying delta vector calculated from the differences between past embeddings. This appears to be like momentum for iterative search in embedding space.", "excitement": 3, "reproducibility": 3, "ethical_concerns": "No", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {}, "extra_text": {}}]

Author rebuttal:

Meta-review: The reviewers agree that the paper provides sufficient support for its major claims/arguments, but were ambivalent about the merits and readiness. The main strengths of the paper are clear motivation, model setup relevant to the application, experimental performance, choice of comparators. The excitement is diminished mainly because of lack of clarity in methodology that left the readers wonder what were the exact contributions of the paper.
Decision: Accept-Findings