Original: G ENERATING A DVERSARIAL C OMPUTER P ROGRAMS USING O PTIMIZED O BFUSCATIONS
Revised: G ENERATING A DVERSARIAL C OMPUTER P ROGRAMS USING O PTIMIZED O BFUSCATIONS

Original: Anonymous authors

Original: Paper under double-blind review

Original: A BSTRACT
Revised: A BSTRACT

Original: Machine learning (ML) models that learn and predict properties of computer programs are increasingly being adopted and deployed.
Revised: Machine learning (ML) models that learn and predict properties of computer programs are increasingly being adopted and deployed.

Original: In this work, we investigate principled ways to adversarially perturb a computer program to fool such learned models, and thus determine their adversarial robustness.
Revised: In this work, we investigate principled ways to adversarially perturb a computer program to fool such learned models, and thus determine their adversarial robustness.

Original: We use program obfuscations, which have conventionally been used to avoid attempts at reverse engineering programs, as adversarial perturbations.
Revised: We use program obfuscations, which have conventionally been used to avoid attempts at reverse engineering programs, as adversarial perturbations.

Original: These perturbations modify programs in ways that do not alter their functionality but can be crafted to deceive an ML model when making a decision.
Revised: These perturbations modify programs in ways that do not alter their functionality but can be crafted to deceive an ML model when making a decision.

Original: We provide a general formulation for an adversarial program that allows applying multiple obfuscation transformations to a program in any language.
Revised: We provide a general formulation for an adversarial program that allows applying multiple obfuscation transformations to a program in any language.

Original: We develop ﬁrst-order optimization algorithms to efﬁciently determine two key aspects – which parts of the program to transform, and what transformations to use.
Revised: We develop ﬁrst-order optimization algorithms to efﬁciently determine two key aspects – which parts of the program to transform, and what transformations to use.

Original: We show that it is important to optimize both these aspects to generate the best adversarially perturbed program.
Revised: We show that it is important to optimize both these aspects to generate the best adversarially perturbed program.

Original: Due to the discrete nature of this problem, we also propose using randomized smoothing to improve the attack loss landscape to ease optimization.
Revised: Due to the discrete nature of this problem, we also propose using randomized smoothing to improve the attack loss landscape to ease optimization.

Original: We evaluate our work on
Revised: We evaluate our work on

Original: Python and Java programs on the problem of program summarization.
Revised: Python and Java programs on the problem of program summarization.

Original: 1 We show that our best attack proposal achieves a 52% improvement over a state-of-the-art attack generation approach for programs trained on a SEQ 2 SEQ model.
Revised: 1 We show that our best attack proposal achieves a 52% improvement over a state-of-the-art attack generation approach for programs trained on a SEQ 2 SEQ model.

Original: We further show that our formulation is better at training models that are robust to adversarial attacks.
Revised: We further show that our formulation is better at training models that are robust to adversarial attacks.

Original:  I NTRODUCTION
Revised:  I NTRODUCTION

Original: Machine learning (ML) models are increasingly being used for software engineering tasks.
Revised: Machine learning (ML) models are increasingly being used for software engineering tasks.

Original: Applications such as refactoring programs, auto-completing them in editors, and synthesizing GUI code have beneﬁted from ML models trained on large repositories of programs, sourced from popular websites like GitHub (Allamanis et al., 2018).
Revised: Applications such as refactoring programs, auto-completing them in editors, and synthesizing GUI code have beneﬁted from ML models trained on large repositories of programs, sourced from popular websites like GitHub (Allamanis et al., 2018).

Original: They have also been adopted to reason about programs, helping in ﬁnding and ﬁxing bugs (Gupta et al., 2017), detecting malware and vulnerabilities in them (Zhou et al., 2019) etc .
Revised: They have also been adopted to reason about programs, helping in ﬁnding and ﬁxing bugs (Gupta et al., 2017), detecting malware and vulnerabilities in them (Zhou et al., 2019) etc .

Original: thus complementing traditional program analysis tools.
Revised: thus complementing traditional program analysis tools.

Original: As these models continue to be adopted for such applications, it is important to understand how robust they are to adversarial attacks .
Revised: As these models continue to be adopted for such applications, it is important to understand how robust they are to adversarial attacks .

Original: Such attacks can have adverse consequences, particularly in settings such as security (Zhou et al., 2019) and compliance automation (Pedersen, 2010).
Revised: Such attacks can have adverse consequences, particularly in settings such as security (Zhou et al., 2019) and compliance automation (Pedersen, 2010).

Original: For example, an attacker could craft changes in malicious programs in a way which forces a model to incorrectly classify them as being benign, or make changes to pass off code which is licensed as open-source in an organization’s proprietary code-base.
Revised: For example, an attacker could craft changes in malicious programs in a way which forces a model to incorrectly classify them as being benign, or make changes to pass off code which is licensed as open-source in an organization’s proprietary code-base.

Original: Adversarially perturbing a program should achieve two goals – a trained model should ﬂip its decision when provided with the perturbed version of the program, and second, the perturbation should be imperceivable .
Revised: Adversarially perturbing a program should achieve two goals – a trained model should ﬂip its decision when provided with the perturbed version of the program, and second, the perturbation should be imperceivable .

Original: Adversarial attacks have mainly been considered in image classiﬁcation (Goodfellow et al., 2014; Carlini & Wagner, 2017; Madry et al., 2018), where calculated minor changes made to pixels of an image are enough to satisfy the imperceptibility requirement.
Revised: Adversarial attacks have mainly been considered in image classiﬁcation (Goodfellow et al., 2014; Carlini & Wagner, 2017; Madry et al., 2018), where calculated minor changes made to pixels of an image are enough to satisfy the imperceptibility requirement.

Original: Such changes escape a human’s attention by making the image look the same as before perturbing it, while modifying the underlying representation enough to ﬂip a classiﬁer’s decision.
Revised: Such changes escape a human’s attention by making the image look the same as before perturbing it, while modifying the underlying representation enough to ﬂip a classiﬁer’s decision.

Original: However, programs demand a stricter imperceptibility requirement – not only should the changes avoid human attention, but the changed program should also importantly functionally behave the same as the unperturbed program.
Revised: However, programs demand a stricter imperceptibility requirement – not only should the changes avoid human attention, but the changed program should also importantly functionally behave the same as the unperturbed program.

Original: Program obfuscations provide the agency to implement one such set of imperceivable changes in programs.
Revised: Program obfuscations provide the agency to implement one such set of imperceivable changes in programs.

Original: Obfuscating computer programs have long been used as a way to avoid attempts at reverse-engineering them.
Revised: Obfuscating computer programs have long been used as a way to avoid attempts at reverse-engineering them.

Original: They transform a program in a way that only hampers humans’ comprehension of parts of the program, while retaining its original semantics and functionality.
Revised: They transform a program in a way that only hampers humans’ comprehension of parts of the program, while retaining its original semantics and functionality.

Original: For example, one common obfuscation operation is to rename variables in an attempt to hide the program’s intent from a reader.
Revised: For example, one common obfuscation operation is to rename variables in an attempt to hide the program’s intent from a reader.

Original: Renaming a variable sum in the program statement int sum = 0 to int xyz = 

Revised: Renaming a variable sum in the program statement int sum = 0 to int xyz =

Original: neither alters how a compiler analyzes this variable nor changes any computations or states in the program; it only hampers our understanding of this variable’s role in the program.
Revised: neither alters how a compiler analyzes this variable nor changes any computations or states in the program; it only hampers our understanding of this variable’s role in the program.

Original: Modifying a very small number of such aspects of a program marginally affects how we comprehend it, thus providing a way to produce changes imperceivable to both humans and a compiler.
Revised: Modifying a very small number of such aspects of a program marginally affects how we comprehend it, thus providing a way to produce changes imperceivable to both humans and a compiler.

Original: In this work, we view adversarial perturbations to programs as a special case of applying obfuscation transformations to them.
Revised: In this work, we view adversarial perturbations to programs as a special case of applying obfuscation transformations to them.

Original: Having identiﬁed a set of candidate transformations which produce imperceivable changes, a speciﬁc subset needs to be chosen in a way which would make the transformed program adversarial.
Revised: Having identiﬁed a set of candidate transformations which produce imperceivable changes, a speciﬁc subset needs to be chosen in a way which would make the transformed program adversarial.

Original: Recent attempts (Yefet et al., 2019; Ramakrishnan et al., 2020; Bielik & Vechev, 2020) which came closest to addressing this problem did not offer any rigorous formulation.
Revised: Recent attempts (Yefet et al., 2019; Ramakrishnan et al., 2020; Bielik & Vechev, 2020) which came closest to addressing this problem did not offer any rigorous formulation.

Original: They recommended using a variety of transformations without presenting any principled approach to selecting an optimal subset of transformations.
Revised: They recommended using a variety of transformations without presenting any principled approach to selecting an optimal subset of transformations.

Original: We present a formulation which when solved provides the exact location to transform as well as a transformation to apply at the location.
Revised: We present a formulation which when solved provides the exact location to transform as well as a transformation to apply at the location.

Original: Figure 1 illustrates this.
Revised: Figure 1 illustrates this.

Original: A randomly selected local-variable ( name ) when replaced by the name virtualname , which is generated by the stateof-the-art attack generation algorithm for programs (Ramakrishnan et al., 2020), is unable to fool a program summarizer (which predicts set item ) unless our proposed site optimization is applied.
Revised: A randomly selected local-variable ( name ) when replaced by the name virtualname , which is generated by the stateof-the-art attack generation algorithm for programs (Ramakrishnan et al., 2020), is unable to fool a program summarizer (which predicts set item ) unless our proposed site optimization is applied.

Original: We provide a detailed comparison in Section 2.
Revised: We provide a detailed comparison in Section 2.

Original: In our work, we make the following key contributions –
Revised: In our work, we make the following key contributions –

Original: • We identify two problems central to deﬁning an adversarial program – identifying the sites in a program to apply perturbations on, and the speciﬁc perturbations to apply on the selected sites.
Revised: • We identify two problems central to deﬁning an adversarial program – identifying the sites in a program to apply perturbations on, and the speciﬁc perturbations to apply on the selected sites.

Original: These perturbations are involve replacing existing tokens or inserting new ones.
Revised: These perturbations are involve replacing existing tokens or inserting new ones.

Original: • We provide a general mathematical formulation of a perturbed program that models site locations and the perturbation choice for each location.
Revised: • We provide a general mathematical formulation of a perturbed program that models site locations and the perturbation choice for each location.

Original: It is independent of programming languages and the task on which a model is trained, while seamlessly modeling the application of multiple transformations to the program.
Revised: It is independent of programming languages and the task on which a model is trained, while seamlessly modeling the application of multiple transformations to the program.

Original: • We propose a set of ﬁrst-order optimization algorithms to solve our proposed formulation efﬁciently, resulting in a differentiable generator for adversarial programs.
Revised: • We propose a set of ﬁrst-order optimization algorithms to solve our proposed formulation efﬁciently, resulting in a differentiable generator for adversarial programs.

Original: We further propose a randomized smoothing algorithm to achieve improved optimization performance.
Revised: We further propose a randomized smoothing algorithm to achieve improved optimization performance.

Original: • Our approach demonstrates a 1 .
Revised: • Our approach demonstrates a 1 .

Original: 5 x increase in the attack success rate over the state-of-the-art attack generation algorithm (Ramakrishnan et al., 2020) on large datasets of Python and Java programs.
Revised: 5 x increase in the attack success rate over the state-of-the-art attack generation algorithm (Ramakrishnan et al., 2020) on large datasets of Python and Java programs.

Original: • We further show that our formulation provides better robustness against adversarial attacks compared to the state-of-the-art when used in training an ML model.
Revised: • We further show that our formulation provides better robustness against adversarial attacks compared to the state-of-the-art when used in training an ML model.

Original:  R ELATED W ORK
Revised:  R ELATED W ORK

Original: Due to a large body of literature on adversarial attacks in general, we focus on related works in the domain of computer programs.
Revised: Due to a large body of literature on adversarial attacks in general, we focus on related works in the domain of computer programs.

Original: Both Wang & Christodorescu (2019) and Rabin et al.
Revised: Wang & Christodorescu (2019), Quiring et al. , Rabin et al.

Revised: (2020), and Pierazzi et al.

Original: (2020) identify obfuscation transformations as potential adversarial examples.
Revised: (2020) identify obfuscation transformations as potential adversarial examples.

Original: They do not, however, ﬁnd an optimal set of transformations to deceive a downstream model.
Revised: They do not, however, ﬁnd an optimal set of transformations to deceive a downstream model.

Original: Liu et al.
Revised: Liu et al.

Original: (2017) provide a stochastic optimization formulation to obfuscate programs optimally by maximizing its impact on an obscu- rity language model (OLM).
Revised: (2017) provide a stochastic optimization formulation to obfuscate programs optimally by maximizing its impact on an obscurity language model (OLM).

Original: However, they do not address the problem of adversarial robustness of ML models of programs, and their formulation is only to ﬁnd the right sequence of transformations which increases their OLM’s perplexity.
Revised: However, they do not address the problem of adversarial robustness of ML models of programs, and their formulation is only to ﬁnd the right sequence of transformations which increases their OLM’s perplexity.

Original: They use an MCMC-based search to ﬁnd the best sequence.
Revised: They use an MCMC-based search to ﬁnd the best sequence.

Original: Yefet et al.
Revised: Yefet et al.

Original: (2019) propose perturbing programs by replacing local variables, and inserting print statements with replaceable string arguments.
Revised: (2019) propose perturbing programs by replacing local variables, and inserting print statements with replaceable string arguments.

Original: They ﬁnd optimal replacements using a ﬁrst-order optimization method, similar to Balog et al.
Revised: They ﬁnd optimal replacements using a ﬁrst-order optimization method, similar to Balog et al.

Original: and HotFlip (Ebrahimi et al., 2017) - an algorithm used to adversarially perturb NLP statements.
Revised: and HotFlip (Ebrahimi et al., 2017).

Original: This is an improvement over Zhang et al.
Revised: This is an improvement over Zhang et al.

Original: (2020), who use the Metropolis-Hastings algorithm to ﬁnd an optimal replacement for variable names.
Revised: (2020), who use the Metropolis-Hastings algorithm to ﬁnd an optimal replacement for variable names.

Original: Bielik & Vechev (2020) propose a robust training strategy which trains a model to abstain from deciding when uncertain if an input program is adversarially perturbed.
Revised: Bielik & Vechev (2020) propose a robust training strategy which trains a model to abstain from deciding when uncertain if an input program is adversarially perturbed.

Original: The transformation space they consider is small, which they search through greedily.
Revised: The transformation space they consider is small, which they search through greedily.

Original: Moreover, their solution is designed to reason over a limited context of the program (predicting variable types), and is non-trivial to extend to applications such as program summarization (explored in this work) which requires reasoning overon an entire program.
Revised: Moreover, their solution is designed to reason over a limited context of the program (predicting variable types), and is non-trivial to extend to applications such as program summarization (explored in this work) which requires reasoning over an entire program.

Original: Ramakrishnan et al.
Revised: Ramakrishnan et al.

Original: (2020) extend the work by Yefet et al.
Revised: (2020) extend the work by Yefet et al.

Original: and is most relevant to what we propose in this work.
Revised: and is most relevant to what we propose in this work.

Original: They experiment with a larger set of transformations and propose a standard min-max formulation to adversarially train robust models.
Revised: They experiment with a larger set of transformations and propose a standard min-max formulation to adversarially train robust models.

Original: Their inner-maximizer, which generates adversarial programs, models multiple transformations applied to a program in contrast to Yefet et al.
Revised: Their inner-maximizer, which generates adversarial programs, models multiple transformations applied to a program in contrast to Yefet et al.

Original: However, they do not propose any principled way to solve the problem of choosing between multiple program transformations.
Revised: However, they do not propose any principled way to solve the problem of choosing between multiple program transformations.

Original: They randomly select transformation operations to apply, and then randomly select locations in the program to apply those transformations on.
Revised: They randomly select transformation operations to apply, and then randomly select locations in the program to apply those transformations on.

Original: Different from the aforementioned works, we show  optimizing for locations alone improves the attack performance.
Revised: We instead show that optimizing for locations alone improves the attack performance.

Original: Further, we propose a joint optimization problem of ﬁnding the optimal location and optimal transformation, only the latter of which Ramakrishnan et al.
Revised: Further, we propose a joint optimization problem of ﬁnding the optimal location and optimal transformation, only the latter of which Ramakrishnan et al.

Original: (2020) (and Yefet et al.
Revised: (2020) (and Yefet et al.

Original: (2019)) address in a principled manner.
Revised: address in a principled manner.

Original: Although formally unpublished at the time of preparing this work, we compare our experiments to Ramakrishnan et al.
Revised: Although formally unpublished at the time of preparing this work, we compare our experiments to Ramakrishnan et al.

Original: (2020), the state-of-the-art in evaluating and defending against adversarial attacks on models for programs, and contrast the advantages of our formulation.
Revised: (2020), the state-of-the-art in evaluating and defending against adversarial attacks on models for programs, and contrast the advantages of our formulation.

Original:  P ROGRAM O BFUSCATIONS AS A DVERSARIAL P ERTURBATIONS
Revised:  P ROGRAM O BFUSCATIONS AS A DVERSARIAL P ERTURBATIONS

Original: In this section, we formalize program obfuscation operations, and show how generating adversarial programs can be cast as a constrained combinatorial optimization problem.
Revised: In this section, we formalize program obfuscation operations, and show how generating adversarial programs can be cast as a constrained combinatorial optimization problem.

Original: Program obfuscations.
Revised: Program obfuscations.

Original: We view obfuscation transformations made to programs as adversarial perturbations which can affect a downstream ML/DL model like a malware classiﬁer or a program summarizer.
Revised: We view obfuscation transformations made to programs as adversarial perturbations which can affect a downstream ML/DL model like a malware classiﬁer or a program summarizer.

Original: While a variety of such obfuscation transformations exist for programs in general (see section 2A, Liu et al.
Revised: While a variety of such obfuscation transformations exist for programs in general (see section 2A, Liu et al.

Original: , we consider two broad classes – replace and insert transformations.
Revised: , we consider two broad classes – replace and insert transformations.

Original: In replace transformations, existing program constructs are replaced with variants which decrease readability.
Revised: In replace transformations, existing program constructs are replaced with variants which decrease readability.

Original: For example, replacing a variable’s name, a function parameter’s name, or an object ﬁeld’s name does not affect the semantics of the program in any way.
Revised: For example, replacing a variable’s name, a function parameter’s name, or an object ﬁeld’s name does not affect the semantics of the program in any way.

Original: These names in any program exclusively aid human comprehension, and thus serve as three replace transformations.
Revised: These names in any program exclusively aid human comprehension, and thus serve as three replace transformations.

Original: In insert transformations, we insert new statements to the program which are unrelated to the code it is inserted around, thereby obfuscating its original intent.
Revised: In insert transformations, we insert new statements to the program which are unrelated to the code it is inserted around, thereby obfuscating its original intent.

Original: For example, including a print statement with an arbitrary string argument does not change the semantics of the program in any way.
Revised: For example, including a print statement with an arbitrary string argument does not change the semantics of the program in any way.

Original: Our goal hence is to introduce a systematic way to transform a program with insert or replace transformations such that a trained model misclassiﬁes a program P that it originally classiﬁed correctly.
Revised: Our goal hence is to introduce a systematic way to transform a program with insert or replace transformations such that a trained model misclassiﬁes a program P that it originally classiﬁed correctly.

Original: Site-selection and Site-perturbation – Towards deﬁning adversarial programs.
Revised: Site-selection and Site-perturbation – Towards deﬁning adversarial programs.

Original: Before we formally deﬁne an adversarial program, we highlight the key factors which need to be considered in our formulation through the example program introduced in Figure 2.
Revised: Before we formally deﬁne an adversarial program, we highlight the key factors which need to be considered in our formulation through the example program introduced in Figure 2.

Original: Consider applying the following two obfuscation transformations on the example program P in
Revised: Consider applying the following two obfuscation transformations on the example program P in

Original: Figure 2.a – replacing local variable names (a replace transform), and inserting print statements (an insert transform).
Revised: Figure 2.a – replacing local variable names (a replace transform), and inserting print statements (an insert transform).

Original: The two local variables b and r in P are potential candidates where the replace transform can be applied, while a print statement can potentially be inserted at the three locations I1 , I2 , I3 (highlighted in Figure 2.b).
Revised: The two local variables b and r in P are potential candidates where the replace transform can be applied, while a print statement can potentially be inserted at the three locations I1 , I2 , I3 (highlighted in Figure 2.b).

Original: We notate these choices in a program as sites – locations in a program where a unique transformation can be applied.
Revised: We notate these choices in a program as sites – locations in a program where a unique transformation can be applied.

Original: Thus, in order to adversarially perturb P , we identify two important questions that need to be addressed.
Revised: Thus, in order to adversarially perturb P , we identify two important questions that need to be addressed.

Original: First, which sites in a program should be transformed?
Revised: First, which sites in a program should be transformed?

Original: Of the n sites in a program, if we are allowed to choose at most k sites, which set of ≤ k sites would have the highest impact on the downstream model’s performance?
Revised: Of the n sites in a program, if we are allowed to choose at most k sites, which set of ≤ k sites would have the highest impact on the downstream model’s performance?

Original: We identify this as the site-selection problem , where the constraint k is the perturbation strength of an attacker.
Revised: We identify this as the site-selection problem , where the constraint k is the perturbation strength of an attacker.

Original: Second, what tokens should be inserted/replaced at the k selected sites?
Revised: Second, what tokens should be inserted/replaced at the k selected sites?

Original: Once we pick k sites, we still have to determine the best choice of tokens to replace/insert at those sites which would have the highest impact on the downstream model.
Revised: Once we pick k sites, we still have to determine the best choice of tokens to replace/insert at those sites which would have the highest impact on the downstream model.

Original: We refer to this as the site-perturbation problem .
Revised: We refer to this as the site-perturbation problem .

Original: Mathematical formulation.
Revised: Mathematical formulation.

Original: In what follows, we propose a general and rigorous formulation of adversarial programs.
Revised: In what follows, we propose a general and rigorous formulation of adversarial programs.

Original: Let P denote a benign program which consists of a series of n tokens {P i } ni =1 in the source code domain.
Revised: Let P denote a benign program which consists of a series of n tokens {P i } ni =1 in the source code domain.

Original: For example, the program in Figure 2.a, when read from top to bottom and left to right, forms a series of n = 12 tokens { def , b , ..., r , + , 5 } .
Revised: For example, the program in Figure 2.a, when read from top to bottom and left to right, forms a series of n = 12 tokens { def , b , ..., r , + , 5 } .

Original: We ignore white spaces and other delimiters when tokenizing.
Revised: We ignore white spaces and other delimiters when tokenizing.

Original: Each P i ∈ { 0 , 1 } | Ω | here is considered a one-hot vector of length | Ω | , where Ω is a vocabulary of tokens.
Revised: Each P i ∈ { 0 , 1 } | Ω | here is considered a one-hot vector of length | Ω | , where Ω is a vocabulary of tokens.

Original: Let P (cid:48) deﬁne a perturbed program (with respect to
Revised: Let P (cid:48) deﬁne a perturbed program (with respect to

Original: P ) created by solving the site-selection and site-perturbation problems, which use the vocabulary Ω to ﬁnd an optimal replacement.
Revised: P ) created by solving the site-selection and site-perturbation problems, which use the vocabulary Ω to ﬁnd an optimal replacement.

Original: Since our formulation is agnostic to the type of transformation, perturbation in the remainder of this section refers to both replace and insert transforms.
Revised: Since our formulation is agnostic to the type of transformation, perturbation in the remainder of this section refers to both replace and insert transforms.

Original: In our work, we use a shared vocabulary Ω to select transforms from both these classes.
Revised: In our work, we use a shared vocabulary Ω to select transforms from both these classes.

Original: In practice, we can also assign a unique vocabulary to each transformation we deﬁne.
Revised: In practice, we can also assign a unique vocabulary to each transformation we deﬁne.

Original: To formalize the site-selection problem, we introduce a vector of boolean variables z ∈ { 0 , 1 } n to indicate whether or not a site is selected for perturbation.
Revised: To formalize the site-selection problem, we introduce a vector of boolean variables z ∈ { 0 , 1 } n to indicate whether or not a site is selected for perturbation.

Original: If z i = 

Revised: If z i =

Original: then the i th site (namely, P i ) is perturbed.
Revised: then the i th site (namely, P i ) is perturbed.

Original: If there exist multiple occurrences of a token in the program, then all such sites are marked 1 .
Revised: If there exist multiple occurrences of a token in the program, then all such sites are marked 1 .

Original: For example, in Figure 2.d, if the site corresponding to local variable b is selected, then both indices of its occurrences, z 3 , z 9 are marked as 1 as shown in z i .
Revised: For example, in Figure 2.d, if the site corresponding to local variable b is selected, then both indices of its occurrences, z 3 , z 9 are marked as 1 as shown in z i .

Original: Moreover, the number of perturbed sites, namely, 1 T z ≤ k provides a means of measuring perturbation strength .
Revised: Moreover, the number of perturbed sites, namely, 1 T z ≤ k provides a means of measuring perturbation strength .

Original: For example, k =is the minimum perturbation possible, where only one site is allowed to be perturbed.
Revised: For example, k =is the minimum perturbation possible, where only one site is allowed to be perturbed.

Original: To deﬁne site-perturbation , we introduce a one-hot vector u i ∈ { 0 , 1 } | Ω | to encode the selection of a token from Ω which would serve as the insert/replace token for a chosen transformation at a chosen site.
Revised: To deﬁne site-perturbation , we introduce a one-hot vector u i ∈ { 0 , 1 } | Ω | to encode the selection of a token from Ω which would serve as the insert/replace token for a chosen transformation at a chosen site.

Original: If the j th entry [ u i ] j = 1 and z i = 1 , then the j th token in Ω is used as the obfuscation transformation applied at the site
Revised: If the j th entry [ u i ] j = 1 and z i = 1 , then the j th token in Ω is used as the obfuscation transformation applied at the site

Original: i (namely, to perturb P i ).
Revised: i (namely, to perturb P i ).

Original: We also have the constraint 1 T u i = 1 , implying that only one perturbation is performed at P i .
Revised: We also have the constraint 1 T u i = 1 , implying that only one perturbation is performed at P i .

Original: Let vector u ∈ { 0 , 1 } n ×| Ω | denote n different u i vectors, one for each token i in P .
Revised: Let vector u ∈ { 0 , 1 } n ×| Ω | denote n different u i vectors, one for each token i in P .

Original: Using the above formulations for site-selection , site-perturbation and perturbation strength , the perturbed program P (cid:48) can then be deﬁned as
Revised: Using the above formulations for site-selection , site-perturbation and perturbation strength , the perturbed program P (cid:48) can then be deﬁned as

Original: P (cid:48)
Revised: [Equation]

Original: = ( 1 − z ) · P + z · u , where 1 T z ≤ k, z ∈ { 0 , 1 } n , 1 T u i = 1 , u i ∈ { 0 , 1 } |Ω | , ∀ i, (1) where · denotes the element-column wise product.
Revised: = ( 1 − z ) · P + z · u , where 1 T z ≤ k, z ∈ { 0 , 1 } n , 1 T u i = 1 , u i [Equation] [Equation] 1 } |

Original: The adversarial effect of P (cid:48) is then measured by passing it as input to a downstream ML/DL model θ and seeing if it successfully manages to fool it.
Revised: [Equation] | [Equation] i

Revised: [Equation]

Revised: 1) where · denotes the element-column wise product.

Revised: The adversarial effect of P (cid:48) is then measured by passing it as input to a downstream ML/DL model θ and seeing if it successfully manages to fool it.

Original: Generating a successful adversarial program is then formulated as the optimization problem, 
Revised: Generating a successful adversarial program is then formulated as the optimization problem, minimize z

Original: [Equation]
Revised: [Equation]

Original: subject
Revised: subject

Revised: [Equation]

Revised: constraints

Original: [Equation]
Revised: [Equation]

Original: where (cid:96) attack denotes an attack loss.
Revised: where (cid:96) attack denotes an attack loss.

Original: In this work, we specify (cid:96) attack as the cross-entropy loss on the predicted output evaluated at P (cid:48) in an untargeted setting (namely, without specifying the prediction label targeted by an adversary) (Ramakrishnan et al., 2020).
Revised: In this work, we specify (cid:96) attack as the cross-entropy loss on the predicted output evaluated at P (cid:48) in an untargeted setting (namely, without specifying the prediction label targeted by an adversary) (Ramakrishnan et al., 2020).

Original: One can also consider other speciﬁcations of (cid:96) attack , e.g., C&W untargeted and targeted attack losses (Carlini & Wagner, 2017).
Revised: One can also consider other speciﬁcations of (cid:96) attack , e.g., C&W untargeted and targeted attack losses (Carlini & Wagner, 2017).

Original:  A DVERSARIAL P ROGRAM G ENERATION VIA F IRST -O RDER O PTIMIZATION
Revised:  A DVERSARIAL P ROGRAM G ENERATION VIA F IRST -O RDER O PTIMIZATION

Original: Solving problem (2) is not trivial because of its combinatorial nature (namely, the presence of boolean variables), the presence of a bi-linear objective term (namely, z · u ), as well as the presence of multiple constraints.
Revised: Solving problem (2) is not trivial because of its combinatorial nature (namely, the presence of boolean variables), the presence of a bi-linear objective term (namely, z · u ), as well as the presence of multiple constraints.

Original: To address this, we present a projected gradient descent (PGD) based joint optimization solver ( JO ) and propose alternates which promise better empirical performance.
Revised: To address this, we present a projected gradient descent (PGD) based joint optimization solver ( JO ) and propose alternates which promise better empirical performance.

Original: PGD as a joint optimization ( JO ) solver.
Revised: PGD as a joint optimization ( JO ) solver.

Original: PGD has been shown to be one of the most effective attack generation methods to fool image classiﬁcation models (Madry et al., 2018).
Revised: PGD has been shown to be one of the most effective attack generation methods to fool image classiﬁcation models (Madry et al., 2018).

Original: Prior to applying
Revised: Prior to applying

Original: PGD, we instantiate (2) into a feasible version by relaxing boolean constraints to their convex hulls, minimize z , [Equation] where for ease of notation, the attack loss in (2) is denoted by (cid:96) attack ( z , u ) .
Revised: PGD, we instantiate (2) into a feasible version by relaxing boolean constraints to their convex hulls, minimize z , u (cid:96) attack ( z , u ) subject to 1 T z ≤ k, z ∈ [0 , 1] n , 1 T u i = 1 , u i ∈ [0 , 1] | Ω | , ∀ i, (3) where for ease of notation, the attack loss in (2) is denoted by (cid:96) attack ( z , u ) .

Original: The continuous relaxation of binary variables in (3) is a commonly used trick in combinatorial optimization to boost the stability of learning procedures in practice (Boyd et al., 2004).
Revised: The continuous relaxation of binary variables in (3) is a commonly used trick in combinatorial optimization to boost the stability of learning procedures in practice (Boyd et al., 2004).

Original: Once the continuous optimization problem (3) is solved, a hard thresholding operation or a randomized sampling method (which regards z and u as probability vectors with elements drawn from a Bernoulli distribution) can be called to map a continuous solution to its discrete domain (Blum & Roli, 2003).
Revised: Once the continuous optimization problem (3) is solved, a hard thresholding operation or a randomized sampling method (which regards z and u as probability vectors with elements drawn from a Bernoulli distribution) can be called to map a continuous solution to its discrete domain (Blum & Roli, 2003).

Original: We use the randomized sampling method in our experiments.
Revised: We use the randomized sampling method in our experiments.

Original: The PGD algorithm is then given by
Revised: The PGD algorithm is then given by

Original: [Equation]
Revised: [Equation]

Original: where t denotes PGD iterations, z (0) and u (0) are given initial points, α > 0 is a learning rate, ∇ z denotes the ﬁrst-order derivative operation w.r.t.
Revised: where t denotes PGD iterations, z (0) and u (0) are given initial points, α > 0 is a learning rate, ∇ z denotes the ﬁrst-order derivative operation w.r.t.

Original: the variable z , and Proj represents the projection operation w.r.t.
Revised: the variable z , and Proj represents the projection operation w.r.t.

Original: the constraints of (3).
Revised: the constraints of (3).

Original: The projection step involves solving for z and u i simultaneously in a complex convex problem.
Revised: The projection step involves solving for z and u i simultaneously in a complex convex problem.

Original: See Equation 9 in Appendix A for details.
Revised: See Equation 9 in Appendix A for details.

Original: A key insight is that the complex projection problem (9) can equivalently be decomposed into a sequence of sub-problems owing to the separability of the constraints w.r.t.
Revised: A key insight is that the complex projection problem (9) can equivalently be decomposed into a sequence of sub-problems owing to the separability of the constraints w.r.t.

Original: z and { u i } .
Revised: z and { u i } .

Original: The two sub-problems are – 
Revised: The two sub-problems are – minimize z

Original: [Equation]
Revised: [Equation]

Revised: subject to 1 T z ≤ k, z ∈ [0 , 1] n , and minimize u i (cid:107)

Revised: [Equation]

Revised: subject to

Revised: [Equation]

Revised: u

Revised: [Equation]

Revised: [Equation]

Original: The above subproblems w.r.t.
Revised: The above subproblems w.r.t.

Original: z and u i can optimally be solved by using a bisection method that ﬁnds the root of a scalar equation.
Revised: z and u i can optimally be solved by using a bisection method that ﬁnds the root of a scalar equation.

Original: We provide details of a closed-form solution and its corresponding proof in Appendix A.
Revised: We provide details of a closed-form solution and its corresponding proof in Appendix A.

Original: We use this decomposition and solutions to design an alternating optimizer, which we discuss next.
Revised: We use this decomposition and solutions to design an alternating optimizer, which we discuss next.

Original: Alternating optimization ( AO ) for fast attack generation.
Revised: Alternating optimization ( AO ) for fast attack generation.

Original: While JO provides an approach to solve the uniﬁed formulation in (2), it suffers from the problem of getting trapped at a poor local optima despite attaining stationarity (Ghadimi et al., 2016).
Revised: While JO provides an approach to solve the uniﬁed formulation in (2), it suffers from the problem of getting trapped at a poor local optima despite attaining stationarity (Ghadimi et al., 2016).

Original: We propose using AO (Bezdek & Hathaway, 2003) which allows the loss landscape to be explored more aggressively, thus leading to better empirical convergence and optimality (see Figure 4a).
Revised: We propose using AO (Bezdek & Hathaway, 2003) which allows the loss landscape to be explored more aggressively, thus leading to better empirical convergence and optimality (see Figure 4a).

Original: AO solves problem (2) one variable at a time – ﬁrst, by optimizing the site selection variable z keeping the site perturbation variable u ﬁxed, and then optimizing u keeping z ﬁxed.
Revised: AO solves problem (2) one variable at a time – ﬁrst, by optimizing the site selection variable z keeping the site perturbation variable u ﬁxed, and then optimizing u keeping z ﬁxed.

Original: That is, 
Revised: That is, z ( t ) =

Original: [Equation]
Revised: arg min 1 T z ≤ k, z ∈ [0 , 1] n (cid:96) attack ( z , u ( t − 1) ) and u ( t ) i = arg min 1 T u i =1 , u i ∈ [0 , 1] | Ω | (cid:96) attack ( z ( t ) , u ) ∀ i.

Original: We can use PGD, as described in (6), to similarly solve each of z and u separately in the two alternating steps.
Revised: We can use PGD, as described in (6), to similarly solve each of z and u separately in the two alternating steps.

Original: Computationally, AO is expensive than JO by a factor of 2 , since we need two iterations to cover all the variables which JO covers in a single iteration.
Revised: Computationally, AO is expensive than JO by a factor of 2 , since we need two iterations to cover all the variables which JO covers in a single iteration.

Original: However, in our experiments, we ﬁnd AO to converge faster.
Revised: However, in our experiments, we ﬁnd AO to converge faster.

Original: The decoupling in AO also eases implementation, and provides the ﬂexibility to set a different number of iterations for the u -step and the z -step within one iteration of
Revised: The decoupling in AO also eases implementation, and provides the ﬂexibility to set a different number of iterations for the u -step and the z -step within one iteration of

Original: AO .
Revised: AO .

Original: We also remark that the AO setup in (7) can be speciﬁed in other forms, e.g. alternating direction method of multipliers (ADMM) (Boyd et al., 2011).
Revised: We also remark that the AO setup in (7) can be speciﬁed in other forms, e.g. alternating direction method of multipliers (ADMM) (Boyd et al., 2011).

Original: However, such methods use an involved alternating scheme to solve problem (2).
Revised: However, such methods use an involved alternating scheme to solve problem (2).

Original: We defer evaluating these options to future work.
Revised: We defer evaluating these options to future work.

Original: Randomized smoothing ( RS ) to improve generating adversarial programs.
Revised: Randomized smoothing ( RS ) to improve generating adversarial programs.

Original: In our experi- ments, we noticed that the loss landscape of generating adversarial program is not smooth (Figure 3).
Revised: In our experi- ments, we noticed that the loss landscape of generating adversarial program is not smooth (Figure 3).

Original: This motivated us to explore surrogate loss functions which could smoothen it out.
Revised: This motivated us to explore surrogate loss functions which could smoothen it out.

Original: In our work, we employ a convolution-based RS technique (Duchi et al., 2012) to circumvent the optimization difﬁculty induced by the non-smoothness of the attack loss (cid:96) attack .
Revised: In our work, we employ a convolution-based RS technique (Duchi et al., 2012) to circumvent the optimization difﬁculty induced by the non-smoothness of the attack loss (cid:96) attack .

Original: We eventually obtain a smoothing [Figure] (cid:96) 
Revised: We eventually obtain a smoothing loss (cid:96) smooth :

Original: [Equation]
Revised: [Equation]

Original: (8) where ξ and τ are random samples drawn from the uniform distribution within the unit Euclidean ball, and µ > 0 is a small smoothing parameter (set to 0 .
Revised: (8) where ξ and τ are random samples drawn from the uniform distribution within the unit Euclidean ball, and µ > 0 is a small smoothing parameter (set to 0 .

Original: 01 in our experiments).
Revised: 01 in our experiments).

Original: [Figure]
Revised: [Figure]

Revised: [Figure]

Revised: [Figure]

Original: [Figure]
Revised: [Figure]

Original: The rationale behind RS (8) is that the convolution of two functions (smooth probability density function and non-smooth attack loss) is at least as smooth as the smoothest of the two original functions.
Revised: The rationale behind RS (8) is that the convolution of two functions (smooth probability density function and non-smooth attack loss) is at least as smooth as the smoothest of the two original functions.

Original: The advantage of such a formulation is that it is independent of the loss function, downstream model, and the optimization solver chosen for a problem.
Revised: The advantage of such a formulation is that it is independent of the loss function, downstream model, and the optimization solver chosen for a problem.

Original: We evaluate RS on both AO and JO .
Revised: We evaluate RS on both AO and JO .

Original: In practice, we consider an empirical
Revised: In practice, we consider an empirical

Original: Monte Carlo approximation of (8), (cid:96) [Equation] ( z , u ) =
Revised: Monte Carlo approximation of (8), (cid:96) smooth ( z , u ) =

Original: [Equation]{+(cid:80) mj =
+}
Revised: {+(cid:80) mj =
+}

Original: [Equation] µ ξ j , u + µ τ j )] .
Revised: [ (cid:96) attack ( z + µ ξ j , u + µ τ j )] .

Original: We set m = 10 in our experiments to save on computation time.
Revised: We set m = 10 in our experiments to save on computation time.

Original: We also ﬁnd that smoothing the site perturbation variable u contributes the most to improving attack performance.
Revised: We also ﬁnd that smoothing the site perturbation variable u contributes the most to improving attack performance.

Original: We hence perturb only u to further save computation time.
Revised: We hence perturb only u to further save computation time.

Original:  E XPERIMENTS & R ESULTS5. S ETUP
Revised:  E XPERIMENTS & R ESULTS

Original: We begin by discussing the following aspects of our experiment setup – the classiﬁcation task, the dataset and model we evaluate on, and the evaluation metrics we use.
Revised: We begin by discussing the following aspects of our experiment setup – the classiﬁcation task, the dataset and model we evaluate on, and the evaluation metrics we use.

Original: Task, Transformations, Dataset.
Revised: Task, Transformations, Dataset.

Original: We evaluate our formulation of generating optimal adversarial programs on the problem of program summarization, ﬁrst introduced by Allamanis et al.
Revised: We evaluate our formulation of generating optimal adversarial programs on the problem of program summarization, ﬁrst introduced by Allamanis et al.

Original: Summarizing a function in a program involves predicting its name, which is usually indicative of its intent.
Revised: Summarizing a function in a program involves predicting its name, which is usually indicative of its intent.

Original: We use this benchmark to test whether our adversarially perturbed program, which retains the functionality of the original program, can force a trained summarizer to predict an incorrect function name.
Revised: We use this benchmark to test whether our adversarially perturbed program, which retains the functionality of the original program, can force a trained summarizer to predict an incorrect function name.

Original: We evaluate this on a well maintained dataset of roughly 150K Python programs(Raychev et al., 2016) and 700K Java programs (Alon et al., 2018).
Revised: We evaluate this on a well maintained dataset of roughly 150K Python programs(Raychev et al., 2016) and 700K Java programs (Alon et al., 2018).

Original: They are pre-processed into functions, and each function is provided as input to an ML model.
Revised: They are pre-processed into functions, and each function is provided as input to an ML model.

Original: The name of the function is omitted from the input.
Revised: The name of the function is omitted from the input.

Original: The ML model predicts a sequence of tokens as the function name.
Revised: The ML model predicts a sequence of tokens as the function name.

Original: We evaluate our work on six transformations (4 replace and 2 insert transformations); see Appendix B for details on these transformations.
Revised: We evaluate our work on six transformations (4 replace and 2 insert transformations); see Appendix B for details on these transformations.

Original: The results and analysis that follow pertains to the case when any of these six transformations can be used as a valid perturbation, and the optimization selects which to pick and apply based on the perturbation strength k .
Revised: The results and analysis that follow pertains to the case when any of these six transformations can be used as a valid perturbation, and the optimization selects which to pick and apply based on the perturbation strength k .

Original: This is the same setting employed in the baseline (Ramakrishnan et al., 2020).
Revised: This is the same setting employed in the baseline (Ramakrishnan et al., 2020).

Original: Model.
Revised: Model.

Original: We evaluate a trained SEQ 2 SEQ model.
Revised: We evaluate a trained SEQ 2 SEQ model.

Original: It takes program tokens as input, and generates a sequence of tokens representing its function name.
Revised: It takes program tokens as input, and generates a sequence of tokens representing its function name.

Original: We note that our formulation is independent of the learning model, and can be evaluated on any model for any task.
Revised: We note that our formulation is independent of the learning model, and can be evaluated on any model for any task.

Original: The SEQ 2 SEQ model is trained and validated on 90% of the data while tested on the remaining 10% .
Revised: The SEQ 2 SEQ model is trained and validated on 90% of the data while tested on the remaining 10% .

Original: It is optimized using the cross-entropy loss function.
Revised: It is optimized using the cross-entropy loss function.

Revised: CODE 2 SEQ is a similar model which has been evaluated on the task of program summarization.

Revised: Its architecture is similar to that of SEQ 2 SEQ and contains two encoders - one which encodes tokens, while another which encodes AST paths.

Revised: The model when trained only on tokens performs similar to a model trained on both tokens and paths (Table 3, Alon et al.

Revised: Thus adversarial changes made to tokens, as accommodated by our formulation, should have a high impact on the model’s output.

Original:  Evaluation metrics.
Revised: Owing to the similarity in these architectures, and since our computational bench is in [Table] Pytorch while the original CODE 2 SEQ implementation is in TensorFlow, we defer evaluating the performance of our formulation on CODE 2 SEQ to future work. Evaluation metrics.

Original: We report two metrics – Attack Success Rate (ASR) and F1-score.
Revised: We report two metrics – Attack Success Rate (ASR) and F1-score.

Original: ASR is deﬁned as the percentage of output tokens misclassiﬁed by the model on the perturbed input but correctly predicted on the unperturbed input, i.e. ASR =
Revised: ASR is deﬁned as the percentage of output tokens misclassiﬁed by the model on the perturbed input but correctly predicted on the unperturbed input, i.e. ASR =

Original: (cid:80) i,j 1 ( θ ( x (cid:48)
Revised: (cid:80) i,j 1 ( θ ( x (cid:48)

Original: i ) (cid:54) =
Revised: i ) (cid:54) =

Original: y ij ) (cid:80) i,j 1 ( θ ( x i )=
Revised: y ij ) (cid:80) i,j 1 ( θ ( x i )=

Original: y ij ) for each token j in the expected output of sample i .
Revised: y ij ) for each token j in the expected output of sample i .

Original: Higher the ASR, better the attack.
Revised: Higher the ASR, better the attack.

Original: Unlike (Ramakrishnan et al., 2020), we evaluate our method on those samples in the test-set which were fully, correctly classiﬁed by the model.
Revised: Unlike (Ramakrishnan et al., 2020), we evaluate our method on those samples in the test-set which were fully, correctly classiﬁed by the model.

Original: Evaluating on such fully correctly classiﬁed samples provides direct evidence of the adversarial effect of the input perturbations (also the model’s adversarial robustness) by excluding test samples that have originally been misclassiﬁed even without any perturbation.
Revised: Evaluating on such fully correctly classiﬁed samples provides direct evidence of the adversarial effect of the input perturbations (also the model’s adversarial robustness) by excluding test samples that have originally been misclassiﬁed even without any perturbation.

Original: We successfully replicated results from (Ramakrishnan et al., 2020) on the F1-score metric they use, and acknowl- [Table] edge the extensive care they have taken to ensure that their results are reproducible.
Revised: We successfully replicated results from (Ramakrishnan et al., 2020) on the F1-score metric they use, and acknowledge the extensive care they have taken to ensure that their results are reproducible.

Original: As reported in
Revised: As reported in

Original: Table 2 of (Ramakrishnan et al., 2020), a trained SEQ 2 SEQ model has an F1-score of 34 .
Revised: Table 2 of (Ramakrishnan et al., 2020), a trained SEQ 2 SEQ model has an F1-score of 34 .

Original: 3 evaluated on the entire dataset.
Revised: 3 evaluated on the entire dataset.

Original: We consider just those samples which were correctly classiﬁed.
Revised: We consider just those samples which were correctly classiﬁed.

Original: The F1-score corresponding to ‘No attack’ in Table 1 is hence 100 .
Revised: The F1-score corresponding to ‘No attack’ in Table 1 is hence 100 .

Original: In all, we perturb 2800 programs in Python and 2300 programs in Java which are correctly classiﬁed.
Revised: In all, we perturb 2800 programs in Python and 2300 programs in Java which are correctly classiﬁed.

Original: 5. E XPERIMENTS
Revised: 5. E XPERIMENTS

Original: We evaluate our work in three ways – ﬁrst, we evaluate the overall performance of the three approaches we propose – AO , JO , and their combination with RS , to ﬁnd the best sites and perturbations for a given program.
Revised: We evaluate our work in three ways – ﬁrst, we evaluate the overall performance of the three approaches we propose – AO , JO , and their combination with RS , to ﬁnd the best sites and perturbations for a given program.

Original: Second, we evaluate the sensitivity of two parameters which control our optimizers – the number of iterations they are evaluated on, and the perturbation strength ( k ) of an attacker.
Revised: Second, we evaluate the sensitivity of two parameters which control our optimizers – the number of iterations they are evaluated on, and the perturbation strength ( k ) of an attacker.

Original: Third, we use our formulation to train an adversarially robust SEQ 2 SEQ model, and evaluate its performance against the attacks we propose.
Revised: Third, we use our formulation to train an adversarially robust SEQ 2 SEQ model, and evaluate its performance against the attacks we propose.

Original: Overall attack results.
Revised: Overall attack results.

Original: Table 1 summarizes our overall results.
Revised: Table 1 summarizes our overall results.

Original: The ﬁrst row corresponds to the samples not being perturbed at all.
Revised: The ﬁrst row corresponds to the samples not being perturbed at all.

Original: The ASR as expected is 0 .
Revised: The ASR as expected is 0 .

Original: The ‘Random replace’ in row 2 corresponds to both z and u being selected at random.
Revised: The ‘Random replace’ in row 2 corresponds to both z and u being selected at random.

Original: This produces no change in the ASR, suggesting that while obfuscation transformations can potentially deceive ML models, any random transformation will have little effect.
Revised: This produces no change in the ASR, suggesting that while obfuscation transformations can potentially deceive ML models, any random transformation will have little effect.

Original: It is important to have some principled approach to selecting and applying these transformations.
Revised: It is important to have some principled approach to selecting and applying these transformations.

Original: Ramakrishnan et al.
Revised: Ramakrishnan et al.

Original: (2020) (referred to as B ASELINE in Table 1) evaluated their work in two settings.
Revised: (2020) (referred to as B ASELINE in Table 1) evaluated their work in two settings.

Original: In the ﬁrst setting, they pick 1 site at random and optimally perturb it.
Revised: In the ﬁrst setting, they pick 1 site at random and optimally perturb it.

Original: They refer to this as Q 1 G .
Revised: They refer to this as Q 1 G .

Original: We contrast this by selecting an optimal site through our formulation.
Revised: We contrast this by selecting an optimal site through our formulation.

Original: We use the same algorithm as theirs to optimally perturb the chosen site i.e. to solve for u .
Revised: We use the same algorithm as theirs to optimally perturb the chosen site i.e. to solve for u .

Original: This allows us to ablate the effect of incorporating and solving the site-selection problem in our formulation.
Revised: This allows us to ablate the effect of incorporating and solving the site-selection problem in our formulation.

Original: In the second related setting, they pick 5 sites at random and optimally perturb them, which they refer to as Q 5 G .
Revised: In the second related setting, they pick 5 sites at random and optimally perturb them, which they refer to as Q 5 G .

Original: In our setup, Q 1 G and Q 5 G are equivalent to setting k = 1 and k
Revised: In our setup, Q 1 G and Q 5 G are equivalent to setting k = 1 and k

Original: = 5 respectively, and picking random sites in z instead of optimal ones.
Revised: = 5 respectively, and picking random sites in z instead of optimal ones.

Original: We run AO for 3 iterations, and JO for 10 iterations.
Revised: We run AO for 3 iterations, and JO for 10 iterations.

Original: We ﬁnd that our formulation consistently outperforms the baseline.
Revised: We ﬁnd that our formulation consistently outperforms the baseline.

Original: For k = 1 , where the attacker can perturb at most 1 site, both AO and JO provide a 3 point improvement in ASR, with
Revised: For k = 1 , where the attacker can perturb at most 1 site, both AO and JO provide a 3 point improvement in ASR, with

Original: JO marginally performing better than AO .
Revised: JO marginally performing better than AO .

Original: Increasing k improves the ASR across all methods – the attacker now has multiple sites to transform.
Revised: Increasing k improves the ASR across all methods – the attacker now has multiple sites to transform.

Original: For k = 5 , where the attacker has at most 5 sites to transform, we ﬁnd AO to provide a 6 point improvement in ASR over the baseline, outperforming
Revised: For k = 5 , where the attacker has at most 5 sites to transform, we ﬁnd AO to provide a 6 point improvement in ASR over the baseline, outperforming

Original: JO by 1 . 5 points.
Revised: JO by 1 . 5 points.

Original: Smoothing the loss function has a marked effect.
Revised: Smoothing the loss function has a marked effect.

Original: For k = 1 , we ﬁnd smoothing to provide apoint increase ( ∼ 52% improvement) in ASR over the baseline when applied to AO , while JO+RS provides a 4 point increase.
Revised: For k = 1 , we ﬁnd smoothing to provide apoint increase ( ∼ 52% improvement) in ASR over the baseline when applied to AO , while JO+RS provides a 4 point increase.

Original: Similarly, for k = 5 , we ﬁnd AO+RS to provide a 14 point increase ( ∼ 38% improvement), while JO+RS provides a 11 point increase, suggesting the utility of smoothing the landscape to aid optimization.
Revised: Similarly, for k = 5 , we ﬁnd AO+RS to provide a 14 point increase ( ∼ 38% improvement), while JO+RS provides a 11 point increase, suggesting the utility of smoothing the landscape to aid optimization.

Original: Overall, we ﬁnd that accounting for site location in our formulation combined with having a smooth loss function to optimize improves the quality of the generated attacks by nearly 1 .
Revised: Overall, we ﬁnd that accounting for site location in our formulation combined with having a smooth loss function to optimize improves the quality of the generated attacks by nearly 1 .

Original: 5 times over the state-of-the-art attack generation method for programs.
Revised: 5 times over the state-of-the-art attack generation method for programs.

Original: Effect of solver iterations and perturbation strength k .
Revised: Effect of solver iterations and perturbation strength k .

Original: We evaluate the attack performance (ASR) of our proposed approaches against the number of iterations at k = 5 (Figure 4a).
Revised: We evaluate the attack performance (ASR) of our proposed approaches against the number of iterations at k = 5 (Figure 4a).

Original: For comparison, we also present the performance of B ASELINE , which is not sensitive to the number of iterations (consistent with the empirical ﬁnding in (Ramakrishnan et al., 2020)), implying its least optimality.
Revised: For comparison, we also present the performance of B ASELINE , which is not sensitive to the number of iterations (consistent with the empirical ﬁnding in (Ramakrishnan et al., 2020)), implying its least optimality.

Original: Without smoothing, JO takes nearly 10 iterations to reach its local optimal value, whereas AO achieves it using only 3 iterations but with improved optimality (in terms of higher ASR than JO ).
Revised: Without smoothing, JO takes nearly 10 iterations to reach its local optimal value, whereas AO achieves it using only 3 iterations but with improved optimality (in terms of higher ASR than JO ).

Original: This supports our hypothesis that AO allows for a much more aggressive exploration of the loss landscape, proving to empirically outperform JO .
Revised: This supports our hypothesis that AO allows for a much more aggressive exploration of the loss landscape, proving to empirically outperform JO .

Original: With smoothing, we ﬁnd both AO+RS and JO+RS perform better than AO and JO respectively across iterations.
Revised: With smoothing, we ﬁnd both AO+RS and JO+RS perform better than AO and JO respectively across iterations.

Original: We thus recommend using
Revised: We thus recommend using

Original: AO+RS with 1 - 3 iterations as an attack generator to train models that are robust to such adversarial attacks.
Revised: AO+RS with 1 - 3 iterations as an attack generator to train models that are robust to such adversarial attacks.

Original: [Figure]
Revised: [Figure]

Original: [Figure]

Original: In Figure 4b, we plot the performance of the best performing methods as we vary the attacker’s perturbation strength ( k ).
Revised: In Figure 4b, we plot the performance of the best performing methods as we vary the attacker’s perturbation strength ( k ).

Original: We make two key observations – First, allowing a few sites ( < 5 ) to be perturbed is enough to achieve 80% of the best achievable attack rate.
Revised: We make two key observations – First, allowing a few sites ( < 5 ) to be perturbed is enough to achieve 80% of the best achievable attack rate.

Original: For example, under the AO+RS
Revised: For example, under the AO+RS

Original: at-
Revised: at-

Original: tack, the ASR is 50 when k =and 60 when k
Revised: tack, the ASR is 50 when k =and 60 when k

Original: = 20 .
Revised: = 20 .

Original: From an attacker’s perspective, this makes it convenient to effectively attack models of programs without being discovered.
Revised: From an attacker’s perspective, this makes it convenient to effectively attack models of programs without being discovered.

Original: Second, we observe that the best performing methods we propose consistently outperform B ASELINE across different k .
Revised: Second, we observe that the best performing methods we propose consistently outperform B ASELINE across different k .

Original: The performance with B ASELINE begins to converge only after k = 10 sites, suggesting the effectiveness of our attack.[Figure]
Revised: The performance with B ASELINE begins to converge only after k = 10 sites, suggesting the effectiveness of our attack.

Original: [Table] [Table]
Revised: [Table] [Table]

Original: Improved adversarial training under proposed attack.
Revised: Improved adversarial training under proposed attack.

Original: Adversarial training (AT) (Madry et al., 2018) is a min-max optimization based training method to improve a model’s adversarial robustness.
Revised: Adversarial training (AT) (Madry et al., 2018) is a min-max optimization based training method to improve a model’s adversarial robustness.

Original: In AT, an attack generator is used as the inner maximization oracle to produce perturbed training examples that are then used in the outer minimization for model training.
Revised: In AT, an attack generator is used as the inner maximization oracle to produce perturbed training examples that are then used in the outer minimization for model training.

Original: Using AT, we investigate if our proposed attack generation method ( AO+RS ) yields an improvement in adversarial robustness (over B ASELINE ) when it is used to adversarially train the SEQ 2 SEQ model.
Revised: Using AT, we investigate if our proposed attack generation method ( AO+RS ) yields an improvement in adversarial robustness (over B ASELINE ) when it is used to adversarially train the SEQ 2 SEQ model.

Original: We evaluate AT in three settings – ‘No AT’, corresponding to the regularly trained SEQ 2 SEQ model (the one used in all the experiments in Table 1), B ASELINE - SEQ 2 SEQ trained under the attack by (Ramakrishnan et al., 2020), and AO+RS - SEQ 2 SEQ trained under our AO+RS attack.
Revised: We evaluate AT in three settings – ‘No AT’, corresponding to the regularly trained SEQ 2 SEQ model (the one used in all the experiments in Table 1), B ASELINE - SEQ 2 SEQ trained under the attack by (Ramakrishnan et al., 2020), and AO+RS - SEQ 2 SEQ trained under our AO+RS attack.

Original: We use three attackers on these models – B ASELINE and two of our strongest attacks - AO and AO+RS .
Revised: We use three attackers on these models – B ASELINE and two of our strongest attacks - AO and AO+RS .

Original: The row corresponding to ‘No AT’ is the same as the entries under k = 1 in Table 1.
Revised: The row corresponding to ‘No AT’ is the same as the entries under k = 1 in Table 1.

Original: We ﬁnd AT with B ASELINE improves robustness by ∼ 11 points under AO+RS , our strongest attack.
Revised: We ﬁnd AT with B ASELINE improves robustness by ∼ 11 points under AO+RS , our strongest attack.

Original: However, training with AO+RS provides an improvement of ∼ 16 points.
Revised: However, training with AO+RS provides an improvement of ∼ 16 points.

Original: This suggests AO+RS provides better robustness to models when used as an inner maximizer in an AT setting.
Revised: This suggests AO+RS provides better robustness to models when used as an inner maximizer in an AT setting.

Original:  C ONCLUSION
Revised:  C ONCLUSION

Original: In this paper, we propose a general formulation which mathematically deﬁnes an adversarial attack on program source code.
Revised: In this paper, we propose a general formulation which mathematically deﬁnes an adversarial attack on program source code.

Original: We model two key aspects in our formalism – location of the transformation, and the speciﬁc choice of transformation.
Revised: We model two key aspects in our formalism – location of the transformation, and the speciﬁc choice of transformation.

Original: We show that the best attack is generated when both these aspects are optimally chosen.
Revised: We show that the best attack is generated when both these aspects are optimally chosen.

Original: Importantly, we identify that the joint optimization problem we set up which models these two aspects is decomposable via alternating optimization.
Revised: Importantly, we identify that the joint optimization problem we set up which models these two aspects is decomposable via alternating optimization.

Original: The nature of decomposition enables us to easily and quickly generate adversarial programs.
Revised: The nature of decomposition enables us to easily and quickly generate adversarial programs.

Original: Moreover, we show that a randomized smoothing strategy can further help the optimizer to ﬁnd better solutions.
Revised: Moreover, we show that a randomized smoothing strategy can further help the optimizer to ﬁnd better solutions.

Original: Eventually, we conduct extensive experiments from both attack and defense perspectives to demonstrate the improvement of our proposal over the state-of
Revised: Eventually, we conduct extensive experiments from both attack and defense perspectives to demonstrate the improvement of our proposal over the state-of-the-art attack generation method.  A CKNOWLEDGMENT

Original: -the- art attack generation method
Revised: This work was partially funded by a grant by MIT Quest for Intelligence, and the MIT-IBM AI lab. We thank David Cox for helpful discussions on this work. We also thank Christopher Laibinis for his constant support with computational resources. This work was partially done during an internship by Shashank Srikant at MIT-IBM AI lab.

Original: A S OLVING THE PAIR OF PROJECTION SUB - PROBLEMS IN E Q .
Revised: A S OLVING THE PAIR OF PROJECTION SUB - PROBLEMS IN E Q .

Original: The projection step (5) can formally be described as ﬁnding the solution of the convex problem
Revised: The projection step (5) can formally be described as ﬁnding the solution of the convex problem

Original: [Equation]
Revised: [Equation]

Original: subject to 1 T z ≤ k [Equation] z ∈ [ [Equation]
Revised: subject to 1 T z ≤ k, z ∈ [0 , 1] n , 1 T u i = 1 , u i ∈ [0 , 1] | Ω | , ∀ i.

Original: [Equation]

Original: T u

Original: [Equation]

Original: [Equation]

Original: u i ∈

Original: [Equation]

Original: 1] | Ω |

Original: [Equation]
Revised: [Equation]

Original: ∀

Original: i [Equation] [Equation] In Equation 6, we proposed to decompose the projection problem on the combined variables z and u i into the following sub-problems –
Revised: In Equation 6, we proposed to decompose the projection problem on the combined variables z and u i into the following sub-problems –

Original: [Equation] [Equation]
Revised: [Equation] [Equation]

Original: The following proposition is a closed-form solution which solves them.
Revised: The following proposition is a closed-form solution which solves them.

Original: Proposition 1 Let z ( t +1) and { u ( t +1) i } denote solutions of problems given in (6).
Revised: Proposition 1 Let z ( t +1) and { u ( t +1) i } denote solutions of problems given in (6).

Original: Their expressions are given by
Revised: Their expressions are given by

Original: [Equation]
Revised: [Equation]

Original: and µis the root of 1 T [ z
Revised: µ

Original: [Equation]
Revised: [Equation]

Original: t

Original: [Equation]
Revised: [Equation]

Original: = 1; (11)

Original: [Equation]
Revised: [Equation]

Original: (12) where [ · ] + = max { 0 , ·} denotes the (elementwise) non-negative operation, and P [0 , 1] ( · ) is the (elementwise) box projection operation, namely, for a scalar x P [0 , 1] ( x )
Revised: where [ · ] + = max { 0 , ·} denotes the (elementwise) non-negative operation, and P [0 , 1] ( · ) is the (elementwise) box projection operation, namely, for a scalar x P [0 , 1] ( x )

Original: = x if x ∈ [0 , 1] , 0 if x < 0 , and 1 if x > 1 .
Revised: = x if x ∈ [0 , 1] , 0 if x < 0 , and 1 if x > 1 .

Original: Proof.
Revised: Proof.

Original: We ﬁrst reformulate problems in (6) as problems subject to a single inequality or equality constraint.
Revised: We ﬁrst reformulate problems in (6) as problems subject to a single inequality or equality constraint.

Original: That is, min 1 T z ≤ k (cid:107)
Revised: That is, min 1 T z ≤ k (cid:107)

Original: z − z ( t ) (cid:107) 

Revised: z − z ( t ) (cid:107)

Original: + I [0 , 1] n ( z ) and min 1 T u i =1 (cid:107) u i − u
Revised: + I [0 , 1] n ( z ) and min 1 T u i =1 (cid:107) u i − u

Original: t ) i (cid:107) 

Revised: t ) i (cid:107)

Original: + I u i ≥ 0 ( u i ) , where I C ( P ) denotes an indicator function over the constraint set C and I C
Revised: + I u i ≥ 0 ( u i ) , where I C ( P ) denotes an indicator function over the constraint set C and I C

Original: ( P ) = 

Revised: ( P ) =

Original: if P ∈ C and ∞ otherwise.
Revised: if P ∈ C and ∞ otherwise.

Original: We then derive Karush–Kuhn–Tucker (KKT) conditions of the above problems for their optimal solutions; see (Parikh & Boyd, 2014; Xu et al., 2019) for details.
Revised: We then derive Karush–Kuhn–Tucker (KKT) conditions of the above problems for their optimal solutions; see (Parikh & Boyd, 2014; Xu et al., 2019) for details.

Original: (cid:3)
Revised: (cid:3)

Original: In (11) and (12), we need to call an internal solver to ﬁnd the root of a scalar equation, e.g., 1 T [ z ( t ) − µ 1 ] + = 1 in (11).
Revised: In (11) and (12), we need to call an internal solver to ﬁnd the root of a scalar equation, e.g., 1 T [ z ( t ) − µ 1 ] + = 1 in (11).

Original: This can efﬁciently be accomplished using the bisection method in the logarithmic rate O ( − log 2 (cid:15) ) for the solution of (cid:15) -error tolerance (Boyd et al., 2004).
Revised: This can efﬁciently be accomplished using the bisection method in the logarithmic rate O ( − log 2 (cid:15) ) for the solution of (cid:15) -error tolerance (Boyd et al., 2004).

Original: Eventually, the PGD-based
Revised: Eventually, the PGD-based

Original: JO solver is formed by (4), (5), (11) and (12).
Revised: JO solver is formed by (4), (5), (11) and (12).

Original: B P ROGRAM T RANSFORMATIONS
Revised: B P ROGRAM T RANSFORMATIONS

Original: To compare our work against the state-of-the-art (Ramakrishnan et al., 2020), we adopt the transformations introduced in their work and apply our formulation to them.
Revised: To compare our work against the state-of-the-art (Ramakrishnan et al., 2020), we adopt the transformations introduced in their work and apply our formulation to them.

Original: This allows to solve the same setup they try while contrasting the advantages of our formulation.
Revised: This allows to solve the same setup they try while contrasting the advantages of our formulation.

Original: They implement the following transformations -
Revised: They implement the following transformations -

Original: • Renaming local variables.
Revised: • Renaming local variables.

Original: This is the most common obfuscation operation, where local variables in a scope are renamed to less meaningful names.
Revised: This is the most common obfuscation operation, where local variables in a scope are renamed to less meaningful names.

Original: We use this to replace it with a name which has an adversarial effect on a downstream model.
Revised: We use this to replace it with a name which has an adversarial effect on a downstream model.

Original: • Renaming function parameters.
Revised: • Renaming function parameters.

Original: Similar to renaming variable names, function parameter names are also changeable.
Revised: Similar to renaming variable names, function parameter names are also changeable.

Original: We replace names of such parameters with • Renaming object ﬁelds.
Revised: We replace names of such parameters with • Renaming object ﬁelds.

Original: A class’ referenced ﬁeld is renamed in this case.
Revised: A class’ referenced ﬁeld is renamed in this case.

Original: These ﬁelds are referenced using self and this in Python and Java respectively.
Revised: These ﬁelds are referenced using self and this in Python and Java respectively.

Original: We assume that the class deﬁnitions corresponding to these objects are locally deﬁned and can be changed to reﬂect these new names.
Revised: We assume that the class deﬁnitions corresponding to these objects are locally deﬁned and can be changed to reﬂect these new names.

Original: • Replacing boolean literals A boolean literal (True, False) occurring in the program is replaced with an equivalent expression containing optimizable tokens.
Revised: • Replacing boolean literals A boolean literal (True, False) occurring in the program is replaced with an equivalent expression containing optimizable tokens.

Original: This results in a statement of the form <token> ==
Revised: This results in a statement of the form <token> ==

Original: <
Revised: <

Original: token> and <token> !
Revised: token> and <token> !

Original: = <
Revised: = <

Original: token> for True and False respectively.
Revised: token> for True and False respectively.

Original: • Inserting print statements.
Revised: • Inserting print statements.

Original: A print with optimizable string arguments are inserted at a location recommended by the site-selection variable.
Revised: A print with optimizable string arguments are inserted at a location recommended by the site-selection variable.

Original: See Figure 2 for an example.
Revised: See Figure 2 for an example.

Original: • Adding dead code.
Revised: • Adding dead code.

Original: A statement having no consequence to its surrounding code, guarded by an if -condition with an optimizable argument, is added to the program.
Revised: A statement having no consequence to its surrounding code, guarded by an if -condition with an optimizable argument, is added to the program.

Original: The last two transformations are insert transforms, which introduce new tokens in the original program, whereas the others are replace transforms, which modify existing tokens.
Revised: The last two transformations are insert transforms, which introduce new tokens in the original program, whereas the others are replace transforms, which modify existing tokens.

Original: They implement two other transformations – inserting a try-catch block with an optimizable parameter, and unrolling a while loop once.
Revised: They implement two other transformations – inserting a try-catch block with an optimizable parameter, and unrolling a while loop once.

Original: We did not add these to our ﬁnal set of transforms since they are very similar to adding print statements and dead code, while producing a negligible effect.
Revised: We did not add these to our ﬁnal set of transforms since they are very similar to adding print statements and dead code, while producing a negligible effect.

Original: Adding every insert transformation increases the number of variables to be optimized.
Revised: Adding every insert transformation increases the number of variables to be optimized.

Original: Since these two transforms did not 
Revised: Since these two transforms did not impact

Original: C
Revised: C

Original: A N A TTACK E XAMPLE
Revised: A N A TTACK E XAMPLE

Original: D A DDITIONAL R ESULTS D. J AVA DATASET
Revised: D A DDITIONAL R ESULTS D. J AVA DATASET

Original: We tabulate results of evaluating our formulation on an additional dataset containing Java programs.
Revised: We tabulate results of evaluating our formulation on an additional dataset containing Java programs.

Original: This dataset was released by Alon et al.
Revised: This dataset was released by Alon et al.

Original: (2018) in their work on CODE 2 SEQ .
Revised: (2018) in their work on CODE 2 SEQ .

Original: The transformations were implemented using Spoon (Pawlak et al., 2016).
Revised: The transformations were implemented using Spoon (Pawlak et al., 2016).

Original: We ﬁnd the results to be consistent with the results on the Python dataset.
Revised: We ﬁnd the results to be consistent with the results on the Python dataset.

Original: AO + RS provides the best attack.
Revised: AO + RS provides the best attack.

Original: [Table]
Revised: [Table]

Original: See Table 1 for results 
Revised: See Table 1 for results on a Python

Original: D. F ALSE POSITIVE RATE (FPR) UNDER DIFFERENT ATTACKS
Revised: D. F ALSE POSITIVE RATE (FPR) UNDER DIFFERENT ATTACKS

Original: We evaluated the False Positive Rate (FPR) of the model when provided perturbed programs as inputs.
Revised: We evaluated the False Positive Rate (FPR) of the model when provided perturbed programs as inputs.

Original: It is important the perturbations made to the programs cause the model to start predicting ground truth tokens instead of incorrect tokens.
Revised: It is important the perturbations made to the programs cause the model to start predicting ground truth tokens instead of incorrect tokens.

Original: In principle, our optimization objective strictly picks replacements which degrade the classiﬁer’s performance.
Revised: In principle, our optimization objective strictly picks replacements which degrade the classiﬁer’s performance.

Original: As a consequence, we should expect that the model does not end up perturbing the program in a way which leads it closer to the ground truth.
Revised: As a consequence, we should expect that the model does not end up perturbing the program in a way which leads it closer to the ground truth.

Original: We empirically ﬁnd the FPR consistent with this explanation.
Revised: We empirically ﬁnd the FPR consistent with this explanation.

Original: As seen in Table 5, in all our attacks, the FPR is almost 0, validating that our perturbations do not introduce changes which result in the model predicting the right output.
Revised: As seen in Table 5, in all our attacks, the FPR is almost 0, validating that our perturbations do not introduce changes which result in the model predicting the right output.

Original: [Table]
Revised: [Table]

Original: D. E FFECT OF VARIOUS TRANSFORMATIONS
Revised: D. E FFECT OF VARIOUS TRANSFORMATIONS

Original: We study the effect of different transformations used in our work on the attack success rate of our attacks.
Revised: We study the effect of different transformations used in our work on the attack success rate of our attacks.

Original: In our analysis, we found that omitting print statements do not affect the ASR of our attacks.
Revised: In our analysis, we found that omitting print statements do not affect the ASR of our attacks.

Original: This is helpful since for the current task we evaluate (generating program summaries), a print statement likely affects the semantics of the task.
Revised: This is helpful since for the current task we evaluate (generating program summaries), a print statement likely affects the semantics of the task.

Original: [Table]
Revised: [Table]

Original: We further investigated the effect of variable names and function parameter names on our attacks.
Revised: We further investigated the effect of variable names and function parameter names on our attacks.

Original: The SEQ 2 SEQ model and the task of generating program summaries can appear to overly rely on the presence of variable and parameter names in the program.
Revised: The SEQ 2 SEQ model and the task of generating program summaries can appear to overly rely on the presence of variable and parameter names in the program.

Original: To empirically ascertain this, we masked out all the variable and parameter names from our training set and trained the model with placeholder tokens.
Revised: To empirically ascertain this, we masked out all the variable and parameter names from our training set and trained the model with placeholder tokens.

Original: We ﬁnd the model’s performance to remain similar (row All - vars, function params : 1 , Table 6).
Revised: We ﬁnd the model’s performance to remain similar (row All - vars, function params : 1 , Table 6).

Original: Likewise, the attack performance also remains similar.
Revised: Likewise, the attack performance also remains similar.

Original: We test another condition where we mask all these token names in the test set as well, during inference.
Revised: We test another condition where we mask all these token names in the test set as well, during inference.

Original: We ﬁnd a decrease in performance (row 4, Table 6) under attack strength k = 1 , but the trend observed remains – AO+RS > AO > B ASELINE .
Revised: We ﬁnd a decrease in performance (row 4, Table 6) under attack strength k = 1 , but the trend observed remains – AO+RS > AO > B ASELINE .

Original: The ASR is largely unchanged under k = 5 .
Revised: The ASR is largely unchanged under k = 5 .