Venue: ICLR 2021 Conference

Reviews:

final_score: rating: 6: Marginally above acceptance threshold
review_content: Summary:
This paper presents a principled way of applying obfuscation transformation to create adversarial programs. A general mathematical formulation is presented to model the site locations and the transformation choice for each location in a perturbed program. Then, a set of first-order optimization algorithms are used to solve the proposed formulation.

Reasons for score:
The central contribution of this work is the addition of a formal approach to select perturbation locations along with the types of transformations for adversarial program generation. The mathematical formulation enables a principled approach to achieve this. However, the overall efficacy of the proposed approach over the state-of-the are demonstrated using only one model and dataset. Also, the evaluation method could use some clarification (Please see the weakness section below). I believe a more elaborate analysis will strengthen the claim and make the incremental contribution more generalizable.

Strength
1. The formulations for site-selection, site-perturbation, and perturbation strength provides a principled way of generating adversarial programs by casting it as a constrained optimization problem. It focuses on the problem of finding both optimal location and optimal transformation whereas the state-of-the-art addresses only the latter. 
2. The proposed joint optimization improves attack success rate compared to the state-of-the-art approach.
3.  Ablation study is performed to demonstrate  the effectiveness of randomized smoothing. 

Weakness
1. The comparative study would be more convincing with evaluation results from more than one model architecture and dataset.
2. Evaluation is performed only on fully correctly classified samples. Although this approach provides a good way of understanding the perturbation effects on original positive examples, any impact on original negative examples is not visible from these experiments. I think the change in the false positive rates from the original model results could be useful to understand the impact of the generated obfuscations.


Questions to author: (Please address the cons above)
1. Yefet et al. (2019) generates adversarial examples for “targeted attacks”. How effective is the proposed approach for  a “targeted attack”?

review_title: Generating Adversarial Computer Programs using Optimized Obfuscations 

final_score: rating: 7: Good paper, accept
review_content: Summary:
* This paper proposes an optimization problem to adopt insert/replace operations (program obfuscations) to generate adversarial programs. They apply it to the task of program summarization, and show that they outperform the existing baseline published in 2020. In particular, one of the main contributions is the identification of the site-perturbation and site-selection process, and formalizing them as practical optimization based on PGD. 

Strengths:
* The problem is relevant and challenging
* Formalization is elegant and clear 
* Contributions and related works are relevant and significant
* Experimental comparison against relevant, recent work 

Weaknesses: 
* The task of program summarization is somehow sinmple
* It is not well-motivated why to limit the perturbation strength k, since we are talking about a program (see also discussion in [2])
* Attack success rate improvement over baseline is moderate. 
* There seems to be no plan for the authors to publicly release the code.

Detailed comments: 
* The intuition of using site-selection and site-perturbation is extremely interesting and is well developed. The formalization of the problem is very elegant, and is convincing. Modeling the “location” of the program modification and identifying that this increases attack success rate is a very important contribution.  
* Related work is generally relevant and significant. I also strongly appreciate that there is an experimental comparison against a recent paper from 2020, representing the state of the art. As related work, I would also recommend the authors to have a look at, which seems highly related from the security domain (see [1,2,3] below). 
* I think the task of program summarization is fairly simple, and it would have been interesting to see something a bit more complex. Some “replace” operations such as renaming of variables seems to be very related to the attack in [1]. The “insertion” operation seems to be quite trivial, such as adding prints or dead code. This could be easily defeated via preprocessing commonly done by compiler’s program analysis, such as unreachable code elimination (see problem-space constraint [2]).
* Nevertheless, I do believe that the paper proposes a very interesting perspective. I would also encourage the authors to consider releasing the source code. 
* Minor comments: 
    * Typo “reproducability” on page 6

References:
* [1] Quiring et al. "Misleading authorship attribution of source code using adversarial learning.” USENIX Security Symposium, 2019.
* [2] Pierazzi et al, “Intriguing Properties of Adversarial ML Attacks in the Problem Space”, IEEE Symp. S&P 2020
* [3] Demetrio, Luca, et al. "Adversarial EXEmples: A Survey and Experimental Evaluation of Practical Attacks on Machine Learning for Windows Malware Detection." arXiv preprint arXiv:2008.07125 (2020).

review_title:  ICLR 2021 Review Conference Paper1336

final_score: rating: 6: Marginally above acceptance threshold
review_content: # Summary

This work tackles the problem of adversarial attacks against ML models for code-understanding tasks, such as function summarization. It formulates the problem as the adversarial application of existing semantics-preserving program transformations (e.g., renaming variables), by jointly optimizing on the location of such a transformation, and the argument to the transformation (e.g., what to replace an existing identifier with). It shows that such adversarial examples increase the attack success rate over baseline approaches, and training with such examples increases the robustness of the resulting model to the same or baseline attacks.

# Strengths:
1. Clear description of the mathematical formulation and the approach.
1. Results over the baselines and seq2seq model seem to be very promising.

# Weaknesses:
1. The core model used is not a state-of-the-art model for code understanding. Transformer or GGNN-based models have been shown to do much better on code-understanding tasks than seq2seq models.
1. The choice of task and perturbation actions are expedient to try out the approach, but practically questionable (see below).
1. The choice of model on which to evaluate the approach is poor; seq2seq is a weak baseline for function-naming tasks.


# Feedback

1. As an application of techniques for adversarial robustness to code-understanding tasks, this paper is a great read. I enjoyed the description of the approach (although I'm not an expert on adversarial examples), and found the direction intuitive and well presented. I enjoyed reading your paper.

1. However, digging below the surface, I'm uneasy with several aspects of the concrete realization of your approach.

1. Perturbations must be semantics preserving. Although most of the perturbations you describe have that property, inserting `print` statements does not. Changing the output of a program constitutes a major modification of semantics, unless you further limit, precisely, your definition of "semantics preserving". As an obvious example, a function
```
def ___(a):
    result = a*2
    return result
```
is very similar to 
```
def ___(a):
    result = a*2
    print(result)
    return result
```
however I could argue that first should be named `compute_double` while the second should be named `compute_and_print_double`. I would include in this category logging statements as well; they alter the program semantics. What might not are operations that have no side effects (as far as inputs/outputs/state are concerned), for instance writing to `/dev/null`

1. I'm also a little skeptical about renaming local variables and function parameters. Those may have an inordinate influence on the particular model and task you chose. However, some work has found that tokens themselves don't affect the results of some models. For example, in `Allamanis, Miltiadis, Marc Brockschmidt, and Mahmoud Khademi. "Learning to represent programs with graphs." arXiv preprint arXiv:1711.00740 (2017).`, having token labels makes no difference when solving a variable-misuse task, but actually helps when solving variable naming (see Table 2).

1. What's more, there's some evidence that completely removing identifiers from a program (interestingly enough, to obfuscate it) may not destroy the relevant information needed to recover the identifiers. See, for example `Bichsel, Benjamin, Veselin Raychev, Petar Tsankov, and Martin Vechev. "Statistical deobfuscation of android applications." In Proceedings of the 2016 ACM SIGSAC Conference on Computer and Communications Security, pp. 343-355. 2016.` which recovers missing identifiers. In principle, that tells me that useful identifiers may still be recoverable from completely obfuscated programs from contextual structure. So this leads me to believe that what you present here is more about a model that wasn't meant to understand relational properties of its input rather focusing instead on tokens alone, and less about an inherent limitation in how we build neural models for code.

1. Even ignoring the above, I'd want to see more about how imperceptibility applies to the task at hand. Your attacks may well change a predicted name from its original to something else, but how far away is that new prediction, and does it matter? When classifying, switching classes is an important effect of attack. However, when predicting a description, changing predicted tokens doesn't necessarily change the meaning of the description. What kind of change constitutes a successful attack and what kind of change doesn't disturb the correctness of the prediction? As a perhaps naive example, if you attacked a function to change its name from `read_file_from_disk`  to `collect_contents_from_storage`, although a huge change in tokens involved, I'd say the attack was unsuccessful.

1. Finally, to lend credence to your results, I would want to see your approach applied to state-of-the-art models for program understanding. seq2seq is not state of the art. For variable-name prediction, something like `Alon, Uri, Shaked Brody, Omer Levy, and Eran Yahav. "code2seq: Generating sequences from structured representations of code." arXiv preprint arXiv:1808.01400 (2018).` might be more appropriate.

1. All in all, I learned much about adversarial training from reading your paper (since I hadn't read much about it before). However, I don't find that your results will change how models of code are built, because they are evaluated on what is now considered a fairly weak model basis, and the criteria for what constitutes a successful attack are not well defined and appear overly pessimistic.


review_title: Adversarial code mutations by jointly choosing mutation site and type 

Meta-review: This is a nice paper on generating adversarial programs. The approach is to carefully use program obfuscators. After discussion and improvements, reviewers were generally satisfied with the approach and evaluation. The problem domain was also found to be of interest.
Decision: Accept (Poster)