# Evolutionary Approaches for Adversarial Attacks on Neural Source Code Classifiers
## 1 Introduction  
When vulnerability assessment is AI-assisted, the trust in the system we use to classify source code becomes critical. In this work, we focus on a method to evaluate the risks coming from possible adversarial attacks, that is, on the possibility of adjusting input instances to the system in a way that makes them wrongly classified by our checking system.  
Moreover, we are not interested in just showing that adversarial instances do exist and can be automatically generated, but we also try to find hints about what in the input instances induces mistakes in the behavior of the neural network being used. Neural models are mostly black box systems with respect to how their internal dynamics lead to the eventual classification of an instance; therefore, our second goal is to help look for ways to make the AI systems that are used more secure.  
The overall approach for generating adversarial instances against a neural classifier is well known and developed in domains where the input data consist of real-valued vectors. In that setting, the enabling feature is that deceiving instances can be found by moving in the input space along the gradient of the classification probability that the neural system is outputting [1]. Our focus, instead, is on the task of checking source code snippets for vulnerabilities, and therefore, a different way of exploring the discrete-valued input space is in order. Our choice is an evolutionary optimization of the output of the network. For instance, our system will try to minimize the output when it represents, according to the attacked neural classifier, the probability of having a vulnerability in the input source code, even when it is actually a malicious snippet.  
Moving more to a defending role, from the same process of generating adversarial instances we will derive evidence of what could possibly have a role in deceiving the classifier. Previous works [2], for example, pointed out how the natural elements that occur in the source code have crucial effects in affecting the prediction of a neural network. Therefore, the attack we propose manipulates code snippets by changing identifiers in it, allowing us to extract statistics about which terms are more relevant in confusing the neural network. Such data will point to biases affecting the classification process inside the neural network, and this could finally suggest more robust ways of training it.  
Along these research lines, we advance with the following novel contributions:  
* Definition of an evolutionary-based method to look for adversarial modifications of source code;
* Evaluation of the proposed deceiving method on a state-of-the-art neural model trained on vulnerability detection;
* Investigation on which syntactical elements, related to identifiers in source code, are most impacting the decisions of the classifier.  
In the rest of the paper, Section 2 presents a brief literature review on artificial intelligence for vulnerability detection and adversarial attacks, Section 3 describes the background and the threat model, Section 4 details the performed experiments, Section 5 discusses the results, and Section 6 concludes the work and outlines possible future research directions.