## 1 Introduction  
Deep neural networks (DNNs) have become the central approach in modern-day artificial intelligence (AI) research. They have attained superb performance in multifarious complex tasks and are behind fundamental breakthroughs in a variety of machine-learning tasks that were previously thought to be too difficult. Image classification, object detection, machine translation, and sentiment analysis are just a few examples of domains revolutionized by DNNs.  
Despite their success, recent studies have shown that DNNs are vulnerable to adversarial attacks. A barely detectable change in an image, for example, can cause a mis-classification in a well-trained DNN. Targeted adversarial examples can even evoke a misclassification of a specific class (e.g., misclassingly a car as a cat). Researchers have demonstrated that adversarial attacks are successful in the real world and may be produced for data modalities beyond imaging, e.g., natural language and voice recognition [1; 2; 3; 4]. DNNs' vulnerability to adversarial attacks has raised concerns about applying these techniques to safety-critical applications.  
To discover effective adversarial instances, most past work on adversarial attacks has employed gradient-based optimization [5; 6; 7; 8; 9]. Gradient computation can only be executed if the attacker is fully aware of the model architecture and weights. Thus, these approaches are only useful in a white-box scenario, where an attacker has complete access and control over a targeted DNN. Attacking real-world AI systems, however, might be far more arduous. The attacker must consider the difficulty of implementing adversarial instances in a black-box setting, in which no information about the network design, parameters, or training data is provided. In this situation, the attacker is exposed only to the classifier's input-outputpairs. In this context, a typical strategy has been to attack trained replacement networks and hope that the generated examples transfer to the target model [10]. The substantial mismatch of the model between the alternative model and the target model, as well as the significant computational cost of alternative network training, often renders this technique ineffective.  
In our work we assume a real-world, black-box attack scenario, wherein a DNN's input and output may be accessed but not its internal configuration. We focus on a scenario in which a specific DNN is an image classifier, specifically, a convolutional neural network (CNN), which accepts an image as input and outputs a probability score for each class.  
Herein, we present an evolutionary, gradient-free optimization approach for generating adversarial instances, more suitable for real-life scenarios, because usually there is no access to a model's internals, including the gradients; thus, it is important to craft attacks that do not use gradients. Our proposed attack can deal with either constrained (\(\epsilon\) value that constrains the norm of the allowed perturbation) or unconstrained (no constraint on the norm of the perturbation) problems, and focuses on constrained, untargeted attacks. We believe that our framework can be easily adapted to the targeted setting.  
In the next section we review the literature on adversarial attacks. Section 3 summarizes the threat model we assume for our proposed evolutionary attack algorithm. The algorithm itself--_QuEry Attack_ (for **Qu**ery-Efficient **E**volutionary **Attack**)--is delineated in Section 4. The experiments conducted to test the method, along with results, are described in Section 5. We discuss our findings and present concluding remarks in Section 6.  
Figure 1 shows examples of successful and unsuccessful instances of images generated by QuEry Attack, evaluated against ImageNet, CIFAR10, and MNIST.