# Partial Retraining Substitute Model for Query-Limited Black-Box Attacks
## 1 Introduction  
Deep neural network (DNN) classifiers have made significant progress in many domains such as image classification [1, 2], voice recognition [3, 4], malware detection [5, 6], and natural language processing [7]. Despite their great success, recent studies have demonstrated that DNNs are vulnerable to well-designed input samples called _adversarial examples_[8, 9]. Legitimate inputs are altered by adding small perturbations to force a trained classifier to misclassify them, while humans cannot perceive the alteration. In other words, the purpose of these attacks is to maximize misclassification while minimizing the perturbations. The adversarial examples represent a serious threat to DNN classifiers. For example, manipulated traffic signs can confuse autonomous vehicles [10, 11] and adversarial voices can deceive automatic voice recognition models [12, 13] such as Apple's Siri and Amazon's Alexa. Early studies of adversarial examples use white-box environments, in which the adversaries can access the target model without restrictions or own the same model. They therefore focus on the attack success ratio and have little interest in how many times they must access the target model. In white-box attacks [8; 9], adversaries have full knowledge regarding the target model, such as network architecture, training data, parameters, and classification results (predicted labels and probabilities). With this information, white-box attacks can calculate the gradients of the target model and craft precise adversarial examples based on the gradients. The success rates in this type of attack are almost 100%. Some studies [14; 15] have demonstrated that defending DNNs from white-box attacks is extremely difficult. In contrast, black-box attacks involve limited knowledge of the target model. In the real world, DNN classification services normally do not provide information regarding their systems to prevent security threats or for business reasons. They wish to prevent adversaries and competitors from obtaining the details of their models. The adversaries can access the target model only through exposed service interfaces, i.e., using queries. Therefore, white-box attack algorithms based on gradients cannot be directly applied, and the number of queries becomes an important issue. A well-known black-box attack strategy creates a local DNN, called a substitute model [16; 17], to emulate the target model. The adversaries then craft adversarial examples using the substitute model instead of the unknown target model. The substitute model is trained using a labeled dataset obtained by repeated queries made to the target model. For successful attacks, previous studies focus on generating a dataset composed of proper samples to efficiently train substitute models. However, emulating the target model usually requires a massive number of queries because training a new DNN from the beginning normally requires numerous training data. Moreover, these studies have to emulate the entire target model even for one-sample attacks. They attempt to build a substitute model as a master key for the target model with which adversaries can craft adversarial examples from any samples. In many cases, adversaries wish to perform one attack (from one class to another class) instead of multiple attacks (from classes to classes). For example, an adversary may attempt to deceive a face authentication system by pretending to be a legitimate user. In autonomous vehicle systems, misclassifying a stop sign as a speed limit sign is enough to lead to a traffic accident. Therefore, we argue that black-box attacks can be classified into one-sample attacks and any-sample attacks. Existing studies are suitable for any-sample attacks but are inefficient for one-sample attacks in terms of the number of queries required. In this study, we propose a novel training method applied to the substitute model for one-sample attacks to minimize the number of queries.