# Partial Retraining Substitute Model for Query-Limited Black-Box Attacks
## 3 Proposed Method
### Retraining the Substitute Model  
The proposed method sends a query containing the crafted adversarial example to the target model. After classification by the target model, the proposed method receives the result, i.e., a label, in response to the query. If the attack fails, the labeled example becomes a new training sample for the substitute model. We then have to change the classification boundary before the next attack. Note that the substitute model classifies the sample as the target class because it is crafted based on the substitute model. In the retraining process, the proposed method iteratively inputs the sample into the substitute model until it classifies the sample as the received label. Because the substitute model is already trained, this retraining process may require a certain number of iterations. In this study, we ignore the training efficiency of the substitute model. For practical black-box attacks, we focus only on the number of queries and attack success ratio. After retraining, the proposed method crafts another adversarial example based on the retrained substitute model. Subsequently, crafting of an adversarial example and retraining the substitute model are repeated until the attack is successful or the number of iterations reaches a pre-set threshold. Figure 2 indicates the differences between the previous substitute-model attack and proposed method. Simple classification boundaries are considered when only two-dimensional features are involved. The previous method attempts to emulate the entire target model. All boundaries of the substitute model (dotted lines) are moved toward the boundaries of the target model (solid lines). If the previous  
method sends numerous queries to the target model, the boundaries may become similar to the boundaries of the target model. In contrast, the proposed method modifies only the partial boundary corresponding to the current attack based on one sample. In the current attack, the original is labeled as class 2 and the target is class 1. When the boundary is different from that of the target model and the adversarial example (a red dot) does not cross the solid line, the attack fails. After retraining the substitute model using the failure sample, the partial boundary is changed. This process is repeated until the red dot finally crosses the solid line. Note that the substitute model generated by the previous method can be used for any samples and targeted attacks. In Figure 2, using the original sample (class 2), the previous method can generate adversarial examples targeting classes 0, 1, and 3, whereas the proposed method can target only class 1. However, it is assumed that in a real-world system the possible number of queries is limited. As previously mentioned, the previous method requires a large number of queries to train the substitute model. Under a limited-query condition, the attack success ratio of the proposed method is higher than that of the previous method, as demonstrated by the experiments conducted and presented later in this paper. Algorithm 1 presents the pseudocode for partial retraining of the substitute model. This process repeats until the query limit (line 1) is reached, or when the attack succeeds (line 4-5). In each iteration, an adversarial example, \(x^{\prime}\), is crafted by white-box attack methods applied to the pre-trained substitute model, \(F()\) (line 2). The adversary sends a query containing the adversarial example to the target model, \(\widetilde{F}()\), and receives a classification result, label \(y\) (line 3). If the attack fails, the adversarial example is relabeled with the received label, \(y\), (line 7) and the substitute model is retrained using the relabeled adversarial example (line 9). For efficient training, we construct a retraining data set, \(S\), with adversarial examples from every iteration (line 8), and feed these to the substitute model as a mini-batch. The substitute model is retrained until every adversarial example is classified as a received (valid) label from the target model.