Black-Box Dissector: Towards Erasing-based Hard-Label Model Stealing Attack
Authors: Yixu Wang
Jie Li
Hong Liu
Yan Wang
YONGJIAN WU
Feiyue Huang
Rongrong Ji
Venue: NeurIPS 2021 Submitted (2021)

Abstract: Previous studies have verified that the functionality of black-box models can be stolen with the full probability outputs. However, under the more practical hard-label setting, we observe that existing methods suffer from catastrophic performance degradation. We argue this is due to the lack of rich information in the probability prediction and the over-fitting caused by hard labels. To this end, we propose a novel hard-label model stealing method termed black-box dissector, which consists of two erasing-based modules. One is a CAM-driven erasing strategy that is designed to increase the information capacity hidden in hard labels from the victim model. The other is a random-erasing-based self-knowledge distillation module that utilizes soft labels from the substitute model to mitigate over-fitting. Extensive experiments on four widely-used datasets consistently demonstrate that our method outperforms state-of-the-art methods, with an improvement of at most $8.27\%$. We also validate the effectiveness and practical potential of our method on real-world APIs and defense methods. Furthermore, our method promotes other downstream tasks, i.e.,  transfer adversarial attacks.

Black-Box Dissector: Towards Erasing-based Hard-Label Model Stealing Attack
Anonymous Author(s)
Affiliation Address
email
Abstract
Previous studies have verified that the functionality of black-box models can be stolen with full probability outputs. However, under the more practical hard-label setting, we observe that existing methods suffer from catastrophic performance degradation. We argue this is due the lack of rich information in the probability prediction and the overfitting caused by hard labels. To this end, we propose a novel hard-labed model stealing method termed black-box dissector, which consists of two erasing- brasing-based modules. One is CAM-driver erasing strategy that is designed increase the information capacity hidden hard labels from the victim model. The other is random-erasing. based self-knowled knowledge distillation module that utilizes soft labels from the substitute model to mitigate overfitting. Extensive experiments on four widely-used datasets consistently demonstrate that our method outperforms state-of-the-ar methods, with an improvement of most 8.27%. We also validate the effectiveness and practical potential of our method on real-world APIs and defense methods. Furthermore, our method promotes other downstream tasks, Le.. transfer adversarial attacks.
10
11
12
14
Introduction
16
Machine learning models deployed the cloud can serve users through the application program interfaces (APIs) to improve productivity. Since developing these cloud models is a product of intensive labor and monetary effort, these models valuable intellectual property and Al companies risk adversary
18
20 try to keep them private. However, the exposure of the model's predictions represents a significant
21
leverage this information steal the model's functionality, a.k.a. model stealing attack 22 20 21 With such an attack, adversaries are able to not only use the stolen model make profit, but also mount further adversarial attacks K4] 29]. Besides, the model stealing 24 attacks is a kind of black-box knowledge distillation which is hot research topic. Studying various mechanisms of model stealing attack is of great interest both AI companies and researchers.
22
25
26 Previous methods 20 211 mainly assume the complete probability predictions of the victim model available, while the real-worls APIs usually only return partial probability values (top-k predictions) or even the top-1 prediction (i.e., hard label). this paper, we focus on the more challenging and realistic scenario, i.e., the victim model only outputs the hard labels. However, under 30 this setting, existing methods suffer from a significant performance degradation, even by 30.50% (as shown in the Fig. D (a) and the appendix Tab. l).
28
31
32 To investigate the reason for the degradation, we evaluate the performance of attack methods with different numbers of prediction probability categories available and hard labels in Fig. (b)) With the observation that the performance degrades when the top-A information missing, we conclude that the top-k predictions are informative as it indicates the similarity of different categories or
34
36 multiple objects in the picture, and previous attack methods suffer from such information obscured
Submitted to 35th Conference on Neural Information Processing Systems (NeurIPS 2021). Do not distribute.

---

55 setting will suffer from overfitting, which leads to worse performance, and we verify it by plotting the loss curves in Fig. [ (c). To mitigate 59 this problem, we introduce a simple self-knowledge distillation module with random erasing (RE) to utilize soft labels for generalization. Partic- 37 by the t prediction under the hard-label setting. It motivates us to re-mere re-mine this information by eliminating the top-1 prediction. Particularly, we design a novel CAM-based erasing method, which erases the important area on the pictures based on the substitute model's top-1 class activation maps
38
39
40
(CAM) 22 [3 and queries the victim model for a new prediction. Note that we can dig out other class information in this sample if the new prediction changes. Otherwise, it proves that the substitute model pays attention to the wrong area. Then we can align the attention of the substitute and the victim model by learning clean samples and the corresponding erased samples simultaneously.
41
42
43
44
Besides, previous works on the Kolf-Knowledge Distillation (self-
86
label
84
45
84
hard labe!
82
47
48
46
label BT have pointed out the
hard and noisy labels will introduce overfitting and miscalibration. More
KD) [5]. calibration [8), and noisy
80
82
78
76
78
76
80
AT(K)
AT(E)
74
49
50
specifically, the attack algorithms cannot access the training data, and thus can only use the synthetic data or other datasets as a substitute, which
72
ATIE) AT(K) AA+D)T((+*K))
74
top-k(%)
(altiard label vs soll
(bb
Performance with top- available
51
52
16-
14
16-
144
Training set Test set
53
12
54 is noisy. Therefore, the hare-label
10
10-
56
100
125
200 17
X55
25
100
180 15
58
step
step
(c)W/O self-KD
(d)W/ self-KD
60
Figure 1: (a) The test accuracies of previous methods with hard labels are much lower than the ones with soft labels. (KN: KnockoffNets, 'AT": ActiveThief, 'E': entropy, 'K': k-Center, 'D': DFAL) (b) The performance decreases as the number of available classes decreases (dotted line : hard- label setting). (c) & (d) Loss curves for training/test set during model training without and with self-KD. All results are on the CIFARIO dataset.
61
62
63 ularly, we randomly erase one sample
64 a certain number of times, query
65 the substitute model for soft-label outputs, and take the average value of these outputs as the pseudo-label. After that, we use both hard labels from the victim model and pseudo labels from the previous substitute model to train a new substitute model. Therefore, we can also consider the ensemble of the two models as the teacher in knowledge distillation. As in Fig. (d), such a module helps generalization and better performance.
66
87
68
69
70
71
72 In summary, we propose a novel model stealing framework termed black-box dissector, which
73 includes a CAM-driven erasing strategy and RE-based self-KD module. Our method is orthogonal
74 to previous approaches [20] [2] and can be integrated with them. The experiments on four widely-
75 used datasets demonstrate our method achieves 43.04 90.57% test accuracy (47.60 91.37% agreement) to the victim model, which is at most 8.27% higher than the state of the art method.
78
77 We also proved that our method can defeat popular defense methods and effective for real-worlds APIs like services provided by Amazon Web Services (AWS). Furthermore, our method promotes downstream tasks, i.e., transfer adversarial attack, with 4.91% 16.20% improvement.
78
79
Background and Notions
80
+ R. that performs as similarly 82 as possible to the black-box victim model f: [o, 1]dd + N (with only outputs accessed). Papernot et al. (22) first observed that online models could be stolen through multiple queries. After that, due
81 Model stealing attack is aim to find a substitute model f: [0, 1]
84 to the practical threat to weal-world APIs, several studies paid attention to this problem and proposed
85 many attack algorithms.
These algorithms consist of two stages: 1) constructing a transfer dataset Dy (step in Fig. 2 and 86
87
2) training a substitute model. The transfer dataset is constructed based on data synthesis or data selection and then feed into the victim model for labels. Methods based on data synthesis [4 14
88
89
adopt the GAN-based models to generate a virtual dataset. And the substitute model and the GAN
90 model are trained alternatively on this virtual dataset by querying the victim model iteratively. The
91
data selection methods prepare an attack dataset as the data pool, and then sample the most informative

---

Data flow
Training model
meddmm ereeing
Step-
Average
Step-2.2
fnew
Step
Dr
Top
300TIN maad 1E)
Figure 2: Details of our proposed black-bol box dissector with a CAM-driven crasing strategy (step 2.1) and RE-based self-KD module (step 2.2). In step 2.1, the images in transfer set Dr erased according to the Grad-CA.M, and we selected the erased images with the largest difference from the original images according to the substitute model's 's outputs. In step 2.2, we randomly erase the unlabeled image times, and then average the outputs of the N erased images by the substitute model the pseudo-label.
data via machine learning algorithms, e.g., reinffrrcmeet learning 220 or active learning strategy (21] 93 uncertainty- strategy [17), k-Center strategy [25], and DFAL strategy [5] Considering that querying the victim model will be costly, the attacker usually sets budget the number of the 95 queries, so the size of the transfer dataset should be limited as well. Previous methods assume the victim model returns complete probability prediction f(æ), which is less practical.
97 In this paper, we focus on more practical scenario that is about hard-label (((()) setting, where 38 is the truncation function used to truncate the information contained in the victim's 's output and return
89 the corresponding one-hot vector:
[1 if arg max, f(x),
(((+)),
o otherwise.
With the transfer dataset, the substitute model is optimized by minimizing a loss function & (e.g., cross-entropy loss function):
101
[(((t)) I Ey Dr ((o(f(=)), f())), for hard labels.
E Er Dr
for soft labels;
Knowledge distillation (KD) has been widely studied in machine learning [0 6), which transfers knowledge from teacher model to student model. Model stealing attacks can regarded as a black-box KD problem where the victim model is the teacher with only outputs accessible and the substitute model is the student. The main reason for the success KD is the valuable information that defines rich similarity structure over the data in the probability prediction [10]. However, for the hard-label setting discussed in this paper, this valuable information is lost. Inspired by KD, method tries to dig out the hidden information in the data and models, and then transfers more knowledge to the substitute model.
103
105
107
109
110
The erasing- based method, e.g., random erasing (RE) 32 3, is currently one of the widely used data augmentation methods, which generates training images with various levels occlusion, thereby reducing the risk of over-fitiing and improving the robustness of the model. Our work is inspired
111
113
by RE and designs a prior-driven erasing operation, which erases the area corresponding to the hard label to re-mine missing information.

---

115 3
Method
The overview of our proposed black-box dissector is shown in Fig. In addition the conventional process (i.e., the transfer dataset Dy constructing in step and the substitute model training in the right), introduce two key modules: a CAM-driven erasing strategy (step 2.1) and a RE-based self-KD module (step 2.2).
2
117
118
119
120
3.1
A CAM-driven erasing strategy
Since the lack of class similarity infor- mation degrades the performance of previous methods under the hard-label setting, we try to re-dig out such hid- den information. Taking an example from the ILSVRC-2012 dataset for il- lustration as in Fig.Bl 3 Querying the
122
124
126
Commen yrunennyydoooaee vontboot
CUBS200 trained victim model with this image, get two classes with the highest confidence score: "Anna hummingbird" (0. (3.4 and "Com- mon yellowthroat" 165), and show their corresponding attention map in the first column of Fig. 3 It is easy to conclude that two different attention
128
129
130
132
Alsecoonn map
Original image
Zoom
Comesponding image
133
134
Figure 3: An example from the ILSVRC-2012 dataset and its attention map corresponding to two most likely class "Anna humming bird" and "Common yellow throat" on the CUBS200 trained model. The attention areas share similar visual apparent with images of "Anna humming bird" and "Common yellow throat", respectively.
135
136
regions response for different classes according to the attention map. When training the substitute model with the hard label "Anna hummingbird" and without the class similarity informa- tion, the model can not learn from the area related to the "Common yellowthroat" class, which means this area is wasted. To re-dig out the information about the "Common yellowthroat" class, need to crase the impact the "Anna hummingbird" class.
137
138
139
140
141
143
144 To this end, natural idea is to erase the response area corresponding to the hard label. Since the victim model is a black-box model, we use the substitute model to approximately calculate the attention map instead. If the attention map calculated by the substitute model is inaccurate and the
145
148
147
victim model's prediction on the erased image does not change, we can also align the attention map of
148
two models by letting the substitute model learn the original image and the erased one simultaneously. The attention map also kind of supervision signal pushing two models to be similar [30. To
149
get the attention map, we utilize the Grad-CAM [24] in this paper. With the input image [0, 1]d
150
and the trained DNN F: [0, 11tt + RR-VV we let of denote the weight of class corresponding to the aF(w)) where Z is the number of pixels in the Σ; E, aA",
151
152
k-th feature map, and calculate it as of
159
feature map, F(z)) is the score of class c and A is the value of pixel at (i in the k-th feature map. After obtaining the weights corresponding to all feature maps, the final attention map can be obtained as St CAM CAM = ReLU (EE At) via weighted summation.
154
155
To erase the corresponding area, inspired by (32). define prior-driven crasing operation as (I, P), shown in Alg. [ which randomly crases a rectangle region in the image I with random values while the central position of the rectangle region is randomly selected following the prior probability P. The prior probability P is of the same size as the input image and is used to determine the probability of different pixels being erased. Here, we the attention map from Grad-CAM as max [(x) the prior. Let x E [o, 1)d denote the input image from the transfer set and Grad CAM (ar, f) denote the attention map of the substitute model f. This CAM-driven erasing operation can be represented:
157
159
160
162
arg max f(x) vb [x.
Srad CAM (ar, f)
We abbreviate it as (xx S(x, f)). To alleviate the impact of inaccurate CAM caused by the difference 164 between the substitute model and the victim one, for each image, we perform this operation N times (t) means the i-th crasing) and select the one with the largest difference from the original label.
163

---

Algorithm 1: Prior-driven Erasing Operation () P)
Input:
Input image L, prior probability P. image size W and H, area of image S, crasing area ratio range 9ı and Sh, erasing aspect ratio range ri and r2- Output: Erased image I'.
Se + Rand(st, 5n) X S. <- Rand(ri,
2 He t vSe 2e/2,, W. + -V
Te: Me sampled randomly according to P
4 Le f (Ie- We, Be He, Te + Wes Ue + He)
I(Ie) f Ro,d(0, 255)
6 1' f+1
186
Such a data augment operation helps the erasing process to more robust. We use the cross-entropy to calculate the difference between the new label and the original label, and we want to select the sample with the biggest difference. Formally, we define 11(x:) as the function to select the most different variation of image x:
167
188
189
[I()) (x, f),,
where k arg max
Ed(f(E)) llog (((,) f(1),))
VE(N]
arg max log ((,,, arg max (f(x))
(4)
(E(N]
max arg min / ( (vx(,,,(,;,)) a (fw))
TE[N]
Due to the limitation of the number of queries, cannot query the victim model for each erased image to obtain a new label. We continuously choose the erased image with the highest substitute's confidence until reaching the budget. To measure the confidence of the model, adopt the Maximum Softmax Probability (MSP) for its simplicity:
171
173
arg max MSP (f (II()))
z~D,
(5)
arg max f (II (I)) arg max f(Il()))*
2~Dr
where Dr is the transfer set. The erased images selected in this way most likely to change the prediction class. Then, we query the victim model to get these erased images' labels and construct erased sample set Dg- Note that when the victim model's predictions the erased images change, it means our erasing method does dig out other related class information in the sample. With the unchanged predictions, it points out the attentions of the substitute model and the victim inconsistent. Though wrong attention areas erased, training with these samples benefits aligning the attentions of two models. As 30] stated, the attention alignment can help more powerful KD.
175
177
179
3.2
A random-erasing- self-KD module
182
We also find that in training with limited hard-label OOD samples, the substitute model is likely to overfit the training set, which damages its generalization ability [15] B1]. Therefore, based on the above erasing operation, we further design simple RE-based self-KD method improve the generalization ability of the substitute model.
183
184
Formally, let [0, 1] denote the unlabeled input image. We perform the erasing operation with uniform prior U' on it N times, and then average the substitute's 's outputs on these erased images as the pseudo-label of the original image:
186
187
188
N Σ (v)(,,
yp (2,
(6)
N
1=1
'Rand(a, b) returns evenly distributed random real number in the range of a to

---

Algorithm 2: Black-box Dissector
Input: Unlabeled pool Du, victim model maximum number of queries Q.
Output: Substitute model f. Initialize - 0, Dr + 2, DE +
while q Q
// Step
Select samples from Dy according budget and query f to updata Dr
ạ = q + budget
ExED, f'(o(f()).f())))
+ update(f.L)
A CAM-driven erasing strategy (step 2.1)
Erase samples in Dr according to E
10
Choose samples from erased samples according to Eq and budget Query f to get labels and updata D
EZED,UDA ('(o(f())).f(x))
12
f + update(f,
14
= + budget
15
I/ A random-erasing-
self-KD (step 2.2) Select samples from Du
16
17
Get pseudo- labels according to Eq. and construct pseudo-label set Dp
L Σze DEED, ('(o(f()), f(x)) + ErED, (') f())
18
19
+ update(f.
end
189
This is a type of consistency regularization, which enforces the model to have the same predictions for the perturbed images and enhances the generalization ability. With Eq we construct a new soft pseudo label set Dp (Ix (p(2, f).....
191
192
With the transfer set Dr, the erased sample set DE. and the pseudo-label set Dp, we train new substitute model using the ensemble of the victim model and the previous substitute model as the teacher. Our final objective function is:
194
min C = min
Σ C(o(((=)), + E((n(tt,)))))))
(7)
DEDUUD
DEDr
185
where L' can be commonly used loss functions, e.g., cross-entropy loss function.
To up, built our method on the conventional process of the model stealing attack (step 1). and proposed a CAM-driven erasing strategy (step 2.1) and RE-based self-KD module (step
197
2.2) unified by novel erasing method. The former strategy digs out missing information between classes and aligns the attention while the latter module helps to mitigate overfitting and enhance the generalization. We name the whole framework black-box dissector and present the algorithm detail of it in 2
199
201
Experiment
202
4.1
Experiment settings
204
Victim model. The victim models we used (ResNet-34 (9) are trained on four datasets, namely,
CIFARIO [6], SVHN [19), Caltech256 [7], and CUBS200 28], and their test accuracy are 91.56% 96.45%. 78.40%, and 77.10%, respectively. All models are trained using the SGD optimizer with
206
momentum (of 0.5) for 200 epochs with base learning rate of 0.1 decayed by factor of 0.1 every
208
30 epochs. Following 220 21 34], we use the same architecture for the substitute model and will
209
analyze the impact of different architectures in the supplementary.
210
Attack dataset. We use 1.2M images without labels from the ILSVRC-2012 challenge 23 as the attack dataset. In real attack scenario, the attacker may use pictures collected from the Internet, and

---

The agreement and test accuracy (in %) of each method under 30k queries. For our model, we report the average accuracy as well as the standard deviation computed over runs. (Boldface: the best value, italics: the second best value.)
Table 1:
Method
CIEARTO
SVHN
Caltec2256
CUBS200
Agreement
Aprecmeen
KnockoffNees
15.32
74,44
$5.00
$1.50
37.61
55,28
30.01
28.03
Thiefi Entrop))
75.26
74.21
90.47
59.85
56.28
54,14
32.05
29.43
Thiefiiiif Center)
75.71
74.24
81,45
80.79
61.19
58.84
37.68
34,64
Active DFAI))FAL)
76.72
75.62
84.79
84.17
20.31
18.69
Active DEAL
74.97
73.98
$1,40
80.86
55.70
53.69
26.60
24.42
Ours-Randon
82.14:0.16
N0.47+00022
92.33=00.77
91.57=00229
02. 12.15=00.52
59.91+0.558
3N.28=+0.31
35, (5.24400499
Qursek.Cente
80.54-0.21
29.2740.15
91.47+0.09
90.68-00114
65.12+0.56
62.72+0.57
46,69+0.87
42.91+00.46
Linder
1906 10.
(a) CIFARIO
(b) SVHN
(c) Caltech256
(d) CUBS200
Figure 4: Curves of the test accuracy versus the number of queries.
the ILSVRC-2012 dataset can simulate this scenario well. Note that resize all images in the attack dataset to fit the size of the target datasets, which is similar to the existing setting [20] [1 34).
213
214
Training process. We use the SGD optimizer with momentum (of 0.9) for 200 epochs and a base learning rate of 0.02 x batchaise decayed by a factor of 0.1 every 60 epochs. The weight decay is
215
128
216
set to 5 x 10 for small datasets (CIFAR10 10 [16] and SVHN (19) and O for others. We set up query sequence {0.1K, 0.2K, 0.5K, 0.8K, 1K, 2K, 5K, 10K, 20K, 30K} as the iterative maximum query budget, and stop the sampling stage whenever reaching the budget at each iteration.
217
218
Baselines and evaluation metric. We mainly compare our method with KnockoffNets (00 and
220
ActiveTThies (21) Follow Jagielski et al. [12], we mainly report the test accuracy (Acc) as the evaluation metric. We also report the Agreement metric proposed by Pal al. [2I] which counts how often the prediction of the substitute model is the same as the victim's 's as supplement.
222
223
4.2
Experiment results
224
We first report the performance of our method compared with previous methods. After that, we conduct ablation experiments to analyze the contribution of each module. Finally, we also analyze the performance of our method when encountering defense methods and real-world online APIs. More experiments (e.g., adversarial attack and overfitting analysis) can be found in supplcmentary.
226
228
Effectiveness of our method. As in Tab. T the test accuracy and agreement of our method are all better than the previous methods. We also plot the curves of the test accuracy versus the number of queries Fig. The performance of our method consistently outperforms other methods throughout the process. Since our method does not conflict with the previous sample selection strategy, they can used simultaneously to further improve the performance of these attacks. Here, we take the k-Center algorithm as an example. Note that, with or without the sample selection strategy, our method beats the previous methods by a large margin. Particularly, the test accuracies of our method are 4.85%. 1.72%, 3.88%, and 8.27% higher than the previous best method, respectively. And the agreement metric shares similar results. It is also interesting that it is less necessary to use the k-Center algorithm datasets with a small number of classes (i.e., CIFARIO 10 and SVHN). While for the datasets with large number of classes, the k-Center algorithm can make the selected samples better cover each class and improve the effectiveness of the method.
229
230
232
234
236
238
240
Ability to evade the SOTA defense method. The SOTA perturbation- based defense method, adap- (OOD) detection module based on the
tive misinformation [3], introduces an maximum predicted value and punishes the OOD samples with a perturbed model f'(:;0". The model f( 00)) is trained with arg ming (aa,)) |- log(1 fx(;;; 0') 0'),)] to minimize the probability of
242
7

---

Table 2: Ability to evade the state-of-the-art defense method (adaptive misinformation) on CIFARIO dataset. The larger the threshold, the better the defence effect while the low victim model's accuracy (threshold 0 means no defence). Our method evades the defense best, and the self-KD part makes great difference.
Threshold
Method
0
0.5
0.7
0.9
KnockoffNets
74.44%
74.13%
73.61%
54.98%
74.24%
69.14%
59.78%
50.19%
ActiveThiie(Entropy))
74.21%
71.61%
64.84%
51.07%
Ours
80.47%
79.95%
78.25%
74.40%
Ours w/o self-KD
79.02%
78.66%
73.61%
61.81%
victim model
91.56%
91.23%
89.10%
85.14%
244 the correct class. Finally, the output will be:
V = (1 (1;)) (x))
(8)
245
where a = 1/(1 + (mmaxx (x:1))-)) with a
y/ is the coefficient to control how much correct results will be returned, and is the threshold used for OOD detection. The model returns incorrect predictions for the OOD samples without having much impact on the in-distriuution samples.
247
249
We choose four values of the threshold r to compare the effects of our method with the previous methods. The threshold value of 0 means no defence. The result is shown in Tab. Compared with other methods, adaptive misinformation is almost invalid to our method. Furthermore, we find that if remove the self-KD in method, the performance is greatly reduced. We conclude that this is because adaptive misinformation adds noise labels to the substitute model's training dataset, and self-KD KD can alleviate the overfitting of the substitute model to the training dataset, making this defence method not effective enough.
251
253
255
256 A blation study. To evaluate the contribution of different mod- ules our method, we conduct the ablation study on CUBS200 dataset and plot the results in Fig.5 If the CAM-driven erasing strategy is removed, the performance greatly reduced, showing that it has an indispensable position in our method. also give some visual examples in Fig to demonstrate that this strategy can help align the attention of two models. As depicted in the Fig. 7 the beginning time, the substitute model learns the wrong attention map. Along with the iterative training stages, the attention area of the substitute model tends fit the victim model's, which conforms to intention. We further remove the self-KD module to evaluate its performance. It can be found from Fig. and Fig. 5 that the self-KI can improve the generalization of our method and further improve the performance.
257
259
our method will be
Acc(%)
1.
261
Ours
w/a sett-KD
Ours w/o CAM- crasing
263
1888s
1tace
zies
20ee
soom
Queries
265
Ablation study on CUBS200 dataset for the contribu- tion of the CAM-driven erasing and the self-KD our method.
Figure 5:
267
269
270
271
Stealing functionality
real-world API We validate our method is applicable to real-world APIs. The AWS Marketplace is an online store that provides a variety of trained ML models for users. It can only be used in the form of black-box setting.
65
273
Acc(%) 1
275 We choose a popular model (waste classifier as the victim model. We ILSVRC-2012 dataset the attack dataset and choose another small public waste classifier dataset contain- ing 2, 527 images as the test dataset. As in Fig. 6 the substitute model obtained by our method achieves 12.63% and 7.32% improvements in test accuracy compared with previous methods, which show our method has stronger practicality in
KnockoftNats Mcclveehiefll-Ceetee
277
20
1s68e
206s
2000 2000
279
Queries
280
Figure 6: The experiment on AWS online API.
281
the real world.
Scctp:://amzn.to/3haaaa
https://github. http://:ittuu...mmaaaaaaaaa htwp:ttwww..oe/ggggtuuugnreeeeee

---

Table 3: Transferability of adversarial samples generated with PGD attack on the substitute models.
Substitute's s architecture
Method
ResNet-34
ResNet-18
ResNer-50
VGG-16
DenseNet
KnockofiNets
57.85%
63.33%
52.04%
42.88%
60.77%
ActiveThiief(--Cente
57.44%
57.90%
57.01% %
16.49%
60.72%
Active (atteeThie((Entropy))
63.56%
66.76%
58.19%
55.43%
62.05%
Ours
76.63%
74.10%
74.28%
67.03%
66.96% %
Original image
Substitute's CAM of different stages
Victim's CAM
Figure 7: The visualized attention maps of the victim model and different stages substitute models using the Grad-CAM. Along with the training stages, the attention map of the substitute model tends to fit the victim model's.
283
Transterability of adversarial samples. Though with the dominant performance on a wide range of tasks, deep neural networks are shown to be vulnerable to imperceptible perturbations, i.e., adversarial examples [27]. Since the model stealing attack can obtain a functionally similar substitute model, some previous works (e.g., JBDA 22), DaST [34] and ActiveThief (2I) used this substitute model to generate adversarial samples and then performed the transferable adversarial attack on the victim model. We argue that more similar substitute model leads to more successful adversarial attacks. We test the transferability of adversarial samples on the test set the CIFAR 10 dataset. Keeping the architecture of the victim model the ResNet-34, evaluate the attack success rate of adversarial samples generated from different substitute models (i.e., ResNet-34, ResNet- 18, ResNet-50 (9, VGG-
285
287
289
291
292
16 226), DenseNet (T). All adversarial samples are generated using Projected Gradient Descent (PGD) attack [18] with maximum Los -norm of perturbations as 8/255. As shown in Tab the adversarial samples generated our substitute models have stronger transferability in all substitute's architectures. This again proves that our method is more practical in real-world scenarios.
293
295
Conclusion
296
287
We investigated the problem of model stealing attacks under the hard-label setting and pointed out why previous methods are effective enough. We presented new method, termed black-box dissector, which contains a CAM-driven erasing strategy and RE-based self-KD module. We showed its superiority on four widely-used datasets and verified the effectiveness of our method with defense methods, weal-world APIs, and the downstream adversarial attack. Though focusing
298
300
302
on image data in this paper, our method general for other tasks as long as the CAM and similar erasing method work, e.g., synonym saliency words replacement for NLP tasks [4. We believe our
303
304
method can be easily extended other fields and inspire future researchers. Model stealing attack
poses threat to the deployed machine learning models. We hope this work will draw attention to
306
the protection of deployed models and furthermore shed more light on the attack mechanisms and prevention methods.

---

References
308
309
[1] Rohan Anil, Gabriel Pereyra, Alexandre Passos, Robert Ormandi, George E Dahl, and Geof- frey E Hinton. Large scale distributed neural network training through online distillation. arXiv preprint arXiv:1804.03235, 2018.
310
311
312
[2] Antonio Barbalau, Adrian Cosma, Radu Tudor lonescu, and Marius Popescu. Black-box ripper: Copying black-box models using generative evolutionary algorithms. In NeurIPS, 2020.
313
314
[3] Terrance DeVries and Graham W Taylor. Improved regularization of convolutional neural networks with cutout. arXiv preprint arXiv: 04552, 2017.
[4] Xinshuai Dong, Anh Tuan Luu, Rongrong Ji, and Hong Liu. Towards robustness against natural language word substitutions. In ICI.R, 2021.
317
318
[5] Melanie Ducoffe and Frederic Precioso. Adversarial active learning for deep networks: a margin based approach. In ICML, 2018.
319
320
66]
Tommaso Furlanello, Zachary Lipton, Michael Tschannen, Laurent Itti, and Anima Anandku- mar. Born again neural networks. In ICML, 2018.
321
Gregory Griffin, Alex Holub, and Pietro Perona. Caltech-256 object category dataset. 2007.
[8] Chuan Guo, Geoff Pleiss, Yu Sun, and Kilian Q Weinberger. On calibration of modern neural networks. In JCML, 2017.
324
325
[9] Kaiming He, Xiangyu Zhang, Shaoqing Ren, and Jian Sun. Deep residual learning for image recognition. In CVPR, 2016.
326
327
[10]
Geoffrey Hinton, Oriol Vinyals, and Jeff Dean. Distilling the knowledge in neural network. In NIPS Deep Learning Workshop, 2015.
328
[1]]
Gao Huang. Zhuang Liu, Laurens Van Der Maaten, and Kilian Q Weinberger. Densely connected convolutional networks. In CVPR, 2017.
330
331
[1] Matthew Jagielski, Nicholas Carlini, David Berthelot, Alex Kurakin, and Nicolas Papernot. High accuracy and high fidelity extraction of neural networks. In 29th Usenix Security 2020.
332
338
[13] Sanjay Kariyappa and Moinuddin K Qureshi. Defending against model stealing attacks with adaptive misinformation. In CVPR, 2020.
334
335
[14] Sanjay Kariyappa, Atul Prakash, and Moinuddin Qureshi. Maze: Data-free model stealing attack using zeroth-order gradient estimation. arXiv preprint arXiv:2005.03161, 2020.
336
337
[15] Kyungyul Kim, ByeongMoon Ji, Doyoung Yoon, and Sangheum Hwang. Self-knowledge distillation: A simple way for better generalization. arXiv preprini arXiv:2006. / 12000, 2020.
[16]
Alex Krizhevsky, Geoffrey Hinton, et al. Learning multiple layers of features from tiny images. 2009.
340
341
177
David D Lewis and William A Gale. A sequential algorithm for training text classifiers. In SIGIR, 1994.
342
343
[18] Aleksander Madry, Aleksandar Makelov, Ludwig Schmidt, Dimitris Tsipras, and Adrian Vladu. Towards deep learning models resistant to adversarial attacks. In ICLR, 2018.
344
345
[19] Yuval Netzer, Tao Wang, Adam Coates, Alessandro Bissacco, Bo Wu, and Andrew Y Ng. Reading digits in natural images with unsupervised feature learning. In NIPS Workshop on Deep Learning and Unsupervised Feature Learning, 2011.
346
[0
Tribhuvanesh Orekondy, Bernt Schiele, and Mario Fritz. Knockoff nets: Stealing functionality of blxck-box models. In CVPR, 2019.
349
350
[21]
Soham Pal, Yash Gupta, Aditya Shukla, Aditya Kanade, Shirish Shevade, and Vinod Ganapathy, Activethief: Model extraction using active learning and unannotated public data. In AAAI, 2020.
351
352
[22] Nicolas Papernot, Patrick McDaniel, Ian Goodfellow, Somesh Jha, Z Berkay Celik, and Anan- thram Swami. Practical black-box attacks against machine learning. In ACM AsiACCS, 2017.
353
[23] Olga Russakovsky, Jia Deng, Hao Su, Jonathan Krause, Sanjeev Satheesh, Sean Ma, Zhiheng Huang, Andrej Karpathy, Aditya Khosla, Michael Bernstein, et al. Imagenet large scale visual recognition challenge. IJCV, 2015.
355
10

---

357
[24] Ramprasaath R Selvaraju, Michael Cogswell, Abhishek Das, Ramakrishna Vedantam, Devi Parikh, and Dhruv Batra. Grad-cam: Visual explanations from deep networks via gradient-based localization. In ICCV, 2017.
358
360
[25] Ozan Sener and Silvio Savarese. Active learning for convolutional neural networks: A core-set approach. In ICLR, 2018.
[26]
Karen Simonyan and Andrew Zisserman. Very deep convolutional networks for large-scale image recognition. In ICLR, 2015.
383
384
[77]
Christian Szegedy, Wojeiech Zaremba, llya Sutskever, Joan Bruna, Dumitru Erhan, Ian Good-
385
fellow, and Rob Fergus. Intriguing properties of neural networks, In ICLR, 2014.
386
[88]
Catherine Wah, Steve Branson, Peter Welinder, Pietro Perona, and Serge Belongie. The caltech-uesd birds-200-2011 dataset. 2011.
367
[29]
Jiancheng Yang. Yangzhou Jiang, Xiaoyang Huang, Bingbing Ni, and Chenglong Zhao. Learn- ing black-box attackers with transferable priors and query feedback. NeurlPS, 2020.
369
370
[30]
Sergey Zagoruyko and Nikos Komodakis. Paying more attention attention: Improving the performance of convolutional neural networks via attention transfer. In ICLR, 2017.
371
372
[31]
Chiyuan Zhang, Samy Bengio, Moritz Hardt, Benjamin Recht, and Oriol Vinyals. Understanding deep learning requires rethinking generalization. In ICLR, 2017.
374
[32] Zhun Zhong, Liang Zheng, Guoliang Kang, Shaozi Li, and Yi Yang. Random erasing data augmentation. In AAAI, 2020.
375
378
[33] Bolei Zhou, Aditya Khosla, Agata Lapedriza, Aude Oliva, and Antonio Torralba. Learning deep features for discriminative localization. In CVPR, 2016.
377
378
[34] Mingyi Zhou, Jing Wu, Yipeng Liu, Shuaicheng Liu, and Ce Zhu, Dast: Data-free substitute training for adversarial attacks. In CVPR, 2020.
Checklist
380
1. For all authors...
382
(a) Do the main claims made in the abstract and introduction accurately reflect the paper's s
383
contributions and scope? [Yes]
384
(b) Did you describe the limitations of your work? [Yes] See section
385
(c) Did you discuss any potential negative societal impacts of your work? [Yes] See section
5
387
(d) Have you read the ethics review guidelines and ensured that your paper conforms to them? [Yes]
388
389
2. If you are including theoretical results...
390
(a) Did you state the full set of assumptions of all theoretical results? [N/A]
391
(b) Did you include complete proofs of all theoretical results? [N/A]
392
3. If you ran experiments..
(a) Did you include the code, data, and instructions needed reproduce the main ex- perimental results (either in the supplemental material or as a URL)? [Yes] See the supplemental material
394
396
(b) Did you specify all the training details (c.g.. data splits, hyperparameters, how they were chosen)? [Yes] See section 4.1
387
(c) Did you report error bars (e.g., with respect to the random seed after running experi- ments multiple times)? [Yes] See Tab.
399
400
(d) Did you include the total amount of compute and the type of resources used (e.g., type of GPUs, internal cluster, cloud provider)? [No]
402
4. If you are using existing assets (e.g., code, data, models) or curating/releasing new assets...
403
(a) If your work uses existing assets, did you cite the creators? [Yes] See section 4.1
404
(b) Did you mention the license of the assets? [No]
(c) Did you include any new assets either in the supplemental material or URL? [No]
11

---

408
(d) Did you discuss whether and how consent was obtained from people whose data you' re using/curating? [No]
407
408
(c) Did you discuss whether the data you are using/curating contains personally identifiable information or offensive content? [No]
5.
If you used crowdsourcing conducted research with human subjects...
411
(a) Did you include the full text of instructions given to participants and screenshots, applicable? [N/A]
412
418
(b) Did you describe any potential participant risks, with links to Institutional Review Board (IRB) approvals, if applicable? [N/A]
(c) Did you include the estimated hourly wage paid participants and the total amount spent on participant compensation? [N/A]
416
12


Reviews:
[{"review_id": "c1gPgjooQdk", "reviewer": "Reviewer_aBhm", "summary": "This paper proposes a new method for model stealing/knowledge distillation with hard label information only, based on a two-part modification of the standard model stealing pipeline. The standard pipeline feeds images to the target model, collects the corresponding labels, and trains a substitute model on the image-label pairs. First, the proposed method erases high-information patches from the images fed to the target model in order to collect additional (nearby) labels, providing extra information about the target model. Second, after training a first substitute model on the target images and their erased counterparts, the proposed method trains a second substitute model on both the original target data, the erased images and their labels, and \"smoothed\" predictions from the first substitute model in order to improve generalization.", "questions": "", "limitations": "", "rating": 6, "confidence": 3, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "Overall, the proposed methodology is not the simplest, but at the same time the modifications of the standard KD pipeline are creative and seem to non-negligibly improve performance on a variety of tasks. My more specific comments and questions are below:\n\n- L149: What does \"the attention map is also a kind of supervision signal pushing two models to be similar\" mean? I also did not understand how this statement follows from the cited source.\n- Does Section 3.1 mean that we actually train 3 substitute models throughout the process? One just on D_T to get the grad-cam, then another on D_T + D_E, then another on D_T + D_E and the smoothed pseudo labels?\n- I think much of the math (e.g., the first two lines of equation 4) adds very little clarity and instead makes things more confusing by introducing additional symbols and notation. Similarly with the first line of equation 5, I think there is no reason to introduce a concept as well as an acronym (MSP) if it is only going to be used once on the immediate next line.\n- The legends in Figure 4 are extremely hard to read, especially in print.\n- Based on Figure 5 and the corresponding part of the results section, it seems as though the self-KD matters much less for performance than the CAM-driven erasing (which I also thought was the more well-motivated modification of the two). To what extent could the self-KD be replaced by more \"conventional\" ways to improve generalization and prevent overfitting like Cutout regularization, more aggressive data augmentation, higher weight decay, etc?\n- The Amazon API demonstration was very interesting but lacked many experimental details---what was the setup here? What architecture was used by the proposed method vs KnockoffNets vs ActiveThief? How were parameters chosen, etc.? At face value the results are very impressive but need to be put into context with experimental details.\n\nOverall, I think the paper showcases a creative idea that significantly improves over prior work, and presents the main ideas fairly clearly (especially in light of how complicated the method is). The clarity could be improved in places and further experimental details should be provided, and some additional results following up on the ablation study would (in my opinion) further improve the paper.", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "N/A", "review_text": "", "extra_scores": {"needs_ethics_review": "No", "ethics_review_area": "", "time_spent_reviewing": "4"}, "extra_text": {}}, {"review_id": "82LTKm0lVNT", "reviewer": "Reviewer_sWAf", "summary": "This submission proposes a new model stealing attack method, black-box dissector, under the hard-label setting. Specifically, this paper uses CAM-driven erasing strategy and random erasing module to increase the information inquired from the victim model. Experiments are conducted on CIFAR-10, SVHN, Caltech256 and CUBS200 datasets and achieved superior performance compared to other attack methods. In addition, ablation studies are conducted to verify the effectiveness of the proposed attack.", "questions": "", "limitations": "", "rating": 7, "confidence": 4, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "The underlying idea that gain more informative pseudo-labels by erasing response area is very insightful. Given recent upsurge in research on model and data security, the notion that model can be efficiently reproduced from APIs that output hard-labels is an emerging concern. There is also clear differentiation between black-box dissector from prior work. The experiment results also show the effectiveness of stealing attack. The reported attacks seem to significantly outperform some competitive baselines. In addition, black-box dissector can achieve high accuracy when the distributions between training dataset of victim model and attack dataset are far apart, which is of great help to other tasks such as transfer learning and domain adaptation.\n\nRecently, there have been some new defense and detection methods against model stealing attacks [1][2]. I wonder if black-box dissector can still maintain high attack performance under these defenses.\n\nIn addition, could the authors comment on whether the label refinement [3] can improve the performance of model stealing attack if the entire attack dataset can be directly input into the victim model and get the hard-labels?\n\n[1] P. Maini, M. Yaghini, and N. Papernot, “Dataset inference: Ownership resolution in machine learning,” in International Conference on Learning Representations, 2021.\n\n[2] X. Wang, Y. Xiang, J. Gao, and J. Ding, “Information laundering for model privacy,” in International Conference on Learning Representations, 2021.\n\n[3] K. Ghasedi Dizaji, A. Herandi, C. Deng, W. Cai, and H. Huang, “Deep clustering via joint convolutional autoencoder embedding and relative entropy minimization,” in Proceedings of the IEEE international conference on computer vision, 2017, pp. 5736–5745.\n", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "The CAM-driven erasing strategy proposed in this paper is mainly addressing the problem of information acquisition when soft labels cannot be obtained. It would be interesting to compare the attack performance between using CAM-driven erasing strategy and using soft label.", "review_text": "", "extra_scores": {"needs_ethics_review": "No", "ethics_review_area": "", "time_spent_reviewing": "10"}, "extra_text": {}}, {"review_id": "4iYFtNCrLnR", "reviewer": "Reviewer_ohZm", "summary": "This paper proposes a new model stealing attack when the victim model only returns the class label to the queries instead of confidence or probability information. Prior attacks struggle in this setting to extract high-accuracy models. The proposes attack has two steps, first they query the model with a sample and probabilistically remove the input region using the GRAD-CAM of the substitute model. This aims to fit more information into a single query and simulates the prediction probabilities. Second, they use a self-knowledge distillation scheme on the substitute model to alleviate overfitting. Overall, their scheme achieves higher stolen accuracy than prior attacks, more resilient to a prior defense and allows the attacker to craft more transferable adversarial examples.", "questions": "", "limitations": "", "rating": 6, "confidence": 5, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "The paper is well-written and the intuitions behind the design is pretty clear. The setting is a relevant and a realistic setting as many of the defenses rely on obfuscating/perturbing the prediction probabilities to thwart the stealing attacks. Along these lines, I would also recommend the authors to look into Prediction Poisoning defense by Orekondy et al and evaluate the efficacy against their setting where the defense does not preserve the top-1 label.\n\nStrengths:\n+ Intuitive, well-designed attack, relevant setting.\n+ Circumvents the one of the prediction perturbation-based defenses (Adaptive Misinformation)\n\nWeaknesses:\n- My first main concern is the baseline attacks the authors have selected. Both ActiveThief and KnockoffNets are active learning-based and they do not modify the query samples themselves, they only select them from a larger set. The proposed attack on the other hand actively modifies the samples. I believe a more suitable baselines would be something like Prada attack that modifies the queries with perturbations to extract more information from them.\n\n- My second main concern is the ablation study itself. The ablation in the paper completely disables one or the other step of the attack and measure the attack success. A better ablation study would replace the steps with something trivial rather than removing altogether to see whether we need a rather complicated design to begin with. For the GRAD-CAM step, experimenting with random data augmentations (for example CutOut) and for the second step using something like label smoothing to prevent overfitting to potentially noisy labels would be more informative to understand the usefulness of the design.\n\n- I also recommend the authors to experiment with the Prediction Poisoning defense, obviously in preserving top-1 label setting, the defense would be ineffective against the proposed attack (which is a good thing), but in other settings the attack's success might be reduced.\n\n- It is also useful to present the attack results that use the full prediction probabilities to give a sense of the upper-bound on how good a label-only attack can get. \n\n\nQuestions:\n\n? Do you start from a pre-trained models before launching the model stealing attacks? This seems to be a standard practice in the prior work but I wasn't sure if it's the case here too.\n\n? If the Adaptive Misinformation defense's OOD detector was trained on the samples generated by your attack, how would your attack fare?\n\n==============\nAFTER REBUTTAL\n==============\nThanks for the extra experiments and the discussion. I'm bumping up my score, please include the detailed ablation study in the paper. I think it strengthens your story much more.", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "(See above)\n", "review_text": "", "extra_scores": {"needs_ethics_review": "Yes", "ethics_review_area": ["Privacy and Security (e.g., consent)"], "time_spent_reviewing": "3"}, "extra_text": {"ethical_concerns": "As far I see, the authors actually steal a commercially available ML model (waste classifier on AWS Marketplace). Since this is done for research, I see no harm but attacking real-world systems also have ethical and potentially legal problems. Did the authors inform the owners of this model (https://appen.com/) before conducting this attack on their system?"}}, {"review_id": "1ALQmL23AM", "reviewer": "Reviewer_WyuJ", "summary": "This work presents a more effective model stealing attack using an erasing-based method which makes use of saliency maps.  The method is clever and intuitive.", "questions": "", "limitations": "", "rating": 6, "confidence": 4, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "Thank you for the interesting work.  This paper seems to make appreciable progress on an important problem.  Below are several minor comments:\n\n1) In general, the writing and grammar could be improved for the sake of clarity.  This is a major area for potential improvement in this paper.\n\n2) Figure 2 is a little hard to parse.  The meaning of some of the arrows is unclear.\n\n3)  Your victim models perform quite a bit worse than I would expect on the tasks given.  I believe hyperparameter tuning could significantly improve their performance, and maybe you could also significantly improve the performance of your attack as well with better hyperparameter tuning.\n\n4)  It seems like the self-kd is not very important for the performance of your method.  I think you should have better justification for this component of your method.\n", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "The authors do adequately address the limitations and impact of their work.", "review_text": "", "extra_scores": {"needs_ethics_review": "No", "ethics_review_area": "", "time_spent_reviewing": "3"}, "extra_text": {}}]

Author rebuttal:

Meta-review: The paper studies model stealing attack in the hard-label black-box setting, where the victim model only returns the top-1 label for each query. To overcome this challenge and acquire more information, the paper designs a novel erasing-based scheme to improve the performance of model stealing attack. 

This is a borderline case where the paper has pros and cons. On the positive side, the paper is proposing a novel idea that improves black-box model stealing attack by an erasing-based technique, and all the reviewers like the novelty of the paper. However, the proposed method is based on heuristics and lack fundamental theoretical justifications. In particular, it is not clear how the proposed method can overcome the challenges when Grad-CAM gives wrong or noisy input region, especially in the initial training stage. Further, the method is quite complicated (pointed out by reviewer ohZm) and there are some presentation issues (pointed out by reviewer WyuJ). After deeply looking into the paper, AC and SAC decide the reject the paper due to lack of analysis and the presentation issues. We encourage the authors to address these issues and resubmit to another top conference. 
Decision: Reject