Dual Student Networks for Data-Free Model Stealing
Authors: James Beetham
Navid Kardan
Ajmal Saeed Mian
Mubarak Shah
Venue: ICLR 2023 poster (2023)

Abstract: Data-free model stealing aims to replicate a target model without direct access to either the training data or the target model. To accomplish this, existing methods use a generator to produce samples in order to train a student model to match the target model outputs. To this end, the two main challenges are estimating gradients of the target model without access to its parameters, and generating a diverse set of training samples that thoroughly explores the input space. We propose a Dual Student method where two students are symmetrically trained in order to provide the generator a criterion to generate samples that the two students disagree on. On one hand, disagreement on a sample implies at least one student has classified the sample incorrectly when compared to the target model. This incentive towards disagreement implicitly encourages the generator to explore more diverse regions of the input space. On the other hand, our method utilizes gradients of student models to indirectly estimate gradients of the target model. We show that this novel training objective for the generator network is equivalent to optimizing a lower bound on the generator's loss if we had access to the target model gradients. In other words, our method alters the standard data-free model stealing paradigm by substituting the target model with a separate student model, thereby creating a lower bound which can be directly optimized without additional target model queries or separate synthetic datasets. We show that our new optimization framework provides more accurate gradient estimation of the target model and better accuracies on benchmark classification datasets. Additionally, our approach balances improved query efficiency with training computation cost. Finally, we demonstrate that our method serves as a better proxy model for transfer-based adversarial attacks than existing data-free model stealing methods.

Published as a conference paper at ICLR 2023
DUAL STUDENT NETWORKS FOR DATA-FREE MODEL STEALING
James Beetham '. Navid Kardan' Ajmal Mian", Mubarak Shah'
Center for Research in Computer Vision
DDeparteett of Computer Science University of Western Australia Crawley WA 6009, Australia { ames beetham, kardan) (rrddan)xxiighttte ucf edu momal.mian@uwee wwwaannuwww........m edu. au shah@crov. ucf. edu
University of Central Florida
Orlando, Florida 32816, USA
ABSTRACT
Data-free model stealing aims to replicate a target model without direct access to either the training data the target model. To accomplish this, existing methods use a generator to produce samples in order to train a student model to match the target model outputs. To this end, the two main challenges estimating gradients the target model without access to its parameters, and generating a diverse set of training samples that thoroughly explores the input space. We propose Dual Student method where two students are symmetrically trained in order to provide the generator a criterion generate samples that the students disagree on. On one hand, disagreement on a sample implies at least one student has classi- fied the sample incorrectly when compared the target model. This incentive towards disagreement implicitly encourages the generator to explore more diverse regions of the input space. On the other hand, our method utilizes gradients of student models to indirectly estimate gradients of the target model. We show that this novel training objective for the generator network equivalent to optimizing lower bound on the generator's ss loss if we had access to the target model gra- dients. In other words, our method alters the standard data-free model stealing paradigm by substituting the target model with separate student model, thereby creating a lower bound which can be directly optimized without additional target model queries separate synthetic datasets. We show that new optimization framework provides more accurate gradient estimation the target model and bet- ter accuracies on benchmark classification datasets. Additionally, our approach balances improved query efficiency with training computation cost. Finally, we demonstrate that our method serves as better proxy model for transfer-based adversarial attacks than existing data-free model stealing methods.
INTRODUCTION
Model stealing has been shown to be serious vulnerability in current machine learning models. Machine learning models are increasingly being deployed in products where the model's output accessible through APIs, also known as Machine Learning as a Service. Companies put a large amount effort into training these models through the collection and annotation of large amounts of data. However, recent work has shown that the ability to query model and get its output without access to the target model's 's weights enables adversaries to utilize different model stealing approaches, where the attacker
train student model have similar functionality to the target model (Kesarwani et al.) 2018 Yu et al. 2020 Yuan et al.) 2022 Truong et al. 2021 Sanyal et al. 2022). Two major motivations for stealing a private model are utilizing the stolen model for down- stream adversarial attacks as well as monetary gains, therefore, model stealing methods present an
increasing problem (Tramer al. 2016 Zhang al. 2022).
"Equal contribution

---

Published as a conference paper at ICLR 2023
Data-free model stealing is generalization of black-box model stealing which is an extension of knowledge distillation. In all three areas, the aim is to obtain a student model which imitates the target model. However, while knowledge distillation environments typically retain full knowledge of the target model training data and weights, black-box model stealing eliminates the need to have access to the target model weights and training data. Going one step further, black-hox model steal- ing typically uses real-world data samples to train the student network, whereas data-free model stealing removes that requirement by leveraging generator. As the relationship between the substi- tute dataset and true dataset is typically unknown, the removal of the substitute data results in more generalizable approaches.
Existing methods for data-free model stealing involve either using min-max adversarial approach (Truong et al. 202T Kariyappa et al. 2021 Zhang et al. 2022), or a eonertor-diserimmnate setup incorporating Generative Adversarial Networks (GANs) (Sanyal et al. 2022). DFME (Trong et al., 202T] and DFMS-HL. (Sanyal et al. 2022) are the two most recent and highest performing methods for each of these approaches, respectively. In both approaches, a student is optimized to minimize the distance between the student and target model outputs. However, in the former, a generator is optimized to maximize the distance between student and target model outputs, while in the latter, the mmmerator-discrinnatte is optimized to generate samples similar to a synthesized dataset which have balanced student classifications. Even though utilizing a synthesized dataset relaxes the truly data-free requirement, the current state of the art (SOTA) is provided by the DFMS-HI method which does utilize synthetic data, while DFME method provided the previous SOTA and doesn't t require synthetic data.
Our proposed Dual Student method alters the generator-studet min-max framework by training two student models to match the target model instead of one. This allows the generator to use one of the students as a proxy for the target model, resulting in a much more effective training criterion. The two students are symmetrically trained to match the target model. while employing the new generator objective to generate samples on which the two students disagree. Disagreement between the two students implies disagreement between at least one of the students and the target model, resulting in the generated sample being hard sample (c.f. hard sample mining (Shrivastava et al. 2016) to at least one student. This hard sample provides the driving force to explore controversial regions in the input space on which the students and teacher disagree on. Finding samples on which the student and teacher disagree is also an implicit goal of DFME. However, while DFME does optimize the generator to find disagreement between the student and target, it uses gradient estimations of the target. Our method, on the other hand, uses the true gradients of the second student which can be obtained using direct backpropagation.
Using two student models estimates than data-free model stealing methods like DFME, which use explicit gradient estimates. Formalizing our setup, substituting second student model for the target model creates an alternative lower bound for the student-teacher optimization. Changing the training objective of the generator- student framework to more directly optimize to match the target model results in student model gradients being better aligned with the target model gradients. In our experiments, we show the gradients used when training dual students is closer to using the true gradients of the target model than the estimation method used by DFME (Truong et al. 2021). This important change eliminates the need of some existing approaches (Truong et al. 202 Kariyappa et al., (02T) to estimate the gradient through the target model. Gradient estimation techniques used in these existing works assume the target model provides soft-label outputs removing the gradient estimation removes this dependence soft-labels and allows existing methods to be easily extended to the more difficult hard-label setting. Removing the gradient estimation also reduces the number of queries made to the target model. Finally, we also explore more difficult data-free setting where the number of classes is not know in advance. To the best of our knowledge, we are the first to address this setting and provide a solution to this new challenging data-free free setup.
proxy for the target model results in better target model gradient
In summary, our contributions in this paper are as follows: (1) We propose the Dual Student method which provides a tractable approach for the data-free model extraction problem with minimal added components. This includes mathematical reframing of the data- free model stealing objective along with an empirical comparison with the previous formulation provided by DFME (Truong et al., 2021). (2) The Dual Student method can be incorporated into existing soft-label approaches to extend them to hard-label settings, and has better classification accuracy than existing methods. (3) We show the effectiveness of utilizing the Dual Student setup to fine-tune fully trained models by
2

---

Published as a conference paper at ICLR 2023
directly improving the existing SOTA from DFMS-HI HHL and evaluate the effectiveness of the top data-free model stealing approaches for transfer-but based adversarial attacks against the target model.
2 RELATED WORK
In this section, we outline the foundational papers our method is built upon, as well as other recent works in this area. We start with knowledge distillation distilling the knowledge out of a target model and into student. Then describe the transition towards model extraction and model stealing, and its extension into the data-free model domain. Finally we discuss the use of Kata-free model stealing for transfer-bsses adversarial attacks.
2.1 KNOWLEDGE DISTILLATION
Knowledge distillation is the process of using transfer learning train different, typically smaller, model. From the area of model compression (Buciluă et al. (0066] Romero et al.) 2014), knowledge distillation was initially used to determine the necessary depth of deep learning models in [a & Caruana (2014). The objective was then expanded towards the general goal of creating smaller and more efficient models (Hinton al., 2015 Urban al., 2016 Yin et.l. 2020). Our approach to data- free model extraction builds off of knowledge distillation techniques for training student model from a target model, otherwise referred to the "teacher" or "victim" model in other methods.
Model extraction, also called model stealing, is used to obtain a new model which performs similarly to the target model without knowledge of the target model's s gradients. First uses of model extrac- tion were in extracting decision trees to help better explain deep learning models (Bastani et al.) 2017b a), however, recent applications have focused on obtaining a substitute model for the purpose of transfer-basee attacks the target model (Wang et al. 2021). l). Though early knowledge distilla- tion works relied on access to the original training data, more recent data-free knowledge distillation methods utilize a generator to remove this dependence (Fang et al., [019 Binici et al.) (022). Re-
cent works utilize activation regularization between the target and student models, in addition to interpolation between generated examples improve training (Qu et al. 2021).
2.2 DATA-FREE MODEL EXTRACTION
Data-free model extraction aims train a student model without being given a dataset prior, Most existing model extraction techniques rely on either some small set of labeled samples, or an alterna- tive training dataset (Orekondy al., 2019 Barbalau et al., [020 Wang] 202T). The first solutions to this data-free model extraction problem are called DF ME and MAZE, and use adversarial setup between generator and student (Truong et al.] 2021 Kariyappa et al., 202T). The student aims to match the target model and the generator aims to find examples which result in different classifications between the student and target models. However, these methods rely on estimating the gradient of the target model, so more recent work has focused removing this dependence by altering the generator objective (Hong et al. 2022] Sanyal et al., (022). The current SOTA for data-free model stealing is DFMS-HL by Sanyal et al. (2022), which only sets the bar for student model classification accuracy, it also extends the problem to the hard-label setting. Their approach utilizes full generator-dise pngerattorddscciminatoo GAN setup generate samples similar synthetic dataset while retaining balanced class classifications of the samples by the student model. These two aspects -eeeping samples similar the synthetic dataset, and balancing the generated class distribution remove the reliance on gradient estimation techniques that methods such as DFME require. We note that DFMS-HL utilizes a synthetic dataset of random shapes generated on colored backgrounds, which breaks the truly data-free setting.
2.3 TRANSFER-BASEN ATTACKS
Adversarial attacks (Szegedy et al. 2013 Akhtar & Mian 2018 Kardan & Stanley 2017 Kardan
et al. 2021; Beetham et al. 2022) pose a significant obstacle against deploying deep models. In particular, mansfer-based adversarial attacks Akhtar et al. (2021) use substitute model's gradients to generate an adversarial image to fool the target model. This setting was explored extensively by early methods, however, recent work has moved towards more realistic data-free setting (Zhang
3

---

Published as a conference paper at ICLR 2023
Gradients
T
4s (1)),,
Generator
Student
c (x))
None
G
S
Es
Sx(x)
S2(x)
x
2 ((0))
S₂
T(x)
Our network consists of a generator, two students, and target model. The generator generates images which the two students are trained to classify. The student losses the Dual Student setup aim reduce the distance between the target and student model outputs, and only require the soft-label or hard-label outputs of the target model. The generator loss competes against the student losses by maximizing the distance between the target and student model outputs.
Figure 1:
et al. 2021). A prominent use-case for model stealing attacks is to use the student model proxy
for transfer-based adversarial attacks (Hong et al.) 2022 Li & Merkel 2021: Yue et al. 2021). The method proposed in Zhang al. 2022) leverages data-free model stealing work to use fewer queries to train substitute model for wransferbbsed adversarial attacks. However, we not include this work in transfer- based attack comparison, that work is targeted towards transfer-basee adver- sarial attacks as opposed to our primary objective of mimicking the performance of a classification model.
DUAL STUDENTS METHOD
This section outlines the framework of the Dual Students method. We first explain why our proposed optimization is better, and then provide an empirical analysis of our algorithm. The Dual Students method consists of generator, two students, and target model with limited black-box access. Similar to the adversarial objective GANs where the generator tries to fool the discriminator, the generator in the Dual Students method generates images on which the two students should have disagreeing outputs. A mismatch between the outputs of the two students on a generated image implies that at least one of the student outputs does not match the teacher output for this image. This beneficial for optimizing the students to match the target model, especially in the hard- label setting where hard examples objectives generator promoting disagreement between the students and the students aiming for agreement with the teacher form the min-max framework of our setup, which facilitates the exact computation of the gradients for all losses.
crucial for correcting the student models. These differing
Architecture of the proposed method is show in Figure I where key contribution is that our gen- erator optimization no longer requires the estimated gradient of the target model. In our method, the student model gradients are not used as a direct proxy for the target model gradients. De- spite this, when compared with ideal gradients from a student model trained in white-box setting, our method results in student-mode gradients closer to the white-box setting than the single-step differrddifferences method.
3.1 DATA-FREE MODEL STEALING FRAMEWORK
For target T, generator G, student S, and noise z, our broad objective for solving data-free model stealing is to optimize:
[T)()))) ([T(()
min max max G
S(G()))llp-
(I)
The original Data-Free Model Extraction framework aimed to extract the target model by utilizing a generator G with parameters 6g to generate images z from noise z ~ 2Q(0, 1Jd), where the dimension of the latent space, in which the student. S with parameters 0g is trained to match the
4

---

Published as a conference paper at ICLR 2023
output of target model T. The loss function for the student is:
min Ls C T(G(::))) 0g
where student loss Ls is typically Ls = DEL or Ls L1 We choose to use the latter when applicable, as it has been shown to perform better than Dx (Truong et al.) 2021). The generator, on the other hand, is optimized to maximize the difference between student and teacher. To this end, the negative of the student loss is typically used:
min (ann-Lo(SSGEE (s)) (TGc) 0g
However, whereas the student loss only requires the gradient through the student model, the genera- tor loss requires the gradient through the student, target, and generator. Due to the limited black-bor access to the target model, existing approaches utilize the forward differences method to estimate the gradient through the target model (Truong et al. (202). An additional problem is that the generator loss does not directly promote a diversity of classes within the generated images. Though this has not been a problem for 10-class datasets like SVHN and CIFARIO 10, note in Section 4.2 that diversity of classes becomes larger challenge when using datasets with more classes like CIFAR100.
3.2 APPROXIMATING GRADIENT WITH DUAL STUDENTS
Towards making the broad DFMS objective outlined in Eq. D differentiable, we propose adding an additional student S to solve the following optimization problem:
min max [1](()) (x))ll + (152(x) T(p) St,22 G
for any tp norm loss. Though, neither the original objective in Eq. or this new objective are differentiable due requiring the gradients through the Target model T, adding the second student S2 allows us to make a substitution to use the generator optimization:
max [)(()) S2(((=))l G
This optimization for Generator G removes the Target model T and makes the problem directly differentiable. This new generator optimization is related to our new objective in Eq. throough the triangle inequality,
[[] - -2 < (1) T(x) + [1)((z)) - (a))lpp-
(4)
The LHS of the inequality in Eq. 4 corresponds to the generator's loss in Dual Student with x G(z) in Eq. 3 and the RHS is the minimization desired in Eq. [ with additional student. In other words, when the LHS is maximized during generator optimization, the lower bound is increasing:
max (x) (2(x)), I max [|S] T(=)ll + (z) Tx
This ensures that either S (1r) S2(x) is getting farther away from T(x) during the generator optimization. To optimize min-max optimization in Eq. 3 we iteratively update the student and generator networks. In the first step, Student St with parameters Os, is optimized to match the output of the target model for a particular query:
min Ls (GISSS((:===))))007700003.. 89:
Next, the generator is updated according to:
max (axx_cc(((((.::.))))))...(( PG
(5)
The student loss used for soft-labels is Ls = éT. whereas the student loss used for hard-labels is cross entropy, i.e. Ls = ce, as this is standard for classification tasks. The generator optimization maximizes the difference between the outputs of the two students, and LG & is used in the generator loss when training on both soft-labels and hard-labels. We provide an algorithm for our method in the supplemental materials.
5

---

Published as a conference paper at ICLR 2023
.DS on Girnented Images
DFME on Geneested Images
DS on Genarited Imagees
DFME on Genarsted Tmague DFME on Real Images
ISS Real Imagee
DFME Real Images
DSS on Real Images
2000
21000
Loss Graffeet Eirvr
2000
CE Craalemtt
1900
1800
2000
1700
1600
800
1500
150
Epoch
Egweh
(a)
(b)
The distance between the true and estimated target model gradients either Generated images or Real images. The gradient is computed w.r.. input image z, and formal description of the distance used is provided in Eq. [ (a) Cross-Entrope loss is used to compute the gradient in the hard-label setting, and (b) l loss is used in the soft-label setting. Real images are from the test split of the CIFARIO 10 dataset.
Figure 2:
A major benefit of our approach is the simplicity of requiring only an additional student to overcome the non-differentiabiliiy of the black-box teacher. Our method is fully differentiable, contrast to methods like DFME which approximate teacher gradients using forward differences. Additionally, the Dual Students approach is compatible with both soft-label and hard-label learning through simple change to the loss function. Existing gradient-estimation based methods like DFME can only be used in the soft-label setting due to the forward differences method needing the additional information provided by the soft-label to remain effective. For other fully trained DFMS, our method can be used to fine-tune the trained models as long as there is a student and generator. In the next subsection, evaluate the accuracy of the different gradient approximation methods compared to our Dual Students method.
3.3 EVALUATING THE GRADIENT APPROXIMATION
The aim for the generator optimization under the black-box scenario proposed in Eq. 5 is to get as close possible to the generator optimization under the white-box scenario outlined in Eq. One way to evaluate the quality of the estimated gradients between the forward differences method and the Dual Students method to compare the gradients computed. Specifically, the gradients from La w.r.t. the generated image r when using the black-box estimate, versus the gradients computed when using the white-box Target model directly. By comparing difference between the normalized gradients, the difference can be shown as:
f(z) V 8s))( (s))
(z. 0g.)) [I S2(2; 0s,))
(6)
The results of the forward differences method compared with our Dual Students using this setup is shown in Figure The figure plots the difference between the normalized gradient of the generated image using the target model (white-box gradient) versus the estimate (forward differences or Dual Students). Figure shows that our method provides gradient estimate closer the white-box set- ting than the forward differences method used in DFME in the hard-label setting for both generated and real images. Figure 2 shows error for the soft-label setting, where we see DFME has similar estimation error between real and generated images, while Dual Students has smaller error for both between real and generated images. We note that the performance on generated images are more important generated images are the only images the models have access to during training.
Empirically, we see that this new generator loss results more accurate gradient estimations for the target model than the forward differences method, as is shown in Figure 2 We note that the forward differences method utilizes soft-labels, and performance much worse when the soft-label information reduced to hard-labels.

---

Published as a conference paper at ICLR 2023
Table 1: Percent student accuracy for different methods with and without the Dual Student (DS) method.
Dataset
Target Accuracy
Method
Probabilities
Hard-Labels
99.66
DFME
99.15
97.85
MNIST
99.66
DS
99.36
99.25
93.84
DFME
85.17
48.91
MNSST
93.84
DS
91.17
80.53
97.21
DFME
96.35
85.69
GTSRB
97.21
DS
96.40
93.20
96.20
DFME
95.33
93.87
SVHN
96.20
DS
95.72
95.43
95.5
DFME
88.10
68.40
95.5
DS
91.34
78.72
CIFARIO
95.5
DFMS-SI/HI.
88.51
79.61
95.5
DFMS-SL/HL + DS
89.38
85.06
77.99
DFME
26.46
6.91
77.99
DS
45.32
9.77
CIFARI00
77.99
DFMS-SL/HL
44.86
35.78
77.99
DFMS-SL/HL + DS
50.98
36.38
4 EXPERIMENTS
We evaluate the Dual Students method on various datasets for standard classification accuracy and transfer-based attack effectiveness. The datasets used are MNIST, FashionMNIST. GTSRB, SVHN, CIFAR1O, and CIFARI00 (Xiao et al., [2177; Stallkamp al. 2011 Notzereetze Netzer et al. 2011 Krizhevsky et al., 2009). The target model architecture is ResNet-34, while the students are ResNet- 18 and are trained for 2 million queries for MNIST, FNShiOnMNIST, GTSRB, and SVHN and 20 million queries for CIFAR10 and CIFAR100 (He et al. 2016). The generator architecture is the same 3- layer convolutional model used in DFME. For the classification task an average of the two student's outputs are used in ensemble. However, for transfer-based attacks, only the higher accuracy student used as proxy order to help make the performance more comparable across methods. More experiment details provided in the supplemental material.
4.1 CLASSIFICATION PERFORMANCE
The standard evaluation setting for data-free model extraction is test the student model accuracy on the test set respective to the dataset the target model was trained on. Table provides a com- parison between DFME (Truong et al., 202T). DFMS-HL (Sanyal et al. 2022). our proposed Dual Students method, and when our Dual Students method is used with DFMS-HL. As can be seen, method outperforms DFME and DFMS- HLL on the different datasets using soft-labels (probabilities), with the most notable increase being FashionMNIST.. On hard-labels, method outperforms DFMS-HL. when used to fine-tune the DFMS-HL. trained student.
methods is an effective way to increase accuracy. Specifically, we take the fully trained student from one method and insert it as pretrained method into our Dual Students setup. At the start of fine-tuning using Dual Students, the generator along with one of the students is uninitialized, while the remaining student uses the weights trained by the original method. After fine-tuning, only the fine-tuned student used during evaluation. This fine-tuning is especially beneficial for methods that have different optimization scheme such as DFMS-HL HI. From high level view, the DFMS-HL method optimizes the generator to generate images that are similar to the proxy data and that have a balanced distribution of classes, when classified by the student model. Due to the student model being optimized to match the target model outputs, the generator is indirectly optimized to generate a balanced distributions of classes as classified by the target model. There is no loss in the setup which directly incentivizes samples on which the student and teacher disagree. Thus when method is used fine-tune the fully trained DFMS-FR HL models for million queries (5% of total), we see an improvement of over 5% in Table We note that the DFMS-SI/HH results shown trained using our target model in order to better
Using Dual Students to fine-tune fully trained seneraatr-studeet--eee

---

Published as a conference paper at ICLR 2023
DEME
Target
Proxy
Dual Students
DFME
DFMS-HI
Dual Stodeats
100
Perceat Class Gevorated
20
75
Accuracy
50
10
25
claxs-0
claxs-]
class-2
class
claxs-
class-
class-6
claxs-7
cless-
class-)
Epsilon
Figure 4: Accuracy on Target Model of perturbed im- ages generated using PGD attack with varying epsilons (255) where different soft-label DFMS student models used.
Figure 3: Class distribution of gener- ated classes at the end of training for CI- FARI0 dataset. Dual Student is able produce more balanced samples.
compare with other methods, and don't fully match the accuracies reported in |Snnyal et al. (2022). They achieve 85.92% accuracy hard- compared with the 79.61 61% accuracy found. When their model is fine-tuned using Dual Students, we observe an increased accuracy of 88.46%.
4.2 QUERY EFFICIENCY AND NUMBER OF CLASSES
The Dual Student generator loss avoids the extra queries necessary for zeroth order gradient es- timation when optimizing the Generator. Using zeroth order gradient estimation adds additional queries to the target model during the generator training. The DFME method has the same al- ternating generator-studert
training and uses the same parameters generator and student iterations. However, due to their use of the forward differences method when updating the generator, each gen- erator iteration requires an additional queries the target model. This results in our method using ~ 71.42% of the queries that DFME does.
When compared with the DFMS-HL our Dual Student method benefits from simpler training setup and requires less compute. Though both methods have the same number of separate models, DFMS-HL. requires the generator be pretrained on the synthetic data, whereas our method can be trained from scratch using alternating generator-student training loops.
DFMS faces challenges when scaling to datasets with larger numbers of classes. Intuitively, datasets with more classes tend to be more difficult than datasets with fewer classes, and this is what found when extending our method to CIFAR 100. With no changes to our setup, DFME achieves around 26.46% accuracy reason for this large reduction in accuracy, when going from CIFARIO to CIFAR 100, is the lack of explicit diversity within the generator loss optimization T. that define. Though the generator is optimized to find confusing images for the two students, there is no guarantee that those confusing images are evenly distributed across classes. Despite this, Figure 3 shows that the generator for the Dual Students method generates a more balanced set of classes than the DFME generator on CIFARIO, where perfectly balanced classes would be 10% prevalence for all classes. We provide additional figures and analysis of generated image class distributions in our supplementary material.
CIFAR100, while Dual Students method achieves around 45.32%.
natural extension the data-free black-box hard-label environment that DFMS works in to remove the knowledge about the number of classes in the target dataset. This raises the challenge of how to learn about the number of classes, and how to alter the classifier to grow as the number of classes grows. To extend our method and address these challenges, the seen classes are mapped in the order they are observed, and student classifier training limited to the number of seen classes. As additional classes are discovered naturally by the generator traversing the image-space, the final classification layer of the student model expands to include these additional classes as separate outputs. Training CIFAR10 using hard-labele without the number of classes known, method achieves 74.95% accuracy, which is comparable to the hard-label accuracy when the number of classes are known.
8

---

Published as a conference paper at ICLR 2023
Table 2: Attack success percentage of different DFMS methods on the Target Model trained on CIFARIO when attack
255 All attacks are evaluated on the Target Model. The Target Model Black-box attack where the proxy
3
row white-box attack, Proxy Model is a transfer-based trained using the same data as the target model. The other DFMS methods provide trained student models which act as the proxy transfer-based black-box attacks.
Untargeted Attacks
Targeted Attacks
Attack
Method
Probabilities
Hard-Labels
Probabilities
Hard-Labels
Turget Model
45.00
19.64
Proxy Model
33.12
14.38
DFME
56.84
39.22
21.07
17.15
FGSM
DS
62.35
44.58
21.58
21.04
54.88
48.89
19.74
21.85
DFMS-HL/SE + DS
54.99
50.41
20.53
23.59
Target Model
96.78
76.32
Proxy Model
55.01
28.33
DFME
83.59
54.71
51.97
31.49
PGD
DS
91.04
62.21
61.96
33.39
DFMS-HL/SL
81.97
72.40
52.64
39.95
DFMS-HL/SL. + DS
81.04
73.08
51.38
42.53
4.3 BLACK-BOX ATTACKS
The effectiveness of using the trained student model as substitute for the target model in black- box transfer-based adversarial attack can seen Table The DaST method (Zhou et al. 2020) and the work done by Zhang et al. (2022) also have similar generator-student setup, however the focus those works are primarily on being used as a transfer-based proxy model. Our work, on the other hand, focuses the classification accuracy of CIFARIO. In the fooling rates in Table 2 can see that our method has a much higher fooling rate for the different attacks. We chose to use small epsilon of 255 instead of the standard 255 For higher epsilons the attack success rate becomes 100%, as can be seen in Figure
to highlight the difference in effectiveness.
Two observations of note in Table M are that the data-free methods have higher attack success rate than the proxy model trained on the real dataset, and that the data-free methods outperform the white-box attack when using FGSM (Goodfellow al., 2015). For the former, might seem counter intuitive that the student models significantly outperform the regularly trained proxy model. However, this result is expected, as the millions of queries to the target model used to train the student model result in the students having gradients more similar to the target model, For the latter, the DFMS methods having a higher fooling rate than the white-box setting surprising. One explanation for this is that the FGSM attack is more susceptible noise within the attacked-model's gradients due to only taking single step the direction of signed gradient. The white-box model is larger model trained the real dataset, and thus has higher tendency for noise within its gradients as compared with the student models. The student models, on the other hand, are trained on images designed be near the class boundaries, as, if there is noise within the student model gradients, the generator will leverage that noise to cause disagreement between the students. PGD takes multiple smaller steps and is thus less susceptible the noise which explains why the white-box setting is more optimal that attack (Madry et al. 2017).
5 CONCLUSION
In summary, outlined new Data-Free Model Stealing optimization cultivated in the Dual Stu- dents method. We showed the Dual Students method is effective in stealing the functionality of the target model as well providing good proxy from which to generate transfer- based adversarial attacks. Towards improving model stcaling methods, reducing the number queries required and extending complex datasets and tasks are major challenges to be explored in future work.

---

Published as a conference paper at ICLR 2023
Ethics Statement: Advancing the model stealing area has the potential to improve the effectiveness of attacks from adversaries with ill intentions. However, for the data-free area specifically, this is not an immediate concemn due to current DFMS methods being very limited when it comes to scaling to more useful or valuable problems. And, as with many areas of machine learning research, is not clear that work in this area will not bring about benefits towards knowledge distillation and machine learning as a whole which may outweigh potentially exacerbating the model stealing threat.
Reproducibility Statement: We outline the general idea of method and provide the key hyper- parameters used during training. In the supplemental material we provide the algorithm as well as additional hyperparameters necessary for replicating the results shown in the tables. Because the dual students method does not require pretraining, other synthetic datasets, or models that require separate training routines, the environment can be setup and trained from scratch without using proprietary datasets or environments.
ACKNOWLEDGEMENT
We thank Dr. Mitch Hill for insightful discussions and feedback. This material is partially based upon work supported by the Defense Advanced Research Projects Agency (DARPA) under Agree- ment No. HR001 12090137, and is approved for public release; distribution is unlimited. Professor Ajmal Mian is the recipient of an Australian Research Council Future Fellowship Award (project number FT210100268) funded by the Australian Government.
REFERENCES
Naveed Akhtar and Ajmal Mian. Threat of adversarial attacks on deep learning in computer vision: A survey. leee Access, 6:14410-14430, 2018.
Naveed Akhtar, Ajmal Mian, Navid Kardan, and Mubarak Shah. Advances in adversarial attacks and defenses in computer vision: A survey. IEEE Access, 9:155161-155196 2021.
Jimmy Ba and Rich Caruana. Do deep nets really need to be deep? In Z. Ghahramani, M. Welling, C. Cortes, N. Lawrence, and K. Q. Weinberger (eds.), Advances in Neural Information Processing Systems, volume 27. Curran Associates, Inc., 2014.
Antonio Barbalau, Adrian Cosma, Radu Tudor Ionescu, and Marius Popescu. Black-box ripper: Copying black-box models using generative evolutionary algorithms. Advances in Neural Infor- mation Processing Systems, 33:20120-20129, 2020.
Osbert Bastani, Carolyn Kim, and Hamsa Bastani. Interpretability via model extraction. arXiv preprint arXiv:1706.0977 09773, 2017a.
Osbert Bastani, Carolyn Kim, and Hamsa Bastani. Interpreting blackbox models via model extrac- tion. arXiv preprint
2017b.
James Beetham, Navid Kardan, Ajmal Mian, and Mubarak Shah. Detecting compromised archi- tecture/weights of deep model. In 2022 26th International Conference on Pattern Recognition (ICPR), pp. 2843-2849. IEEE, 2022.
Kuluhan Binici, Shivam Aggarwal, Nam Trung Pham, Karianto Leman, and Tulika Mitra. Robust and resource-efficient data-free knowledge distillation by generative pseudo replay. arXiv preprint arXiv:2201.030/ 2022.
Cristian Buciluă, Rich Caruana, and Alexandru Niculescu-Mizil. Model compression. In Pro- ceedings of the 12th ACM SIGKDD international conference Knowledge discovery and data mining, pp. 535-541, 2006.
Gongfan Fang, Jie Song, Chengchao Shen, Xinchao Wang, Da Chen, and Mingli Song. Data-free adversarial distillation. arXiv preprint arXiv: 1912. 1006, 2019.
Ian J. Goodfellow, Jonathon Shlens, and Christian Szegedy. Explaining and harnessing adversarial examples. In International Conference on Learning Representations, 2015.
10

---

Published as a conference paper at ICLR 2023
Kaiming He, Xiangyu Zhang. Shaoqing Ren, and Jian Sun. Identity mappings in deep residual networks. In European conference on computer vision, pp. 630-645, Springer, 2016.
Geoffrey Hinton, Oriol Vinyals, Jeff Dean, et al. Distilling the knowledge in neural network. arXiv preprint arXiv: 1r31v:1503.02531, 2(7), 2015.
Chi Hong, Jiyue Huang, and Lydia Y Chen. Mega: Model stealing via collaborative generator- substitute networks. arXiv preprint (rXiv:2202.00008. 2022.
Navid Kardan and Kenneth O Stanley, Mitigating fooling with competitive overcomplete output layer neural networks. In 2017 International Joint Conference Neural Networks (IJCNN) pp- 518-525. IEEE, 2017.
Navid Kardan, Mubarak Shah, and Mitch Hill. Self-joint supervised learning. In International Conference on Learning Representations, 2021.
Sanjay Kariyappa, Atul Prakash, and Moinuddin K Qureshi. Maze: Data-free model stealing attack using zeroth-order gradient estimation. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 13814-13823, 2021.
Manish Kesarwani, Bhaskar Mukhoty, Vijay Arya, and Sameep Mehta. Model extraction warning in mlaas paradigm. In Proceedings of the 34th Annual Computer Security Applications Conference, pp. 371-380, 2018.
Alex Krizhevsky, Geoffrey Hinton, et al. Learning multiple layers of features from tiny images. 2009.
Tommy Li and Cory Merkel. Model extraction and adversarial attacks on neural networks using switching power information. In International Conference on Artificial Neural Networks, pp- 91-101. Springer, 2021.
Aleksander Madry, Aleksandar Makelov, Ludwig Schmidt, Dimitris Tsipras, and Adrian Vladu. Towards deep learning models resistant adversarial attacks, arXiv preprint arXiv:1706.06600 06083, 2017.
Yuval Netzer, Tao Wang, Adam Coates, Alessandro Bissacco, Bo Wu, and Andrew Y Ng. Reading digits in natural images with unsupervised feature learning. 2011.
Tribhuvanesh Orekondy, Bernt Schiele, and Mario Fritz. Knockoff nets: Stealing functionality of black-box models. In Proceedings of the IEFF/CVF Conference on Computer Vision and Pattern Recognition (CVPR), June 2019.
Xiaoyang Qu, Jianzong Wang. and Jing Xiao. Enhancing data-free adversarial distillation with ac- tivation regularization and virtual interpolation. In ICASSP 2021-2021 IEEE International Con- ference on Acoustics, Speech and Signal Processing (ICASSP). pp. 3340-3344. IEEE, 2021.
Adriana Romero, Nicolas Ballas, Samira Ebrahimi Kahou, Antoine Chassang, Carlo Gatta, and Yoshua Bengio. Fitnets: Hints for thin deep nets. arXiv preprint arXiv: 141,6550, 2014.
Sunandini Sanyal, Sravanti Addepalli, and R. Venkatesh Babu. Towards data-free model stealing in hard label setting. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), pp. 15284-15293, June 2022.
Abhinay Shrivastava, Abhinav Gupta, and Ross Girshick. Training region-based object detectors with online hard example mining. In Proceedings of the IEFE conference on computer vision and pattern recognition, pp. 761-769, 2016.
Johannes Stallkamp, Mare Schlipsing, Jan Salmen, and Christian Igel. The german traffic sign recognition benchmark: multi-class classification competition. In The 2011 international joint conference on neural networks, pp. 1453-1460. IEEE, 2011.
Christian Szegedy, Wojciech Zaremba, llya Sutskever, Joan Bruna, Dumitru Erhan, Ian Goodfellow, and Rob Fergus. Intriguing properties of neural networks. arXiv preprint arXiv: 1312,6 2013.

---

Published as a conference paper at ICLR 2023
Florian Tramèr, Fan Zhang, Ari Juels, Michael K Reiter, and Thomas Ristenpart. Stealing machine learning models via prediction (APIs). 25th USENIX security symposium (USENIX Security 16). pp. 601-618, 2016.
lean-Baptiste Truong, Pratyush Maini, Robert Walls, and Nicolas Papernot. Data-free model extraction. In Proceedings of the IEEE/CVE Conference on Computer Vision and Puttern Recog- nition, pp. 4771-4780, 2021.
Gregor Urban, Krzysztof Geras, Samira Ebrahimi Kahou, Ozlem Aslan, Shengjie Wang. Rich Caruana, Abdelrahman Mohamed, Matthai Philipose, and Matt Richardson. Do deep convolu- tional nets really need to deep and convolutional? arXiv preprint wrXiv:1603.05690 2016.
Yixu Wang, Jie Li, Hong Liu, Yan Wang, Yongjian Wu, Feiyue Huang, and Rongrong Ji. Black-box dissector: Towards basing-based hard-label model stealing attack. arXiv preprint arXiv:2105.00622 2021.
Zi Wang. Zero-shot knowledge distillation from a decision-based black-bor box model. In International Conference on Machine Learning, pp. 10675-10685. PMLR, 2021.
Han Xiao, Kashif Rasul, and Roland Vollgraf. Fashion-mnist: a novel image dataset for benchmark- ing machine learning algorithms. arXiv preprint arXiv: 2017.
Hongxu Yin, Pavlo Molchanov, Jose M Alvarez, Zhizhong Li, Arun Mallya, Derek Hoiem, Ni- raj K Jha, and Jan Kautz. Dreaming to distill: Data-free knowledge transfer via deepinversion. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp- 8715-8724, 2020.
Honggang Yu, Kaichen Yang, Teng Zhang, Yun-Yun Tsai, Tsung- Yi Ho, and Yier Jin. Cloudleak: Large-scale deep learning models stealing through adversarial examples. NDSS, 2020.
Xiaoyong Yuan, Leah Ding, Lan Zhang, Xiaolin Li, and Dapeng Oliver Wu. Es attack: Model stealing against deep neural networks without data hurdles. IEEE Transactions on Emerging Topics in Computational Intelligence, 2022.
Zhenrui Yue, Zhankui He, Huimin Zeng, and Julian McAuley. Black-box attacks on sequential recommenders via data-free model extraction. In Fifteenth ACM Conference on Recommender Systems, pp. 44-54, 2021.
Chaoning Zhang, Philipp Benz, Adil Karjauv, and In So Kweon. Data-free universal adversarial perturbation and black-box attack. In Proceedings of the IEEE/CVF International Conference on Computer Vision, pp. 7868-7877, 2021.
Jie Zhang, Bo Li, Jianghe Xu, Shuang Wu, Shouhong Ding, Lei Zhang, and Chao Wu. Towards efficient data free black-box adversarial attack. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), pp. 15115-15125. June 2022.
Mingyi Zhou, Jing Wu, Yipeng Liu, Shuaicheng Liu, and Ce Zhu. Dast: Data-free substitute training for adversarial attacks. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), June 2020.
12


Reviews:
[{"review_id": "uC1TBlGKJ7", "reviewer": "Reviewer_aGyH", "summary": "", "questions": "", "limitations": "", "rating": 8, "confidence": 4, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "", "summary_of_the_paper": "This paper adresses data-free model stealing, an ambitious framework where one attempts to build a model that imitates a target one without accessing the target parameter or any training data point (true or substitute ones). In this field, SOTA approaches use a student network whose outputs are meant to minimize a distance between its outputs and the outputs of the target (which is the only information unveiled by the target model). The student competes with a an adversary as part of a min-max game in order to efficiently explore the input space. \n \nCompared to previous approaches, the authors use two students and exploit disagreement between the two to better explore the input space in regions where harder inputs live. Disagreement is exploited by an adversary input generator which will try to maximize conflict. The generated inputs are the one on which student and teacher prediction should align. ", "summary_of_the_review": "Major remarks: \n\nThe only missing aspect in the paper is an analysis of increased time/memory requirements of the method  compared to prior arts.\n\nLet alone that, the paper is interesting and I do not have compelling remarks to address but rather a few questions that came to my mind :\n\nIs there any benefits in training more than 2 students ? Perhaps more precise finer-grained disagreement regions could be obtained in this way.\n\nDo the author have an opinion on possible counter-measures that a defender could implement to misguide the students ?\n\nMinor :\n\nIn some parts of the paper, the authors use the word « image » as if it was obvious that inputs are images. ", "strength_and_weaknesses": "Pros : \n- the paper is well written and pleasant to read\n- the proposed contribution requires fewer queries to the to-be-stolen teacher model and is also applicable in situations where the number of classes is not known in advance. \n- the proposed approach is fully differentiable which is a nice increment over SOTA.\n\nCons : \n- the paper does not mention computation time/complexity overhead incurred by using an additional student", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {"correctness": "4: All of the claims and statements are well-supported and correct.", "technical_novelty_and_significance": "4: The contributions are significant, and do not exist in prior works.", "empirical_novelty_and_significance": "4: The contributions are significant, and do not exist in prior works."}, "extra_text": {}}, {"review_id": "qUgbfE5hP4Y", "reviewer": "Reviewer_yxtQ", "summary": "", "questions": "", "limitations": "", "rating": 3, "confidence": 3, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "", "summary_of_the_paper": "This paper has addressed the problem of data free model stealing, where a dual student based framework is proposed for better estimating gradients of the target model without access to its parameters, and generating a diverse set of images that thoroughly explores the input space. While the proposed framework looks interesting, its design is not well justified and not well ablated.", "summary_of_the_review": "The paper is interesting with a new architecture, but the design choices are not well grounded. Some important experiments are missing which makes it difficult to judge the effectiveness of the proposals.", "strength_and_weaknesses": "**Strengths**\n\n(1) The paper addresses an interesting problem of data-free learning or data-free model stealing, where they proposed a dual-student based framework.\n\n(2) The paper has reported encouraging results on the available benchmarks for both standard classification accuracy and transfer-based attack effectiveness.\n\n**Weaknesses**\n\n(1) It is not clear why a dual student model is needed for this. How are they creating more effective criteria? In my understanding it can also be done with the target model and only one student. In that case, the only student will be trained to match the target model, while the generator can be tasked to generate samples on which the student and the target model disagree for creating hard samples. More details with writing is needed together with appropriate ablation study.\n\n(2) There are some clarifications on the reasons at the very last paragraph of section 3.1, however those reasoning aren't well grounded. So please cite the papers that utilize the forward differences method to estimate the gradient through the target model. Additionally, ablative studies should be performed or indicated to justify the design choice.\n\n(3) Results on CIFAR 100 dataset are missing in table 1. As the model is motivated based on its effectiveness of generating more diverse samples to be effective for datasets with more number of classes, results on CIFAR 100 is important. In table 1 of the supplementary material, both the rows show results on CIFAR 10, is it correct? If so, why are the numbers in the DFMS-HL/SL rows different?", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {"correctness": "2: Several of the paper’s claims are incorrect or not well-supported.", "technical_novelty_and_significance": "2: The contributions are only marginally significant or novel.", "empirical_novelty_and_significance": "Not applicable"}, "extra_text": {}}, {"review_id": "KAfv2P6EDZ", "reviewer": "Reviewer_vQa1", "summary": "", "questions": "", "limitations": "", "rating": 8, "confidence": 4, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "", "summary_of_the_paper": "This paper proposed a Dual Student (DS) to mimic the behavior of a black-box model without any prior about input data. The main advantage of DS over existing work lies in that it allows a naturally differentiable process for the training of generator. ", "summary_of_the_review": "This paper is overall good. The writing needs improving as many sentences are too long to read, and some grammar mistakes should be corrected as well. I expect the authors’ response to my above comments.", "strength_and_weaknesses": "Pros:\n1. The introduced additive student model provides an alternative way to update generator which looks reasonable and convincing.\n2. The derived gradients are experimentally demonstrated closer to the true gradient (computed by target model) than DFME which only uses a single student.\n3. The experiments showed that DS can approximate the target model more accurately than the baselines across various tasks.\n\nCons:\n1. Is it possible that two student behaves like two experts in MoE. Let’s say two students work in a complementary way. Will this be risky for simply using the average output during inference? Any experimental observation about this concern?\n2. The authors mentioned that Hong el al. maximized the confidence of the student output. According to my experience, this insight might be helpful to learn a data distribution that is close to the real training data. Has this method been included as a baseline?\n3. Following 2, I felt that data-free model stealing has some overlap with the topic of black-box inverse attack which aims to steal the training samples. The authors should discuss their relations. Particularly, according to the presented gradient error (fig 2), I suspected that the generated data by DS or DFME does NOT looks similar to the true training samples.\n4. I agreed that the proposed formulation may benefit the training of generator. However, the claim of exploring diverse training data has not been well supported. Any further evidence about it?\n5. It would be perfect if some theoretical analyses are provided to support the distance of Eq. 7 is minimized by DS.\n6. Please check the equation number, some of which are not properly referred.", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {"correctness": "3: Some of the paper’s claims have minor issues. A few statements are not well-supported, or require small changes to be made correct.", "technical_novelty_and_significance": "3: The contributions are significant and somewhat new. Aspects of the contributions exist in prior work.", "empirical_novelty_and_significance": "3: The contributions are significant and somewhat new. Aspects of the contributions exist in prior work."}, "extra_text": {}}, {"review_id": "G3q_CEUKr8_", "reviewer": "Reviewer_xQYc", "summary": "", "questions": "", "limitations": "", "rating": 5, "confidence": 3, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "", "summary_of_the_paper": "This paper studies how to perform model stealing in the data-free setting.\nThe authors first identify two weaknesses of existing data-free model stealing methods: (1) the generated query data are not informative enough, and (2) the estimation of black-box gradients is not stable for optimization.\n\nTo address these two issues, the authors then design a novel dual-student model stealing method.\nThe dual-student method, which involves two student networks and a query data generator, aims to solve a minimax problem. The two student networks aim to imitate the behavior of the victim model on generated query data distribution, while the generator aims to generate query data that maximize the prediction differences between student networks and the victim.\nThrough a simple substitution, the solution of the maximization problem can be approximated as maximizing the disagreement between the two student networks, which therefore does not require black-box optimization anymore.", "summary_of_the_review": "In general, the proposed method is novel and shows some performance advantages. However, the authors did not fully justify its effectiveness. Besides, the writing of this paper needs to be improved. Therefore, I tend to reject this paper for now. I suggest the authors to improve the writing, add more experiments, and submit the paper to another top conference.", "strength_and_weaknesses": "The proposed dual-student method is interesting and empirical results show that it surpasses existing data-free approaches in some situations. However, I think the current version of the paper can be improved in many aspects, which are listed below:\n\n1. The writing of the paper is not well, making it a little confusing to follow the ideas behind the paper. Specifically, in Section 1, the authors put most of the details of dual-student in a single paragraph, which makes it difficult for reading. Besides, Section 2 uses too much content in reviewing previous works (more than 1 page), which however is not so necessary for introducing the proposed dual-student methods.\n\n2. The insight of the proposed dual-student method is not clear enough.\n    - Why replacing the objective problem in Eq.(1) with Eq.(2) is appropriate? I suggest the author give at least an intuitive explanation about it.\n    - An ablation study on the relationship between the step numbers for updating the generator and student networks would help understand why the dual-student method performs well in practice.\n\n3. Some experiment details are missing.\n    - What is the model architecture of the query data generator?\n    - Please explain how to combine the two methods, DFMS-HL/SL and DS in detail.\n\n4. The authors claim that they conduct experiments on the CIFAR-100 dataset. However, I could not find corresponding experiment results in tables or figures from the paper.\n\n5. Some suggestions about improving the empirical studies.\n    - Study the relationship between query budget and stealing performance of the dual-student method.\n    - Involve more black-box attacks for evaluation, e.g., membership inference attacks, and model-inversion attacks.\n\n6. Typos: In Eq.(3): $G(x)$ -> $G(z)$.", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {"correctness": "2: Several of the paper’s claims are incorrect or not well-supported.", "technical_novelty_and_significance": "3: The contributions are significant and somewhat new. Aspects of the contributions exist in prior work.", "empirical_novelty_and_significance": "2: The contributions are only marginally significant or novel."}, "extra_text": {}}]

Author rebuttal:

Decision: Accept: poster