MExMI: Pool-based Active Model Extraction Crossover Membership Inference
Authors: Yaxin Xiao
Qingqing Ye
Haibo Hu
Huadi Zheng
Chengfang Fang
Jie Shi
Venue: NeurIPS 2022 Accept (2022)
Keywords: AI Safety
Model Extraction
Membership Inference

Abstract: With increasing popularity of Machine Learning as a Service (MLaaS), ML models trained from public and proprietary data are deployed in the cloud and deliver prediction services to users. However, as the prediction API becomes a new attack surface, growing concerns have arisen on the confidentiality of ML models. Existing literatures show their vulnerability under model extraction (ME) attacks, while their private training data is vulnerable to another type of attacks, namely, membership inference (MI). In this paper, we show that ME and MI can reinforce each other through a chained and iterative reaction, which can significantly boost ME attack accuracy and improve MI by saving the query cost. As such, we build a framework MExMI for pool-based active model extraction (PAME) to exploit MI through three modules: “MI Pre-Filter”, “MI Post-Filter”, and “semi-supervised boosting”. Experimental results show that MExMI can improve up to 11.14% from the best known PAME attack and reach 94.07% fidelity with only 16k queries. Furthermore, the precision and recall of the MI attack in MExMI are on par with state-of-the-art MI attack which needs 150k queries.

MExMI: Pool-based Active Model Extraction Crossover Membership Inference
Yaxin Xiao
Qingqing Ye The Hong Kong Polytechnic University qqing. ye@polyu. edu. hk
The Hong Kong Polytechnic University
20034165r0connectt wtw.oconnnntttoo......oom polyu.hk
Haibo Hu
Huadi Zheng
The Hong Kong Polytechnic University haibo. hu@polyu edu. hk
The Hong Kong Polytechnic University huadi zhengSconnect, polyu. hk
Chengfang Fang Huawei International, Singapore f fang.
Jie Shi Huawei International, Singapore shi. j ie @huawei. com
com
Abstract
With increasing popularity of Machine Learning Service (MLaaS), ML models trained from public and proprietary data are deployed in the cloud and deliver prediction services users. However, as the prediction API becomes new attack surface, growing concerns have arisen on the confidentiality of ML models. Existing literatures show their vulnerability under model extraction (ME) attacks, while their private training data is vulnerable to another type of attacks, namely, membership inference (MI). this paper, show that ME and MI can reinforce each other through chained and iterative reaction, which can significantly boost ME attack accuracy and improve MI by saving the query cost. such, we build a framework MExMI for pool-based active model extraction (PAME) to exploit MI through three modules: "MI Pre-Filter". "MI Post-Filter", and "semi-supers sunni-sueeeeieed boosting". Experimental results show that MExMI can improve up to 11.14% from the best known PAME attack and reach 94.07% fidelity with only 16k queries. Furthermore, the accuracy, precision and recall of the MI attack in MExMI are on par with state-of-the--eee art MI attack which needs 150k queries.
Introduction
Recent advances in machine learning (ML) has significantly shifted the paradigm in all walks of life. Thanks to cloud computing, every business and individual can host black-box ML models the cloud (e.g., Microsoft Azure ML, Amazon AWS ML, and Google Cloud Al) provide pay-per-query predictive services, known as Machine Learning as a Service (MLaaS), Obviously, ML models are proprietary assets the providers who spend great efforts training them [22]. However, by exploiting the correspondence between queries and prediction results from an MLaaS model, an adversary could learn the internals of that victim model to large, confidentiality known as model extraction (ME) [43]. It is a fundamental attack in adversarial
full extent. Such attack model's
machine learning because enables follow-up attacks such as adversarial samples [19] model evasion
77. and model inversion attacks that infer private information of the training set [16] 48)
of-te-of-the-oo art ME attacks be categorized into direct recovery [ and aetive-leearnngg based ME attacks 6 34 Active learning (AL) refers to those semi semi-supervised training methods that aim to
"Correspondngg author
36th Conference on Neural Information Processing Systems (NcurIPS 2022).

---

find the most informative training dataset with a limited query budget 17]. Since both AL and ME aim to train a model with as few queries as possible, AL has become increasingly popular in model extraction attacks. Depending on the availability of real-life samples for querying, Al.-based ME can be further divided into pool-based active model extraction (PAME) and query-synthesizing-he active model extraction (SAME). The former assumes the presence of samples, from which query samples are iteratively selected using a pool-based or stream-based AL. Classic PAMEs include ActiveThief [34] and Knockoff [33]. The latter does not assume the presence of such samples and
obtains them by generative methods 22 6.. Classic SAMES include black-box DNNs 35 and
PRADA (26]
However, a critical difference between pool-based AL and pool-based ME has been long neglected. Since pool-based AL is essentially a training method, the data pool is where training samples are drawn, which means this pool must have the same feature distribution as the training dataset. However, in an ME attack the adversary has no access to the victim model's 's training dataset or even samples in the same problem domain [Q] Unfortunately, all previous PAME works ignore this difference and assume the samples in the pool are homogeneous to those in the training dataset. To demonstrate how non-homogeneous datasets can affect the ME performance, we build a Wide-Res.Net-asee victim model using the first 25k training image samples of CIFARIO. Then we extract this model using Active'Thiet [34, a sfate-of-the-are PAME, from three adversary data pools: (1) pool A consists of the second 25k training samples of CIFARIO, (2) pool B consists of the middle 25k training samples, and (3) pool C' consists of the first 25k training samples. In other words, the victim's 's training set does not overlap with A (non- noon-homogeneous) but overlaps with half of B muuasi-homogeneouss and completely with C (fully homogeneous). Table shows the fidelity (i.e., similarity to the victim model) of the extracted copy models from A, B and C. We observe that homogeneous samples contributes more to the success of model extraction than non-homogeneous samples.
In this paper, we address this issue by identify- ing homogeneous samples in the data pool and making full use of them for a "guided" model extraction. We exploit membership in- ference (MI) attack that infers the training samples from a given dataset [4T]. to select them and train a copy model. In turn, the ex-
Table I: Homogeneous v.s. Non- Non-Homogeneous Data Pool for Model Extraction
The Adversary Data Pool Training Data Homogeneity
A
B
C
None
Partial
Full
Fidelity / %
90.32
91.29
92.30
tracted model can enhance the accuracy of membership inference. As such, we propose an iterative extraction framework MExMI where ME and MI reinforce each other through iterations. As such, within limited query budget, the final outcome consists of both a high-fidelity copy model and an accurate set of training samples.
There are several challenges in the design of MEXMI. First, existing MI attacks do not consider query cost [4 39,, which is essential in ME attacks. We exploit the training data property of the copy model to conduct MI attacks without consuming query budget. Second, state-of-the-an MI attacks impose assumptions that are not available in MExMI or PAME in general. For example, mndew MI requires a labelled dataset drawn from the same distribution as the model's training dataset with the same size [41]. In this paper, we propose quality metric to evaluate and optimize a shadow model without the need of a large labelled dataset. With this technique, we adapt both shadow-model MI and unsupervised MI (39 47 the MEXMI framework. Third, existing PAME attacks fail to utilize the training samples purified from the pool. We designed three modules to facilitate PAME to make the most of it. In summary, we make the following contributions:
We propose a taxonomy (S (SAME/PAME) on and an iterative framework MEXMI where PAME and MI reinforce each other.
seearnng-basse model extraction attacks,
We adapt both shadow-model MI and unsupervised MI to MEXMI. To boot-strap shadow- model MI, develop indicative quality metric of shadow models and design a metric- based shadow-moded training algorithm.
We conduct extensive experiments compare the effectiveness of MExMI with ot the-art PAMES [34]. MEXMI outperforms the latter by 11.14% to 94.07% in terms of ME fidelity, and its MI attack achieves 84.13% precision, on par with the state-of-the-arr MI attack (39 77.)5%) that assumes unlimited query budget.
The rest of the paper is organized follows. Section 2 introduces the background and definitions of ME and MI attacks. Section 3 overviews the MExMI framework and its key components, and

---

Section A introdduces adaptive MI algorithms for MExMI. Section presents the evaluation results of MExMI through extensive experiments. Section reviewws related works, and Section concludes this work.
2 Background and Definition
2.1 Victim Models
Our target victim models are multi-class neural network (NN) classifiers, where the input domain is
X S IR and the output is probability vector of classes V _ TRK Here d denotes dimensionality of the input data, and K denotes the number of output classes. neural network model makes inference
on the output class probabilities y based on the input x: y Pr (Y = 0/x). Pr (Y K xx)]. To train such model, we assume a supervised learning process Let D y.) S X x Y denote the victim's 's training dataset with labelled samples, where is the i-th training sample, and y, is its one-hot format label vector.
2.2 Threat Model
We assume well-trained black-box victim model F is deployed in secchineleamiingaaae a-sevice (MLaaS) with chargeable query interface. For ease of presentation we do not consider the case when the victim employs anti-ME (26) or anti-MI defenses, such as perturbation
the output probability vector (52 [53. We also assume that the adversary either only has black-box access to the MLaaS model, or knows its architecture (e.g. in AWS Marketplace ① and Huawei Al Gallery (2). In addition, the adversary can collect large number of unannotated public samples to construct an adversary data pool P.
2.3 Problem Definition
We assume the adversary performs ME attacks against F within a query budget b, with the aim to produce copy model F". The main objective of ME is to let F' approximate the victim model F with high degree of resemblance, a.k.a.,fdellip (2] which is the proportion of label agreement of two models on an evaluation dataset D,-
3 Model Extraction Crossover Membership Inference
In this section, we present our iterative model extraction framework MExMI where ME and MI reinforce each other. An MI attack aims to distinguish those individuals D from a population P that exist in the victim model F's training dataset [41]. As illustrated in Fig. and pseudocode in Appendix A the input of MExMI is an adversary data pool P and the access to a black-box victim model F, and its outputs are the copy model F' and the inferred training dataset D. In the first iteration, the adversary chooses & initial seed samples [x....., Nteo from P without putting them back, where A is the query budget per iteration. These samples are fed to the victim model F, which outputs probability vector (x)). Then an MI attack model FMA A is constructed using the queried dataset [x1, F(x:)] -.. or the copy model (see Section 4p. FAsTA A is used in both MI Post- and MI Pre-Filter modules. The queried dataset is then passed through the MI Post-Filter, which weighs them according to their probability of being a training sample of the victim model. Then the weighted queried dataset used train the copy model F" which then fed the MI Pre-Filter to refine the adversary data pool for AL sample selection of is queries in the next iteration. The process repeated until the total query budget b depleted. Thereafter, without consuming any query budget, FMIA is used to launch MI attacks on F' and the data pool to obtain the final inferred training dataset D. This dataset will be used in semi-supervised learning on F' to release the final copy model F'.
From Fig. I MExMI has three key modules on top of the basic PAME iterative framework (34). namely, MI Fre-Fi, MI Post-Filter and semi-supervised boosting that will be claborated in rest of this section. As will become clear, they are orthogonal each other, they can be turned on off
Throughout this paper, we exclude special training algorithms, e.g., co-training, mutual mean mean- teaching, and sharpness-awe aware minimization [46 49 [5]. Neither the victim nor the adversary uses these algorithms.
3

---

Start: Initial Seeds
sumples
Query Dara Victim Model F
reeodd
Post-Filter
weighted rocunds
Train
[x
[[x, P(x)).
Modkl
Samples
R sumples [R.....
Adversary Data Pool
Query Data from Copy
MI Pre-Filber
Get Filtered Pool
Final Round!
Al. Selection
Copy Model F
Model F
Yes
unlabelled samplee
Semi-upervised Roceting
Final Copy Model F
Openiiion prasosess
Decision Bowetar:
MixMi process
ML modeli
Data flows
Medei floax
Modkll fanns MI upcatting
Figure 1: MExMI iterative framework.
independently. For ease of presentation, the construction of the MI attack model FMIA. a key issue the MExMI framework, will be back introduced Section
3.1 MI Pre-Filter
MI Pre-Filter works before AL sample selection. The core idea is to MI attack model FM to choose from the adversary data pool P only those samples that are highly homogeneous to the victim's 's training data D. Training the copy model with them, both models can thus exhibit strong resemblance. Ideally, FASIA should perform membership inference attack directly on the victim model F, which is truly trained from D. However, since such training causes extra query budget on F, FMIA attacks the copy model F" instead as the latter exhibit similar training data property.
3.2 MI Post-Filter
MI Post-Filter works after querying a sample x from the victim model. The rationale of this filter is that since the victim model returns the probability vectors F(x), the adversary infer if belongs to the training dataset by an MI attack model FMIA Obviously negative membership result means this sample may not lead to high-fidelity copy model F", its contribution the training process should lowered by reducing its weight in the training loss calculation. We use a parameter (w 1) denote the weighted loss ratio of a positive membership sample a negative one.
3.3 Semi-Supervised Boosting
The main idea of this module is that MExMI gain the results of its MI attack- an inferred training dataset D of the victim model. Although this dataset is not labelled and there is budget to label them in the end of MExMI, we can still train the copy model on this dataset together with the queried set using semi-supervised learning algorithms. Note that this module is unique in MExMI as other PAME methods cannot distinguish training samples from others in the data pool. Nonetheless, this module is intended improve model accuracy only, fidelity, because semi-supervised learning can divert the copy model's 's ability to follow the same label distribution as the victim model's training data. Therefore. MExMI only applies sup-ssupeeeieed boosting after the final iteration and when higher accuracy is needed.
query
4 Adaptive Membership Inference
In MExMI MI attack model FMIA is trained after the initial seeds query. For MI play role in MExMI framework in early stage iterations, FMIA must be accurate enough even when there are only few samples. In this section renovate existing MI algorithms to be adaptive to their training sample sizes and thus suitable for the MExMI framework. We focus on two state-of-the-an art black-box MI attack paradigms: (1) shadow-model MI [1 39], and (2) unsupervised MI [39
4.1 Shadow-Model Membership Inference
The rationale of shadow-model MI is obtain shadow model similar to the victim model, so that their output probability vectors for training and non-training samples are also distinguishable in similar manner. As such, the adversary can build binary MI classifier from these samples instead of

---

from those of the victim model. In order to approximate the vietim model, shadow model should (I) draw its training dataset from the same distribution as in the victim model, (2) have roughly the same size of training set as the victim model, and (3) have the same training algorithm. However, in MExMI, neither (1) nor (2) holds because:
a) The queried samples may not be drawn from the same distribution as in the victim's training samples.
b) The number of queried samples is significantly smaller than that the victim's training set size, especially in the beginning phase.
As such, the design principle of our adaptive shadow-mode MI attack is to work under a limited number of labelled training samples in different distribution from the victim model. To start with, we need know how to estimate the quality of shadow model so that we can tell when is good enough for MExMI. Intuitively, the fidelity of the shadow model against the victim model can serve as the performance indicator, but is inaccessible from the adversary's side, especially when MExMI just starts. As such, we need easy-to-accees performance metric.
4.1.1 Measuring Quality of Shadow-model MI
An MI attack works by distinguishing the output probability vector distribution Y of the victim model F on training samples X from non-training samples. shadow-mode MI estimates the above on a shadow model - it distinguishes the output probability vector distribution V(() of shadow models F, training samples x from non-training samples. As such, to improve the attack accuracy, y(s) should be similar as possible to the victim's is Y. We measure the similarity by the bias of the expectation values between (+() and Y, denoted by b. For each shadow model that targets at class j e [ .K] by is formally defined as:
b
Σ n y
Σv. y. nJ yE
(1)
where n denotes the size of training set, and the superscript (s) denotes the shadow model. The following theorem shows that the gap between the training loss Al is positively correlated to the bias of expectations between Y(s) and Y. Therefore, can serve as a quality measurement for Fs. We can minimize it to enhance the quality of y(*) approximating Y.
Theorem (Quality measurement for shadow-model MI). Given a shadow model F, which has the same model structure and hyper-parameters as the victim model F, the gap A/ between the training loss of F, and F is positively correlated to the bias of expectations between y(() and Y, i... Al x See Appendix for the proof.
4.1.2 Metric-based Shadow-model MI
To obtain Al, the gap between the training loss of both models, need them both. However, the victim's training loss is not available to the adversary. Nonetheless, in practice a victim model is valuable for extraction mainly because this model accurately predicts the training data, or equivalently, its training loss is smaller than other models of the same training set size. As such, we can replace A/ with the loss of the shadow model, denoted by I., compensated by its training dataset size n(*). Furthermore, for the sake of comparing various shadow models, only the relative rather than the absolute value of the gap between shadow models and the victim model A F is needed. So we propose the following metric as a negative relative value of AF. The larger the Q, the better the quality of a shadow model. Formally,
AF Q(),n f(1.) x (n())),,
(2)
where f() is non-negative non-linear decreasing function, and E [0, 1] scales down the impact of (+). Note that Q does not require that shadows' training data come from the same distribution as the victim's.
From the above equation, there are two ways to increase the quality metric Q: (1) reducing l, using good training algorithms on shadow models, and (2) increasing (= by augmenting the training set. Few-shot learning (FSL), training paradigm to improve models' accuracy with a limited number
5

---

of examples 44 can serve both purposes. For example, we can use FSL approaches, namely data augmentation 29 and transfer learning [2T, train shadow models. This leads two metric-based soadow-model Mls, which are elaborated in Appendix D
4.2 Unsupervised Membership Inference
Recent works [39 47 have shown the effectiveness of unsupervised learning on MI attack models. In such models, the feature values are usually the top-m score, loss entropy of the output probabilities, and the output value serves as the confidence of membership inference if the value is higher than an siveersay-sppcifiee threshold the sample is inferred as in the training dataset and vice versa. To set this threshold, the adversary first gets a set non non-member samples and then query them to get corresponding top-m scores. The top percentile of these scores can serve threshold [39]. To save the query cost, the copy model instead the victim model should be queried.
5 Experiment
In this section, we first conduct experiments validate the shadow model quality metric Q. Then we evaluate the attack performance of four variants of MExMI Pre-Filter only, Post-Filter only, MExMI without semi-supervised boosting, and MExMI - against state-of-the- tf-tt-- ME and MI attacks. The codes are available at httts:://gtt tiggt.uu.... com www.comm////ccceeeeeee mwwmi//mmmm//////i//////em//mmmm htt//mmxmmmprrrp -pepe//eeeemmeeeejeeettt..e
5.1 Experiment Setup
Datasets. perform PAME attacks on two image datasets, namely CIFARIO 28 and Street View House Number (SVHN) 32 and text dataset AG'S NEWS which contains corpus of AG's news articles [T]
(see Appendix F. for details).
Victim Model. For the image classification task, we use Wide-ResNet-28-- [49 trained on CIFARIO as the victim model with accuracy 10%. We also use a cloud MLaaS, ModelArts 3 to train an online victim model on SVHN that leads to 94.30% accuracy. In the text classification task, we use DPCNN 255 as the victim model which achieves an accuracy of 89.8%%.
Adversarial data pool. In the default CIFARIO experiments, the pool consists of 50k training samples and 100k from the ImageNet32 (9. In the default AG'S NEWS experiments. the pool has 50k training samples and 100k from Dbpedia 44. Note that as with existing pool-based ME (34), MExMI does not require the pool to contain training samples. The main reason for such mixed dataset composition is for us to evaluate the performance of MI [41] and show how much it can be enhanced by ME. In Section 5.4 we evaluate the performance of MExMI when the pool has no training sample at all.
5.2 Metric-based Shadow-Model MI
Implementation Details. Recall that the super-parameters (such as epoch, initial learning rate, and optimizer) of shadow models F, can be adjusted to maximize the metric Q. Parameter in Q is set as 0.05 and f()) is set as 1o (-) { The training dataset of Fs, denoted by D(*), is constructed by random sampling from P. and its size varies from the set (2000, 5000J. A non-training dataset, denoted by ((*) is also randomly sampled from P D)*)_ We implement two metric-based shadow- model MI methods in Section original shadow-model MI [4 and FSL. shadow-model MI that utilizes transfer learning [21] and data augmentation respectively. (See Appendix [E.5 experimental details.) For each MI method, we vary their hyper-parannn parameeersss and thus Q in various settings. An ideal shadow-model MI is built as a reference by using the same training dataset as the victim model.
Results. We plot the MI attack models' accuracy with respect to shadow metric Q in Fig, We observe that Q is positively correlated with the attack accuracy of shadow-mode MI and therefore guide the training process of the shadow models, whether using the original FSL training algorithms. The recall rate of all MI attacks is almost 100%, so it is omitted from the figure. In addition, once the metric Q is large enough (2 6), metric-based shadow-model MI attack models can achieve almost the same accuracy as the ideal shadow-mode MI 39 even for 2k samples.
In experiments 4, (0,1) where --og10(-) is non-negative and ever decreasing.

---

shadow MI acairacyle
shedow accurecy
80
80
70-
60-
Original
-0- FSL Ideal
Ideal
50
50
shadow metric Q
shadow metric Q
(a) Original
(b) FSL
Figure 2: Experiment results of metric-based shadow-modes MI under different settings.
5.3 Overall Performance of MExMI
5.3.1
Implementation Details
We compare six PAME attacks, including four MEXMI variants, namely, baseline ME without MI, Pre-Filter only, Post-Filter only, MExMI without semi-supervised boosting, regular MExMI (which adopts Mix-Match semi-supervised methods [3] for image classifiers, and consistency regulariza- tion 33 semi-superviee
methods for text classifiers), and the ideal ME attack. The baseline ME is ActiveTThie [34, which is the state-of- sene--ff-theeatee art PAME attack F The ideal ME uses the real training samples its pool. Moreover, to comprehensively evaluate MEXMI, we added two additional state-of-the-art query-synthesszing-bern biasse ME baselines, including PRADA (26) and DFME [44] which share the same query budget with PAME attacks. Each MExMI variant has MI result. In our experiments we compare our MI attacks that don't 't cost additional query budget with the existing MI attacks (39 that assume infinite query budget.
The AL algorithms used are entropy uncertainty 31. greedy k-center 44) and adversarial deep- fool [4 (see Appendix B for a brief explanation). The last algorithm is not evaluated on AG'S NEWS since there is no trivial way to adapt it to text classification. The copy models in MExMI share the same architecture the victim models. The MI attack model FMIA is trained initial seed samples. The preset weights ratio w in MI fost-Fiiter is 5 1. For CIFAR10 experiments, MExMI queries 2k samples in each round with total of 8 rounds. For AG'S NEWS experiments, there 6 rounds, each with 5k samples. All experimental results are the average measures of 5 trials. See Appendix E for more implementation details and Appendix F for supplementary experimental results.
5.3.2 Overall Results of MEXMI
We use fidelity and model's accuracy obtained from the test datasets D. (CIFAR10 and AG'S NEWS test sets) evaluate copy models against the victim model (see Section 2.2), and use accuracy, precision and recall of the inferred training datasets against ground truth to evaluate the MI accuracy,
Fig. 3 plots the fidelity of various PAME attacks with respect to iterations CIFARIO and AG'S NEWS (see more figures in Appendix F.1 respectively' 5 and Table 2 shows the final results. Fig. plots the accuracy, precision and recall of the MI attack of each MExMI variant. The results indicate that MEXMI greatly boosts the potential of AL algorithms in PAME, and breaks the curse on query budget of existing MI. Overall, MExMI performs the best and achieves fidelity gain of 7.76%, 7.8%, and 11.14% on CIFAR10 over the baseline ME attack in all three AL methods. A similar gain of 4.46% and 3.46% is observed AG'S NEWS over the baseline PAME attacks in both two AL methods. Without additional queries, the MI attacks of MEXMI yield up to 83.20% accuracy, 84.13% precision and 75.93% recall on CIFARIO, 10, and 78% accuracy, 71.73% precision and 82.53% recall on AG'S NEWS respectively, both on the par with existing MI attacks (3 that assume infinite query budgets.
Impact of MI Post-Filter and MI Pre-Filter. In Fig. 3 on CIFARIO attacks with MI Post-Filter always outperform those without it, by up to 1.64% increase on fidelity in the final results. The gain is more eminent in the beginning iterations, and then gradually decreases. On AG'S NEWS, MI
'INVERSENET [8 is another state-of-the of-the-atttttttooo art work that adopts model inversion to assist ME. We do not include it in the experiments for two reasons. First, its performance is similar to ActiveThief under our 16k 16k-q budget setting. Second, it augments query selection from data pool with query synthesis from the model. which can be considered orthogonal to our method.
The label 'MEXII w/ Boosting' in figures is abbreviation for MExMI without semi-supervised boosting.
7

---

CIFARTO
FFARTD
90
50- J6S
Fitetyns
Bacol
Fieeny's
Dood no
Foowltyrs.
Backlee
Palin only
Preft toroon unly
70
Proit tor unly
Profi ter onl
Pakr only
Fors terenty
60
70-
Micoii wi Boosing
locel case
Dicel case
lreal cast
50
4k
8k 10k 12k 14k 16k Query number
Ge 10k 12k 14k 16k
61
5k 10k
20k Query number
15k
25k
Query number
Quary rumber
(a) Entropy Uncertainty
(b) Greedy K-center
(c) Entropy Uncertainty
(d) Greedy K-center
Figure 3: PAME results on CIFARIO (16k budget) and AG'S NEWS (30k budget). Shadows represent eтror bars.
NCP
Crisgr
Creacy
Gredy
decakel
(a) MI Accuracy (b) MI Precision
(c) MI Recall
(d) MI Accuracy (e) MI Precision
(1) MI Recall
Figure 4: MI attack results of MEXMI. "ML-leaks" refers to the shadow-model MI attack in 33
Post-Filter also performs effectively, with maximum boost of 1.57% on fidelity over those without it. As for MI Pre-Filter, except for the greedy k-center one in CIFAR 10 experiments, attacks with Pre-Filter always outperform those without it. In addition, the gain does not decrease with more iterations, because the adversary data pool is much larger than the total query budget. Interestingly, we find that when MI Pre-filter and MI Post-filter work together, they can achieve greater gain on fidelity than the sum of individual gains when they work separately. This suggests that the two filters truly reinforce each other in our MExMI framework.
Table 2: Results of Default PAME Experiments
CIFARIO
AG'S NEWS
Fidelity (Accuracy)%
Entropy
Greedy
Adversarial
Entropy
Greedy
Uncertainty
K-center
Decp-fool
Uncertainty
K-center
PRADA (6)
44
61.32 (60.12) / 11.20 (10.32)
3/.02 (25.00)
Baseline Thie))
84.65 (83.78)
86.26 (85.69)
82.93 (82.27)
81.36 (78.66)
85.03 (81.92)
Pre-Filter only
85.38 (85.38)
85.84 (85.22)
86.17 (85.48)
941.6(81.57)
86.61 (83.36)
Post-Filter only
85.48 (84.71)
87.68 (86.86)
84.57 (84.00)
82.06 (79.29)
85.84 (82.38)
MExMI w/o Boosting
89.10 (88.69)
90.16 (89.21)
90.14 (89.58)
85.18 (81.98)
87.26 (84.18)
MExMI
92.41 (91.80)
94.06 (93.43)
94.07 (93.47)
85.82 (82.54)
88.49 (85.36)
Ideal case
93.31 (93.03)
93.71 (93.38)
93.66 (93.25)
90.68 (87.51)
91.03 (87.68)
Impact of Semi-SSperrvsed Boosting. In Fig. and Table MExMI outperforms MExMI without supervuuere boosting by at least 3.11% on CIFARIO and 0.56% on AG'S NEWS in terms of accuracy, which indicates that MExMI does benefit from effective MI attacks. A fidelity gain is observed in MEXMI since the copy models' accuracy is closer to that of the victim model.
MI Performance in MEXMI. The precision and recall of the adaptive mhael MI attacks of three MEXMI variants are shown in Fig. 4 MEXMI always performs the best and can achieve up to 83.20% accuracy and 83% precision AG'S NEWS. This precision is even better than the state-of-the-art MI - ML-leaks [39] 275.25% on CIFARIO, 10, 65.37% on AG'S NEWS) which assumes unlimited query budget. Furthermore, our adaptive MI attacks have no additional cost when inferring training samples.
CIFARIO, and 68, 77% and 71.73% precision
Discussion About Potential Defenses. There are two potential defenses against MEXMI. First, MExMI is subject to MI-related defensive strategies that can reduce the accuracy of MI, such as using differential privacy [33, which in turn lower the fidelity of MExMI. Second, as with other model extraction attacks, ME can also defended by model provenance techniques, such as watermark embedding [24. to detected copyright infringement from extracted model.
8

---

5.4 Impact of Adversary Data Pool on PAME
In this experiment, we study how the quality of adversary data pool affects the outcome PAME attacks. Since there are cases where the data pool does not contain any training data, the results of MI attack are not evaluated. To be fair, we fix the size of the pool to 150k samples and change the proportion of training data in it to vary its quality. Since the total training samples are 50k, this proportion is capped at 1/3.
The results on CIFAR1O are shown in Table 3. We can see that the quality of the data pool greatly affects each PAME attack. MExMI consistently outperforms the baseline irrespective of the quality, even in the complete absence of victim's 's training data, i.e., when the adversary has no access to the true training samples. In such extreme case, we also observe that Pre-Filter only outperformed by the baseline. This is due to the fact that the Pre-Filter cannot find any training data in the pool and thus excludes most of them for training. Since the filtered data pool is too small, active learning might not be effective.
Table 3: Impact of the Adversary Data Pool on PAME Attacks with 16k Query Budget
Pre-Filter only
Post-Filter
MExMI w/o
The Proportion of P. in P
Baseline
MExMI
only
Boosting
0%
76.30(75.62)
74.23(73.93)
76.29(75.71)
77.40(76.80)
79.11(78.51)
Fidelity
(Accuracy)%
25 %
81.07(80.48)
83.91
83,55(83.05)
89.99(88.78)
92.91(91.96)
33.33% %
82.93(82.27)
86.17(8)
84.57(84.00)
90.14(89.58)
94.07(93..47)
5.5 Impact of Output Access
We investigate the impact of output access granted to our PAME attacks. We limit the output access to top-1 score and show the results on CIFARIO in Table It indicates that even with limited output access, the three modules of MExMI still perform consistently well. Among various PAME attacks, MExMI always performs the best and can outperform the baseline by 11.47% on fidelity.
Table 4: Impact of Output Access on PAME Attacks
Pre-Filter only
Post-Filter
MExMI w/o
Fidelity (Accuracy)/%
Baseline
MExMI
only
Boosting
Probabilitics 82.93(82.27)
84.57(84.00) 81.94(81.18) (1.8(3)))33)
86. (885.48)
90.14(89.58)
94.07(93.47)
Output Access
Top-1 Scores 79.53(79.00)
85.60(84.89)
91.00(90.40)
5.6
Case Study: Blackbox Attacks on MLaaS ModelArts
We use a real case study to show the fea-
sibility and real-life impact of MEXMI. We target at ModelArts, the MLaaS provided by Huawei Cloud K where developers can train and deploy their ML models in the cloud, and then access them via Web API (e.g., CURL). We train and deploy classification model Mode- IArts without the knowledge of its architecture using SVHN and then perform various PAME attacks it with VGG16. See Appendix E.6 for more implemen- tation details.
85-
Bascline
Pre-fiher only
89
75-
Post-fiker
88
MExMi Boosting
65-
MESMI
Ideal case
5k
6k
3k Query
7k
Figure 5: PAME attacks' results in Mode- 1Arts on SVHN (7k budget).
The experimental results of the adversarial deep-fool PAME are shown in Fig. All four MEXMI variants outperform the baseline. MExMI achieves the highest 90.32% fidelity and MExMI without semi semi-supervised boosting comes the second with 89.45% fidelity. The final MI results of MEXMI framework are shown in Table 5 We observe similar precision and recall of the three variants, all on the par with ML leaks [39], the state-of-the-a the-- unsupervised MI attack that exhausts all pool data, which costs ten times higher.
Table 5: MI attacks' results in ModelArts
MI Attacks
Precision/5: %
Recall/%
Pre-Filter only
86.94
91.69
Post-Filter only
87.17
90.50
MEXMI
87.78
91.77
ML-lcaks on F
87.83
92.38

---

Related Works
Model Extraction. Model extraction (ME) on MLaaS models has drawn a lot of attention since Tramer, et al. (2016) [43], mainly because it can be used for downstream attacks. In terms of query data availability, ME can be categorized into query-synthesziing ME and query-acquiring ME. The former is used when the adversary does not possess enough real data for query, which includes iterative active frameworks [35] [6] and minimax-gami -game frameworks 44 27]. Despite saving the data collection cost, it suffers from huge query budget demands. The latter is used when the adversary spares extra cost to actively collect real data for query [00 23 34 IQ. We summarize the above ME attacks in Table [ and highlight the data pool requirement of each work and their dependency on data selection and data synthesis.
Memership Inference. The vulnerability of MLaaS models' training data privacy has raised great conerns [2] and various attacks against it are proposed. Of these. Membership inference (MI) which attempts infer the training membership of some query data, shows a worrying breach of data privacy. Existing works exploit different sources of information such as outputs 41 39 and model
parameters (30 either black-box or white-box setting to infer the training data. black-bon attack
scenario, moet MI works follow chaeo MI [41] 39 which cast the attack to a supervised
learning problem. Another type of black-box MI to utilize the unsupervised learning [39 37] with
different measure metrics, such as output entropy, predicted confidence [9) and loss [37 On the
other hand, a white-box MI is proposed 30 which infers the training data and non-training data distribution characteristies via the victim model's 's parameters. More recent works focus extending
MI attacks to less favorable scenarios, for example when the model output is label only [8] 30], and extremely low query budget.
Table 6: Existing Learning-baat based Model Extraction Works
Attacks
Data Pool Requirements
Data Synthesis
Data Selection
Tramèr (2016)
[4
None
Knockoff (2019)
33
Probelm Domain, or Public Pool
Papernot (2017)
35
Probelm Domain Initial set
PRADA (2019) 226
Probelm Domain Initial set
DFME (2021)
4-
None
ActiveThief (2020)
Public Pool
Jagielski (2020) 24
Probelm Domain Pool
INVERSENET (2020)
18
Public Pool
MEXMI (this work)
Public Pool
7
Conclusion
In this paper, we proposed PAME crossover MI framework called MExMI where the model and training data privacy can trigger a chain reaction to boost the performance of both attacks. The framework is generic in that it can adopt various PAME and MI attacks. In our experiments, MEXMI improves the fidelity of copy models to 94.07%, up from the basic PAME by 11.14%. Meanwhile, the MI accuracy and precision can reach 53.20% and 84.13% 13% without additional query budget, on par with state-of-the-ar MI attack which requires about 10 times more queries.
Acknowledgments and Disclosure of Funding
This work was supported by National Natural Science Foundation of China (Grant No: 62072390, 62102334), the Research Grants Council, Hong Kong SAR, China (Grant No: 15222118, 18919 15203120, 15226221, 15225921, and C2004-21GF). and a Huawei research grant (TC2020083100)
References
[1] Amazon
machine wetioo-eeaannnnngnen arained-modele http://////sss.aa..ooooo//aaaaaa
aws marketplace.
[2] Huawei cloud http:///ww///oeeoees////////////
[3] Huawei cloud modelarts.
10

---

[4] Sören Auer. Christian Bizer, Georgi Kobilarov, Jens Lehmann, Richard Cyganiak, and Zachary Ives. Dbpedia: A nucleus for web of open data. In The semantic web, pages 722-735. Springer, 2007.
[5] David Berthelot, Nicholas Carlini, Ian J. Goodfellow, Nicolas Papernot, Avital Oliver, and Colin Raffel. Mixmatch: A holistic approach to semi-supervised learning. In Advonces Neural Information Processing Systems 32: Annual Conference on Neural Information Processing Systems 2019, NeurlPS 2019, December 8-14, 2019, Vancouver, BC, Canada, pages 5050-5060, 2019. https://proceedingg www.ieerip..... neurips.c.) cc// paper hoper/2019//aaa http:///ccccceeeee6eeee/eeeeeeee -btpe//eoabbbbeeeeaaaaaaaaaaaa.. Awstract. html
[6] Varun Chandrasckaran, Kamalika Chaudhuri, Irene Giacomelli, Somesh Jha, and Songbai Yan. Exploring connections between active learning and model extraction. In 29th USENIX Security Symposium (USENIX Security 20), pages 1309-1326, 2020.
[7] Lingwei Chen, Yanfang Ye, and Thirimachos Bourlai. Adversarial machine learning in malware detection: Arms race between evasion attack and defense. In 2017 European Intelligence and Security Informatics Conference (EISIC), pages 99-106, 2017.
[8] Christopher A Choquetee-hooo, Florian Tramer, Nicholas Carlini, and Nicolas Papernot. Label-only membership inference attacks. In International Conference on Machine Learning, pages 1964-1974. PMIR, 2021.
[9] Patryk Chrabaszez, Ilya Loshchilov, and Frank Hutter. A downsampled variant of imagenet an alternative to the cifar datasets, arXiv preprint arXiv: 788.98 2017.
[10] Jacson Rodrigues Correia-Silva, Rodrigo F Berriel, Claudine Badue, Alberto F de Souza, and Thiago Oliveira-Santos. Copyeat enn: Stealing knowledge by persuading confession with random non-laheled data, 2018 International Joint Conference on Neural Networks (IJCNN), pages 1-8, IEEE, 2018.
[11] Gianna M Del Corso, Antonio Gulli, and Francesco Romani. Ranking a stream of news. Proceedings of the 14th international conference on World Wide Web, pages 97 106, 2005.
[12] Timothy Dozat. Incorporating nesterov momentum into adam. International Conference on Learning Representations (ICLR) workshop, 2016.
[13] Rong Du, Qingqing Ye, Yue Fu, and Haibo Hu. Collecting high- sigh-dimensional and corelation-constrainn data with local differential privacy. In 2021 18th Annual IEEE International Conference on Sensing, Communication, and Networking (SECON), pages 1-9, IEEE, 2021.
[14] Melanie Ducoffe and Frederic Precioso. Adversarial active learning for deep networks: margin based approach. arXiv preprini arXiv:1802..0988 09841, 2018
[15] Pierre Foret, Ariel Kleiner, Hossein Mobahi, and Behnam Neyshabur. Sharpness aware minimization for efficiently improving generalization. International Conference on Learning Representations, 2021. httts://openrevizn
[16] Mau Fredrikson. Somesh Jha, and Thomas Ristenpart. Model inversion attacks that exploit confidence information and basic countermeasures. In Proceedings of the 22nd A2n ACM SIGSAC Conference on Computer and Communications Security, pages 1322-1333, 2015.
[17] Mingfei Gao, Zizhao Zhang, Guo Yu, Sercan Ö Ank, Larry S Davis, and Tomas Pfister, Consistency- semi-supervised active learning: Towards minimizing labeling cost. In European Conference on Computer Vision (ECCV). pages 510-526. Springer, 2020.
[18] Xucluan Gong, Yanjiao Chen, Wenbin Yang. Guanghao Mei, and Qian Wang. Inversenet: Augmenting model extraction attacks with training data inversion. JICAI, 2021.
[19] Ian Gondfellow, Jonathon Shlens, and Christian Szegedy. Explaining and hamessing adversarial examples. ICLR, 2015.
[20] Lars Kai Hansen and Peter Salamon. Neural network ensembles. IEEE transactions on pattern analysis and machine intelligence, 12(10):993 10000, 1990.
[21] Seira Hidano, Takao Murakami, and Yusuke Kawamoto. Transmia: Membership inference attacks using transfer shadow training. arXiv preprint 2xXi:2001.14661. 2020.
[22] Haibo Hu, Jianliang Xu, Xizhong Xu, Kexin Pei, Byron Choi, and Shuigeng Zhou. Private search on key- value stores with hierarchical indexes. In 2014 JEEE 30th International Conference on Data Engineering, pages 628 639. IEEE, 2014.
11

---

[23] Matthew Jagielski, Nicholas Carlini, David Berthelot, Alex Kurakin, and Nicolas Papernot. High accuracy and high fidelity extraction of neural networks. In 29th USENIX Security Symposium (USENIX Security 20). pages 1345-1362, 2020.
[24] Hengrui Jia, Christopher A Choquette-Choo, Varun Chandrasekaran, and Nicolas Papernot. Entangled watermarks as defense against model extraction. In 30th USENIX Security Symposium (USENIX Security 21). pages 1937-1954, 2021.
[25] Rie Johnson and Tong Zhang. Deep pyramid convolutional neural networks for text categorization. In Regina Barzilay and Yen Kan, editors, Proceedings of the 55th Annual Meeting of the Association for Computational Linguisties, ACL 2017, Vancouver. Canada, July 30 August Volume I: Long Papers. pages 562-570, 570, Association for Computational Linguistics, 2017. https://dos oxg/10. 18653/v1/
P17 1052
[26] Mika Juuti, Sebastian Szyller. Samuel Marchal, and N. Asokan. Prada: Protecting against dnn model stealing attacks. In 2019 IEEE European Symposium on Security and Privacy (EuroSP). pages 512-527, 2019.
[27] Sanjay Kariyappa, Atul Prakash, and Moinuddin K Qureshi. Maze: Data-free model stealing attack using zeroth-order gradient estimation. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 13814 13823, 2021.
[28] Alex Krizhevsky, Geoffrey Hinton, et al. Learning multiple layers of features from tiny images. 2009.
[29] Alex Krizhevsky, Ilya Sutskever, and Geoffrey E Hinton. Imagenet classification with deep convolutional neural networks. Advances in neural information processing systems, 25:1097-1105, 2012.
[30] Klas Leino and Matt Fredrikson. Stolen memories: Leveraging model memorization for calibrated white- box memhership inference. 29th {USENIX) Security Symposium ([USENIX} Security 20), pages 1605-1622, 2020.
[31] David D Lewis and William A Gale. A sequential algorithm for training text classifiers. In SIGIR pages 1-1 Springer, 1994.
[32] Yuval Netzer, Tao Wang, Adam Coates, Alessandro Bissacco, Bo Wu, and Andrew Y Ng. Reading digits in natural images with unsupervised feature learning. 2011.
[33] Tribhuvanesh Orekondy, Bernt Schicle, and Mario Fritz. Knockoff nets: Stealing functionality of black-box models. In 2019 IEEF/CVF Conference on Computer Vision and Pattern Recognition (CVPR), pages 4949-4958, 2019.
[34] Soham Pal, Yash Gupta, Aditya Shukla, Aditya Kanade, Shirish Shevade, and Vinod Ganapathy. Activethief: Model extraction using active learning and unannotated public data. In AAAI Conference Artificial Intelligence, volume 34, pages 865-872, 2020.
[35] Nicolas Paperot, Patrick McDanicl, Ian Goodfellow, Somesh Jha, Z Berkay Celik, and Ananthram Swami. Practical black-box attacks against machine learning. In Proceedings of the 2017 ACM on Asia conference on computer and communications security, pages 506-519, 2017.
[36] Luis Perez and Jason Wang. The effectiveness of data augmentation in image classification using deep learning. arXiv preprint 17xiv:1712.0455 2017.
[37] Alexandre Sablayrolles, Matthijs Douze, Cordelia Schmid. Yann Ollivier, and Hervé Jégou. White-box vs black-box: Bayes optimal strategies for membership inference. In International Conference on Machine Learning, pages 5558-5588 0067-5567717 PMLR, 2019.
[38] Mehdi Sajjadi, Mehran Javanmardi, and Tolga Tasdizen. Regularization with stochastic transformations and perturbations for deep semi- cum-suupereieed learning. Advances in neural information processing systems, 29: 167-1171, 2016.
[39] Ahnued Salem, Yang Zhang, Mathias Humbert, Pascal Berrang, Mario Fritz, and Michael Backes. MI-leaks: Model and data independent membership inference attacks and defenses on machine learning models, 26th Annual Network and Distributed System Security Symposium (NDSS), 2018.
[40] Ozan Sener and Silvio Savarese. Active learning for convolutional neural networks: A core-set approach. In International Conference on Learning Representations (ICLR), 2018. https:// net. forum? hauummid-Hallkkk
12

---

[41] Reza Shokri, Marco Stronati, Congzheng Song, and Vitaly Shmatikov. Membership inference attacks against machine learning models. In 2017 IEEE Symposium Security and Privacy (SP), pages 3-18. IEEE, 2017.
[42] Samrath Sinha, Sayna Ehrahimi, and Trevor Damell.. Variational adversarial active learning. In 20/9 IEEF/CVF International Conference on Computer Vision (ICCV), pages 5971-5980, 2019.
[43] Florian Tramèr, Fan Zhang, Ari Juels, Michael K Reiter, and Thomas Ristenpart. Stealing machine learning models via prediction apis. In 25th USENIX Security Symposium (USENIX Security 16), pages 601-618, 2016.
[44] Jean- Can--Bapistt Truong, Pratyush Maini, Robert J Walls, and Nicolas Papemot. Data-free model extraction. In Proceedings of the IEEE/CVF Conference on Computer Vision and Puttern Recognition, pages 477 4771-4780. 2021.
[45] Yaqing Wang, Quanming Yao, James T. Kwok, and Lionel M. Ni. Generalizing from few examples: survey on few-shot learning. ACM Comput. Surv. 53(3), June 2020. htttps///do..org/100 1145/ 3386252
[46] Bang Wu, Xiangwen Yang. Shirui Pan, and Xingliang Yuan. Model extraction attacks on graph neural networks: Taxonomy and realization. CoRR, 2020. https://aarii .rg/ wawgg/bbs/0010 12751
[47] Samuel Yeom, Irene Giacomelli, Matt Fredrikson, and Somesh Jha. Privacy risk machine learning: Analyzing the connection to overfitting. In 2018 IEEE 31 Computer Security Foundations Symposium (CSF), pages 268-282, 2018.
[48] Yuheng Zhang, Ruoxi Jia, Hengzhi Pei. Wenxiao Wang. Bo Li, and Dawn Song. The secret revealer: Generative model-inversion attacks against deep neural networks. In IEEE/CVF Conference Computer Vision and Pattern Recognition (CVPR), pages 253 261, 2020.
[49] Shuai Zhao, Liguang Zhou, Wenxiao Wang, Deng Cai, Tin Lun Lam, and Yangsheng Xu. Splitnet: Divide and co-training. CoRR, 2020. hattss://arxii org/ abs 2als/201 2211 14660
[50] Zhong- Qiu Zhao, Yiu-ming Cheung, Haibo Hu, and Xindong Wu. Corrupted and occluded face recognition via cooperative sparse representation. Pattern Recognition, 56:77 87, 2016.
[51] Huadi Zheng, Haibo Hu, and Ziyang Han. Preserving user privacy machine learning: local differential privacy or federated machine learning? IEEE Intelligent Systems, 35(4):5-14, 2020.
[52] Huadi Zheng, Qingqing Ye, Haibo Hu, Chengfang Fang, and Jie Shi. Bdpl: A boundary differentially private layer against machine learning model extraction attacks. In European Symposium on Research in Computer Security, pages 66-83. Springer, 2019.
[53] Huadi Zheng, Qingqing Ye, Haibo Hu, Chengfang Fang, and Jie Shi. Protecting decision boundary of machine learning model with differentially private perturbation. IEEE Transactions on Dependable and Secure Computing, 2020.
Checklist
1. For all authors...
(a) Do main claims made the abstract and introduction accurately reflect the paper's contribu- tions and scope? [Yes]
(b) Did you describe the limitations of your work? [Yes] See Section [5.4]and and the discussion about
potential defenses in Section (5.32
(c) Did you discuss any potential negative socictal impacts of your work? [Yes] See Scction and
Scction 5.6
(d) Have you read the ethics review guidelines and ensured that your paper conforms to them? [Yes]
2. If you are including theoretical results...
(a) Did you state the full set of assumptions of all theoretical results? [Yes] See Appendix C
(b) Did you include complete proofs of all theoretical results? [Yes] See Appendix C
If you ran experiments...
(a) Did you include the code, data, and instructions needed to reproduce the main experimental results (either in the supplemental material or as URL)? [Yes] See Section
13

---

(b) Did specify all the training details (e.g.. data splits, hyperparameters. how they were chosen)?
YYs] See Section 5.1 Section 5.2 Section Section 5.6 Appendix
(c) Did you report error bars (e.g.. with respect to the random seed after running experiments
multiple times)? [Yes] See Figure Figure in Appendix
(d) Did you include the total amount of compute and the type of resources used (e.g.. type of GPUs, internal cluster, or cloud provider)? [Yes] See Appendix [E.2]
4. If you are using existing assets (c.g., code, data, models) or curating/relcssing new assets...
(a) your work uses existing assets, did you cite the creators? [Yes]
(b) Did you mention the license of the assets? [Yes]
(c) Did you include any new assets either in the supplemental material URL? [Yes] See
Section
(d) Did you discuss whether and how consent obtained from people whose data you're ing/curating? [N/A] The data sets that used are public.
(e) Did you discuss whether the data you using/curating contains personally identifiable informa- tion or offensive content? [N/A]
5. If you used crowdsourcing or conducted research with human subjects...
(a) you include full text of instructions given to participants and screenshots, if applicable?
[N/A]
(b) Did you descrihe any potential participant risks, with links to Institutional Review Board (IRB) approvals, if applicable? [N/A]
(c) Did you include the estimated hourly wage paid to participants and the total amount spent on participant compensation? [N/A]
14


Reviews:
[{"review_id": "jRUKkhqXQWl", "reviewer": "Reviewer_cjTV", "summary": "In this paper, the authors show that ME and MI can reinforce\neach other through a chained and iterative reaction, which can significantly boost\nME attack accuracy and improve MI by saving the query cost.Al", "questions": "Why are there so many references?", "limitations": "N/A", "rating": 7, "confidence": 4, "soundness": 3, "presentation": 3, "contribution": 3, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "Pros:\n\nPaper is well organized\n\nPaper is written in Good English\n\n\nCons:\n\nRelated work should be moved to the beginning\n\nMethodology and algorithmic content is missing\n\nText in Figures is small (graphs)", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {"ethics_review_area": ""}, "extra_text": {}}, {"review_id": "gt9XSYP_JOr", "reviewer": "Reviewer_LJzA", "summary": "This paper proposes a new pool-based active model extraction (PAME) attack. The key observation is that in PAME, the data pool that the attacker has access to may be different from the data samples used to train the model, which can hurt attack performance. Based on this observation, the paper proposes an iterative attack framework where a membership inference (MI) attack is utilized to identify training samples in the data pool, which is used to improve the extracted model through active learning. In turn, the extracted model can be used to improve the accuracy of membership inference. It is shown through experiments that MExMI outperforms ActiveThief [33], a PAME attack, in terms of model fidelity and obtains similar MI performance as an existing MI attack [38] that has an unlimited query budget. ", "questions": "What is the ideal ME attack considered in the experiments? According to Table 2, MExMI can sometimes outperforms the ideal ME attack. How is that possible?\n\nTable 3 shows that even when the data pool does not contain any training samples, MExMI still outperforms ActiveThief, although the improvement is rather limited. Is the improvement because of MI or something else?\n\n", "limitations": "It would help to briefly discuss potential defenses against the proposed attack. ", "rating": 5, "confidence": 4, "soundness": 2, "presentation": 3, "contribution": 3, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "Both ME and MI have been intensively studied recently. The observation that the attacker may not have access to the victim's training data in ME attacks is not new. But the idea of letting ME and MI reinforce each other is new. To achieve this, the paper has adapted shadow-model MI attacks to the setting when there is a query budget and when queried samples are not drawn from the same distribution as the victim's training samples. Although these adaptations are mainly straightforward, they are clearly explained and seem effective according to the experiment results. That being said, I have two concerns about the paper. \n\nFirst, the paper considers a relatively simple setting when no defense is applied for either ME or MI. Given that various defenses have been proposed for both attacks, it is important to understand if ME and MI can still reinforce each other effectively in the presence of defenses. In particular,  it is unclear if the adaption proposed for shadow-model MI still works in the presence of defenses against MI. Note that without defenses, ActiveThief already obtains a very good ME attack performance. Although the experiments show that MExMI can do even better, the results are not very impressive from a practice point of view. It would be much more interesting if similar results could be shown when state-of-the-art defenses are applied. \n\nSecond, I also have some doubts about the exact role that MI plays in MExMI. According to the introduction and the example in Table 1, the MI component is used to infer training samples from the data pool and best utilize them. However, it is claimed in the experiments that MExMI does not require the data pool to contain training samples. Table 3 shows that even when the data pool does not contain any training samples, MExMI still outperforms ActiveThief, although the improvement is rather limited. Is the improvement because of MI or something else? Note that the MExMI attack becomes less useful from a practice point of view if the MI component only helps when the data pool contains a certain amount of training data, as the attacker typically has no access to true training samples in ME attacks. The proposed attack would be more useful if the MI component could help identify data samples that can best assist ME even if they are not exactly the same as training samples. ", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {"ethics_review_area": ""}, "extra_text": {}}, {"review_id": "_HyGOrDXDdj", "reviewer": "Reviewer_eaMb", "summary": "In this paper, the authors propose a method to boost model extraction attacks. Specifically, they propose to use MIA (membership inference attack) to identify homogeneous samples in the adversary data pool and make full use of them to improve the effectiveness of model extraction.", "questions": "Please refer to Cons.", "limitations": "Please refer to Cons.", "rating": 5, "confidence": 4, "soundness": 3, "presentation": 3, "contribution": 3, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "Pros:\n\n1. The authors proposed a method to boost model extraction attacks under the scenario that the attacker has no knowledge of the victim training dataset.\n\n2. The idea of using membership attacks to boost model extraction attacks is novel.\n\nCons:\n\n1. The author mentioned that the proposed attack does not require the pool to contain training samples. However, the key to the proposed attack is to use MIA (membership inference attack) to infer the victim's training data (or the distribution of the training data) and use these data to perform a model extraction attack. If the pool does not contain the victim training data, the MIA success rate will be low, and the effect of boosting model extraction boost is limited. \n\n2. The attack setting about the adversary pool is impractical. In practice, the adversary always cannot guarantee that the adversary data pool contains the victim training data (especially containing all the victim training data). The assumption that the victim training data accounts for 1/3 of the adversary pool is not reasonable. The attack process is more like the adversary first finds the victim training data through MIA and trains the surrogate model with it.\n\n3 The author only considered ActiveThief as the baseline. Although the proposed method is aimed at boosting the model stealing attack based on active learning, it should also be compared with other types of model stealing attacks (such as PRADA [1], KnockoffNet [2], etc.) in query efficiency.\n\n[1] Mika Juuti, Sebastian Szyller, Samuel Marchal, and N Asokan. Prada: protecting against dnn model stealing attacks. In Proceedings of Euro S&P, pages 512–527, 2019.\n[2] Tribhuvanesh Orekondy, Bernt Schiele, and Mario Fritz. Knockoffnets: Stealing functionality of black-box models. In Proceedings of CVPR, pages 4954–4963, 2019.\n", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {"ethics_review_area": ""}, "extra_text": {}}, {"review_id": "XGOJd40uXh5", "reviewer": "Reviewer_N4YB", "summary": "This paper proposes a new framework that combines model extraction (ME) and membership inference attack methods such that, both attacks can mutually reinforce each other in the attack process, and eventually lead to efficient and effective ME and MI attacks. The experimental results show that, using the combined approach, the fidelity of the attack can be enhanced greatly compared to baseline ME attacks, while the MI in the mean time maintain relatively comparable performance compared to existing MI attacks that uses much more queries. ", "questions": "The weakness part in the review lists up the requested changes and clarifications. An additional question for the authors is: (ME and MI attacks are not in my area) are the considered baselines in this paper reflect the state-of-the-art? It seems the adopted ones in this paper are rather \"old\", considering how hot the area is.  ", "limitations": "The limitations of the paper and the possible ways of improvements are outlined in the weakness section. ", "rating": 7, "confidence": 3, "soundness": 3, "presentation": 2, "contribution": 3, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "The idea of combining ME and MI attack seems to be novel and interesting, and the empirical results also mostly support the claims made in the paper. Overall, I like the originality of the idea presented in this paper and the presentation is also clear. However, I also have the following concerns:\n1) it is better to explain how much of a fidelity improvement is considered significant in practical applications. For example, in Table 1, the author listed the different fidelity scores when the level of training data homogeneity varies, and to me, all the fidelity scores are very high and the improvements may not be that significant. Explaining this will also help readers to better understand significance of the results.\n2) I am not sure how accurate it is claim that training shadow MI model [40] requires training data of same size as the victim's training set. Their method is definitely applicable even if the shadow model training data only contains subset of the victim's training data. Therefore, this part should be stated clearly.\n3) for MI attack results, it is extremely difficulty to properly compare the baseline MI attack [38] and the MExMI approach. The term \"on par\" is vague especially when we use \"precision\" and \"recall\" to compare them individually. Using some unified metrics such as F1-score, AUC scores or simply the MI accuracy might give us better pictures. Note that, MexMI does not have to outperform existing MI attacks that require significantly more resources. \n4) the baseline methods for both ME and MI attacks might not be sufficient. For example, the MI attack [40] is also not included in the paper for baseline comparisons. In general, I believe the authors can first compare MExMI against various existing baselines and then do an ablation study on the impact of each module in MExMI.  ", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {"ethics_review_area": ""}, "extra_text": {}}]

Author rebuttal:

Decision: Accept