MeaeQ: Mount Model Extraction Attacks with Efficient Queries
Authors: Chengwei Dai
Minxuan Lv
Kun Li
Wei Zhou
Venue: EMNLP 2023 Main (2023)
Keywords: Model Extraction Attack; Efficient Query Sampling Strategy; Limited Query Budgets
TL;DR: We propose a effective and low-cost method for model extraction attacks in NLP where attackers can steal victim models by repeatedly querying the open APIs.

Abstract: We study model extraction attacks in natural language processing (NLP) where attackers aim to steal victim models by repeatedly querying the open Application Programming Interfaces (APIs). Recent works focus on limited-query budget settings and adopt random sampling or active learning-based sampling strategies on publicly available, unannotated data sources. However, these methods often result in selected queries that lack task relevance and data diversity, leading to limited success in achieving satisfactory results with low query costs. In this paper, we propose MeaeQ (Model extraction attack with efficient Queries), a straightforward yet effective method to address these issues. Specifically, we initially utilize a zero-shot sequence inference classifier, combined with API service information, to filter task-relevant data from a public text corpus instead of a problem domain-specific dataset. Furthermore, we employ a clustering-based data reduction technique to obtain representative data as queries for the attack. Extensive experiments conducted on four benchmark datasets demonstrate that MeaeQ achieves higher functional similarity to the victim model than baselines while requiring fewer queries.

MeaeQ: Mount Model Extraction Attacks with Efficient Queries
Chengwei Dail., Minxuan Lv Kun Li"; Wei Zhou', Instittte of Information Engineering, Chinese Academy of Sciences School of Cyber Security, University of Chinese Academy of Sciences [daichengwei, lvminxuan, likun2, zhouwei Wiiie.e@@ii... a cn
Abstract
In black-box scenarios, wherein the architecture and training data of the victim model remain un- known, the primary concern for attackers is to con- sider how to design high-quality queries with limited query budgets. This is crucial since fre- quent API calls not only result in considerable costs (Correia-Silve al., 201 8) but also carries the po- tential of triggering the victim model's defense mechanisms (Papernot al., 2017: Juuti al., 2019; Zhang al., 2021), thereby diminishing the effectiveness of the attack.
We study model extraction attacks in natu- ral language processing (NLP) where attack- ers aim to steal victim models by repeatedly querying the open Application Programming Interfaces (APIs). Recent works focus on limited-query budget settings and adopt ran- dom sampling or active marning-basse sam- pling strategies on publicly available, unanno- tated data sources. However, these methods often result in selected queries that lack task relevance and data diversity, leading to lim- ited success in achieving satisfactory results with low query costs. In this paper, we pro- pose MeaeQ (Model extraction attack with efficient Queries), a straightforward yet effec- tive method to address these issues. Specifi- cally, we initially utilize zero-shot sequence inference classifier, combined with API ser- vice information, to filter task-relevant data from public text corpus instead of prob- lem domain-specific dataset. Furthermore, we employ a clustering- -ased data reduction tech- nique to obtain representative data queries for the attack. Extensive experiments con- ducted four benchmark datasets demonstrate that MeaeQ achieves higher functional similar- ity to the vietim model than baselines while requiring fewer queries. Our code available at https::/tteee thtbb Hetn/CCW-DDMMeeee
Some studies (Orekondy al., 2019; Xu et al., 2022; Karmakar and Basu, 2023) sample queries from annotated data associated with the victim model training data, e.g., using BBC News data to extract models trained on AGNews (Zhang et al., 2015), which deviates from the assumption of black-box model extraction, where the training data distribution of the victim model is unknown. Therefore, recent research has focused on leverag- ing publicly available unannotated data sources for model extraction attacks. The pioneering work by Pal et al. (2020) explores the utilization of these data sources and proposes an active learning- based sampling strategy that dynamically adjusts query selection based on self-feedback. Likewise, Kr- ishna al. (2020) presents two query construction methods, one involving sentences combined with random words and the other involving actual sen- tences randomly sampled from Wikipedia. Despite the demonstrated efficacy of these methods in their individual studies, we experimentally find that the queries sampled by these approaches often suffer from category imbalance. For instance, when ap- plying the random strategy to an online hate speech detection API, the ratio of positive to negative sam- ples becomes skewed high as 30 1. Such notable imbalance in sample distribution presents challenges for model training, especially when con- sidering low query costs. We attribute this issuc to task-irrelevant text content and insufficient data
1 Introduction
The adoption of Machine Learning as Service (MLaaS) via APIs has introduced new secu- rity challenge known as model extraction attacks (Tramèr et al., 2016). Attackers repeatedly access the API to acquire outputs by utilizing meticulously crafted inputs (queries) and subsequently train a lo- cal model based the gathered input-output pairs. This attack aims to obtain extracted model that closely approximates the performance of the victim model, thereby posing substantial threat to the intellectual property of the model owner.
Kun Li is the corresponding author.

---

diversity within the queries. In this paper, we propose MeacQ (Model extraction attack with efficient Queries), straight- forward yet effective method to address these is- sues. MeaeQ comprises two modules: Task Rele- vance Filter (TRF) and Data Reduction based on Clustering (DRC). The TRF module aims to select data that is highly relevant to the task. To achieve this, we utilize pre-trained zero-shot sequence inference classifier, combination with API ser- vice information, infer the entailment relations between the pre-designed prompt and all actual sentences extracted from a publicly available text corpus. The second module, DRC, is designed to mitigate information redundancy within the query pool filtered by the TRF module. To accomplish this, DRC initially extracts embeddings from all texts in the query pool and then employs clus- tering method to create multiple clusters. subse- quently selects the data nearest to the centroid of each cluster as the ultimate query. Finally, we send these queries to the victim model and then use the outputs as labels to fine-tune our local model. Extensive experiments on simulated victim mod- els demonstrate that the model extracted by MeacQ exhibits higher functional similarity to the victim model than the baseline methods on four bench- mark datasets. Furthermore, we validate the gener- alizability of MeacQ across diverse model architec- tures. In-depth analyses reveal the significant con- tributions of both the TRF and DRC in the model extraction attack. The primary contributions of this paper can be summarized as follows:
Type of Victim Model. Some works focus on ma- chine learning models (Tramèr et al., 2016; Chan- drasekaran et al., 2020) while others concentrate on neural networks (Shi et al., 2017; Milli et al., 2019), but most which pay attention to the com- puter vision (CV) (Orekondy et al., 2019; Wang et al., 2020; Zhou et al.. 2020; Gong et al., 2021; Wang et al., 2022). In contrast, our study targets pre-trained language models (PLMs) such as BERT same Krishna al. (2020).
Type of API Feedback. Several studies consider using the complete probability vectors of the victim model on all or top-k classes returned by the API as feedback for the query. which is less practical in public API (Tramèr et al., 2016; Orekondy al., 2019). Instead, following (Pal al., 2020; Wang et al., 2022), we focus on the most challenging scenario where the APl only provides the predicted hard label.
Query Source. Most studies utilize query sources derived from public problem domain- related datasets (Papernot et al., 2017: Xu et al., 2022; Karmakar and Basu, 2023). In contrast, fol- lowing Pal et al. (2020), we use a large public text corpus as the query source, ensuring no overlap with the private dataset of the victim model.
Query Sampling Strategy. In CV, query sampling strategies can be broadly categorized into three groups. Some studies employ reinforcement learn- ing techniques (Orekondy et al., 2019), some utilize adversarial example generation methods (Papernot et al., 2017; Juuti et al., 2019; Yu et al., 2020), and some employ model inversion techniques (Gong et al., 2021). However, applying these methods to NLP is challenging due to the discrete nature of text data, in contrast to the continuous nature of image data. In NLP, the earliest work comes from Pal al. (2020), who adopt the active learning method it- eratively sample queries. Krishna et al. (2020) con- struct a nonsensical sequence words or randomly sample actual sentences as queries. Different from them, we improve the sampling strategy by task driving and information redundancy minimization. thus better playing the value of the limited query.
We employ zero-shot sequence inference classifier, combined with API service infor- mation to filter data with high task relevance.
We design data reduction technique based on the clustering method to alleviate the infor- mation redundancy problem in text sampling.
Extensive experiments confirm the effective- ness of our method in model extraction attacks at low query cost. Additionally, the queries sampled by our approach enhance the stability of the extracted model during training.
3 Methodology
2 Related Work
In this section, present our method for NLP model extraction attacks. We first formalize the problem and then elucidate how to sample queries with high task relevance and low information re- dundancy.
We introduce related work from four perspectives of model extraction attacks, including the type of victim model, the type of API feedback, the query source, and the strategy of query sampling.

---

Data Reduction
Task Relevance Filter
Model Extraction Attack
based on Clustering
This hate speech.
API Service:
Public Unannotated Text Corpus

 premise 
<ss> hypothesis </5>
Detect hate speech text
Extract sentence
Sequence Inference
embeddings from
Black Box
Classifier
Victim Model
fine-tuned the private
Select all sentences
Run clustering
Neutral
Entailment
Contradiction
algorithm
Input
API Feedback Hate
Call him a racist in signs if you see him...
Original Query Set a.
Task-related Query Set Gg
Reduced Query Set Q
Fine-tune
Extracted Model
data point
selected data point
cluster
Figure I: Overview of MeaeQ for NLP model extraction attacks. The attackers first build an original query set Q from large text corpus. Then the attackers apply the Task Relevance Filter on Qo to get task-related query Qg. Subsequently, the attacker exploits the Data Reduction based on Clustering to reduce Qg to Q, Finally, the attacker samples the queries from Q, sends them to the API, and then uses the outputs as labels to fine-tune their model such BERT (Devlin al., 2019).
3.1 Problem Formulation
where I (.) is the indicator function. The cal- culation of Accuracy is equivalent to that of Agreement, with the only difference being the substitution of fy with the ground truth label.
Let fu denote the victim model (0,, are all parame- ters included), which represents a black-box API providing services for the task T. We assume the API only returns the predicted hard label rather than probability scores. f. is trained on private dataset that is inaccessible the public. The attackers con- struct query set and utilize the API obtain out- puts by sending sampled queries. This process gen- erates the attacker's dataset (x fo ( where l denotes the numbers of query. Then attackers use the dataset to train their own model fa (0, repre- sents all parameters included). For evaluation. Krishna et al. (2020), namely Accuracy and Agreement. Accuracy measures the prediction accuracy of far while Agreement assesses the functional similarity between Sa and f,. Both metries are calculated on the private test dataset Dtest = (x, v) x с Xtest ; ytest) } of the victim model. Agreement is defined as:
Our goal is construct high-quality query set with a given budget train a thief model with per- formance close to the victim model. The selection of queries directly impacts the parameters of fax so we can formally express the objective as:
0 arg max Agreement (Sor fa)
(2)
0.
where B is fixed and hidden.
adopt the same metrics
3.2 Overview of Meae
In this subsection, we provide an overview MeaeQ. The attackers start by sampling all actual sentences from a text corpus to initialize the origi- nal query set Q {m:) where p is the number of actual sentences in the corpus. Then they utilize sequence inference classifier to filter the input pairs that contain hypothesis (manually designed prompt) and a premise selected from Qo resulting in a task-related query set Qg. Next, the attackers employ a clustering- based data reduction technique
Agreement (ferfa)
Σ 2tet
= (x)
XTnai
(1)

---

on Qg to obtain a query set with low information redundancy, denoted as Qr. Finally, the attackers perform model extraction attack using Qr. The overview MeaeQ is illustrated in Figure
E is the set of edges, we extract the sentence em- bedding of the samples z; E Qg as the vertex representation by the model g:
1/4 = Dembedding (r, 2)
(4)
3.3 Task Relevance Filter
Here, v is a vector of dimension d. We define the edge weights E.j the cosine similarity distance between the vertices vs and Uz:
Inspired by the study of Yin et al. (2019) who pro- pose using a PLM trained on natural language infer- ence task as zero-shot sequence classifier for text classification, we introduce the Task Relevance Fil- ter (TRF) combined with API service information filter queries related to the target task. The TRF consists of sequence inference classifier, which is a pre-trained language model trained on the MNLI dataset (Williams et al., 2018). We denote this clas- sifier as whose input pairs are premise and a hypothesis. We design prompt h as one of the in- puts (hypothesis) based on the service information of the task 7 and take the actual sentence z; from Qo as another input (premise) the model g for reasoning about the relationship between them. For example, the task is to detect if text contains hate speech, we design the prompt as "This is a hate speech" to filter queries that resemble hate speech. Finally. we obtain the probability vector of logical relation classification result of the ac- tual sentence Hg with prompt h by the output of the model:
Cijj 1 sim cos (v: 09))
4 1/
=
|1,1 ||0/1
(5)
Ogl This results in V im and E E= {
19g
ij=1 The objective of this module is to select the most representative samples from Qg to form a reduced query set Q, The selection problem can be viewed as finding subgraph G that maximizes the sum of edge weights:
G, arg max G.CG
Σ C.
Ci.
(6)
However, this problem is NP-hard problem (already proved by He al. (2021)). Therefore, we design an approximate solution that achieves the sample selection within a reasonable time. The steps are as follows:
(x = g
(3)
Step 1 First, we apply a clustering algorithm on the query set Qg for iterations. The number of clusters l is set as Qr/, which represents the num- ber of queries used to access the victim model's API. After clustering, we obtain set of k clusters CLSTR {clstr;) and centroids CTRID = [ctrid:] where clstr; represents the set of samples in the i-th cluster, and ctrid, RRd is the centroid of the i-th cluster.
where P is the probability at relationship label y E [neutral, entailment, contradiction).
To ensure the selection of high-quality task- related queries, the Task Relevance Filter incor- porates a filtering mechanism. We simply de- sign this mechanism as follows: pf pottaimmnt is greater than or equal to a threshold e, the sam- ple n4 is retained, otherwise discarded. This op- eration is repeated until all samples in Q, are classified, resulting in task-related query set Qg {x, Vz, Qo p. eettaimmntt > e).
of clusters
Step For the i-th cluster clstr,, add sample point x. the reduced query set Q,.. satisfying the condition that af the closest to the centroid ctrid, within the cluster, i.e., the cosine similarity distance between them is the smallest:
3.4 Data Reduction based on Clustering
To reduce the information redundancy in the query set Qg. propose a Data Reduction technique based on Clustering (DRC). We transform the infor- mation redundancy problem into a graph-theoretic problem. Given a weighted undirected graph, G = CV where V is the set of vertices and 'Emprrically, we tind that prompt template "This sentence is about TOPICJ"" works well.
x arg min - simoos (a ctrid,) (7) ar Edstr,
Step 3 are traversed, completing the construction of Q,- The idea behind this approach that the clus- tering algorithm generates i clusters with large inter-cluster distances and small intra-cluster dis- tances. We then select the sample point closest to
Repeat step 2 l times until all clusters

---

its centroid within each cluster as candidate sam- ple. This strategy aims achieve an approximate maximum distance between all candidate samples. Time complexity analysis: Step 1 requires O (10glkt) kt) to perform clustering with é iterations in total. Each iteration involves searching the small- est distance within the cluster, resulting in to- tal time complexity equivalent to traversing the entire set Qg t times, with time complexity of O (IQglt). Therefore, the overall time complexity of DRC remains O (1Qgle kt).
query rates, which represent the proportion of the original dataset size. The threshold e in TRF and the number of iterations and 300 respectively. We employ the faiss library' (Johnson et al., 2019) to accelerate the vector re- trieval and use the k-means algorithm (MacQueen, 1967) for clustering. All experiments are repeated 10 times with different random seeds on single 32GB NVIDIA V100 GPU. More details about the hyperparameters can be found in Appendix C.
DRC are set as 0.95
4.2 Baselines
4 Experiments
We compare MeacQ with the following baselines: RS (Random Sampling) The RS is proposed by Krishna al. (2020) 4, which randomly samples real sentences from the WikiText-1033 corpus.
In this section, we conduct extensive experiments to evaluate the effectiveness of our method.
4.1 Experiment Settings
AL-RS The AL-RS a simple variant of active learning, which introduced by Pal et al. (2020). In each iteration, AL-RS utilizes a random strategy. The key difference between AL-RS and RS lies in the multiple rounds of sampling versus single round of sampling.
Datasets of Victim Model We train simulated victim models respectively for four tasks includ- ing a hate speech detection dataset: Hate Speech (de Gibert et al., 2018), a topic classification dataset: AG News (Zhang et al., 2015) and two sen- timent classification datasets SST-2 (Socher et al., 2013) and IMDB (Maas et al., 2011). Details about the data division and statistical information of these datasets can be found in Appendix B.
AL-US The AL-US is proposed by Pal al. (2020), which is also an active learning-based ap- proach. each iteration, the AL-US uses an uncer- tainty strategy that selects the top-k samples with the highest entropy value computed from the pre- dicted probability vector of the attackers' model.
Corpus and Prompts For query source, we use WikiText-103 103 corpus (Merity et al., 2017), ensuring that there is no overlap with the private datasets. We design prompts in TRF for each dataset. Since SST-2 and IMDB are both sentiment classification tasks for movie reviews, we use the same prompt "This is a movie review." " for both. For AG News, which focuses on news topics, we use the prompt "This is a news.". In the case of Hate Speech, we use the prompt "This is hate speech".
4.3 Main Results
In this subsection, we compare MeaeQ with the baselines on the simulated victim models trained on four datasets. The results are presented in Table and Table 6 (due to space limitations, all accuracy results are listed in Appendix A). Table 1, 6 show that MeaeQ consistently outperforms the baselines for all datasets with low query budgets, demon- strating its effectiveness. Especially in Group A, find that MeaeQ significantly outperforms the hop-performing baseline on three datasets, but with slightly smaller gains on AG News. We attribute this to the similarity in data distribution between Wikipedia and AG News, as many Wikipedia arti- cles can be considered as a form of news. We also notice that the baselines exhibit high standard deviation in performance, implying some noise in the sampled queries. In contrast, MeaeQ can sam- ple task-related and informative queries, leading
Implementation Details In our experiment, we use BERT Base the architecture for both the vic- tim model and the extracted model. Additionally, we explore the effectiveness of MeaeQ on differ- ent model architectures in section 4.4. The victim model is trained for 3 epochs and the extracted model is trained for 10 epochs. We select the best checkpoints based on the validation set. We utilize BART (Lewis et al., 2020) as the sequence in- ference c We use two evaluation metrics Agreement and Accuracy, as described in sub- section 3.1. For each dataset, we set up several groups with different query budgets expressed as
hhttps: /gi thub. wtt.hhh..oooe.eeeeeommmeeeeeeeee aacebookresearcchfaaaa
"They also propose query construction method in which random words form sentences, hut we experimentally find that this method does not work well.
hhttps: /aaggs:/g/gginagaa ace. ct/ co/facebook/ Bart-large-mnl

---

Hate Speech
SST-2
IMDB
AG News
Query Budget
Method
(x 1914)
(x == 67349)
(x 40000
(x 120000
RS
45.2 = 0.0 (45.2)
67.8 + 7.7 (80.7)
53.4 3.8 (63.1)
73.9 + 3,4 (79.6)
Group A
AL-RS
44.3 2.4 (45.2)
73.9 6.9 (85.9)
55.2 (64.8)
81.1 2.7 (84.5)
0.1 x 0.003 (191 201 120/ 360)
AL-US
53.2 - 7.7 (68.6)
74.1 : 8.1 (83.6)
56.3 1 4.4 (62.9)
76.3 + 7.4 (84.6)
MeaeQ
75.8 + 4.5 (79.7)
82.5 + 3,6 (86.7)
65.3 + 9.1 (79.8)
81.9 + 3.3 (86.0)
RS
45.1 = 0.1 (45.2)
76.6 - 6.4 (85.4)
59.8 + 7.8 (71.8)
74.0 = 10.4 (86.2)
Group B x 0.2/ x 0.005
AL-RS
46.0 = 3.5 (56.3)
81.6 + 4.2 (86.1)
59.0 + 7.5 (74.6)
82.4 + 3.8 (86.3)
AL-US
67.1 + 7.0 (77.6)
78.5 + 7.6 (85.9)
62.1 8.2 (76.5)
83.0 +: 3.7 (87.6)
(382/ / 335/ / 200 600)
MeaeQ
79.1
(81.4)
84.5
(87.0)
73.4
(78.6)
84.5 1.7 (87.8)
RS
47.7 + 5.2 (61.7)
78.6 :: 8.0 (87.5)
63.2 + 8.0 (73.5)
82.3 + 3.0 (86.7)
Group 0.3/ x 0.008 (574/ 536 320 960)
AL-RS
51.0 (68.0)
84.0 3.9 (88.4)
59.8
(69.2)
80.0 2.2 (82.2)
AL-US
78.7 6.4 (84.3)
82.8 = 5.2 (89.)
72.6 - 5.5 (78.5)
84.0 + 0,9 (85.7)
MeaeQ
81.0 = 2.6 (83.0)
86.5 = 1.5 (88.9)
79.9 + 2.0 (82.5)
85.3 + 1.8 (87.5)
Table Model extraction result (Agreement, %) under different query budgets. 0.1 x 0.2 x 0.3 query budgets are set for Hate Speech dataset. x 0.003 x 0.005 / 0.008 query budgets are set for others. The specific number queries for each dataset is shown in parentheses below the query budgets. The results presented in the table are the mean and standard deviation while the value is shown in green.
higher functional similarity and more stable perfor- mance, even with extremely low query budgets. To further validate the effectiveness of MeaeQ, we conduct experiments on more query budgets and present the results in Figure 3. We find that MeaeQ outperforms baselines in nearly all settings and exhibits lower standard deviations. In particu- lar, for SST-2 and IMDB, MeaeQ's performance at query budget x 0.008 comparable to or signifi- cantly surpasses the highest performance achieved by the baseline method at query budget x 0.02, When achieving the similar performance, MeaeQ reduces the query cost by more than half.
MeaeQ
AL-US
83.1
78.0
83.7
83.3
81.0
R
61.7
85.2
75.1
82.6
82.5
73.6
AL-RS
78.2
79.3
69.3
B
75.3
59.8
49.4
72.0
4.4 Cross Model Extraction
75.9
64.6
73.0
X 51.5
53.1
65.7
We explore the applicability of MeaeQ to differ- ent extracted victim model architectures, includ- ing BERT Base, RoBERTaRee (Liu al., 2019), and XLNet (Yang et al., 2019), as depicted in Figure 2 and Figure 7. It is evident that MeaeQ surpasses the baselines across different model architectures, demonstrating its effectiveness and robustness. Be- sides, we notice that the baselines exhibit good performance solely the matching model archi- tecture, as indicated by the darker color the diagonal of the heatmap. In contrast, MeaeQ con- sistently exhibits superior performance across dif- ferent architcctures, indicating its model-agnostic nature. In summary, MeaeQ can be successfully ap- plied to various model architectures for extraction, while maintaining exceptional performance.
B
R
X
8
X
Figure 2: Cross model extraction results (Agreement, %) on Hate Speech at query budget 0.5. The hori- zontal vertical axes represent victim extracted model architecture respectively. 'B', 'R', and 'X' represent BERT Base: RoBERTagge. and XL.Nel Darker colors represent larger values of Agreement.
4.5 Ablation Study
To better understand the two key modules in MeaeQ, compare MeaeQ with two variants, w/o TRF and w/o DRC, respectively where the corre- sponding modules are removed from MeaeQ. The results are shown Figure 4 and Figure 8. From the figures, MeaeQ apparently outperforms

---

2
R3
U2
AJU2
0.15
3.38
8.45
0.50
0.04
0011
0820
8.94
0.05
0012
0.81
0.03
0013
0.11
0.000
ALUS
ALLS
0.15
3.38
8.45
2.50
0.75
0.04
0.53
3.F2
0011
0600
133
0.83
6812
326
300
0.04
8.00
0012
0.015 6400
(a) Hate Speech
(b) SST-2
(c) IMDB
(d) AG News
Figure 3: Comparison between MeaeQ and the baselines at more query budgets. The first row represents the results of the Accuracy (%) on the four datasets while the second row represents the results of the Agreement (%)
the two variants, particularly terms of higher agreement and shorter error bars at low query bud- gets (e.g., x 0.003 / 0.005). These results highlight the importance of TRF and DRC, as they enhance the efficacy of extraction and its stability. More- over, MeacQ consistently outperforms w/o TRF across all query budgets, emphasizing the critical role of task-related queries screened by TRF which aligns the attackers' data distribution with the vic- tim model training data distribution.
85.00
Wis TRF wto DNC MeseQ
08.75
Arg Agrwormen PS
76.25
However, we also observe that as the query bud- get increases, the performance gap between w/o DRC and MeaeQ narrows, indicating decreas- ing effect of w/o DRC. We know that the query budget determines the number of clusters in DRC. As the query budget increases, the number of clus- ters gradually rises when the size of the query set screened by TRF remains constant, In the extreme case where the query budget equals the size of the candidate query set, the clustering algorithm becomes completely ineffective. Therefore, our method is particularly suitable for scenarios with low query budget. However, for higher query budgets, we recommend evaluating the specific circumstances before deciding whether to utilize DRC, as DRC may degrade into random sampling under the worst-case scenario.
Budget
Figure 4: Ablation study results (Agreement, %) on SST-2 dataset. The horizontal axes represent different query budgets. Error bars represent standard deviation.
victim model simulated on the hate speech detec- tion task using manual instructions to verify the effectiveness of our method in the autoregressive language model extraction. The instruction tem- plate is available in Table 2. We set up several types of extracted models with different parame- ters and architectures, including GPT-2 Radford et al. (2019), RoBERTa Liu et al. (2019). Com- prehensive results be found in Table 3. An analysis of the table reveals that our approach per- forms admirably, even when the victim model is ChatGPT-bass and the extracted model relies on an autoregressive framework (e.g., GPT-2). For instance, GPT-2smal achieves 79.5% % functional similarity (Agreement) and 75.3% test accuracy
4.6 Attack on ChatGPT-bbsee Victim Model In this subsection, conduct model extraction attacks on the ChatGPT-based (OpenAI, 2023)
'Specifically, we call the 3pt-3.5-turho Chat completions API for responses. The document page is https: https://platform..
openai, wwt. com//bocssmodeels/gt--33/

---

Task
Instruction
Hate Speech
You are now working as a hate speech detector. Next, I will give you [batch_size] sentences. You need to indicate whether it contains hate speech. it contains hate speech, output "Hate", otherwise output "Nohate". The required format output per line and the beginning of each output numbered sequentially. Sentences:
1. |sentence__
2. [sentence 2]
Table Instruction Template. The [batch_sine sizec in the instruction represents the number of queries, and [sentence__] 1], [sentence__2 21, etc. represent the specific query content.
(90.4% of the victim model). We also observe that the RoBERTa series models outperforms the GPT-2 series in this detection task, which is clear, as masked language models are better suited for extracting models in classification tasks, while autoregressive models are typically employed for extracting generative models.
by using the SST-2 training data, both of which outperform the use of WikiText-103, This discrep- ancy can be attributed to the similarity between the attacker's corpus and the training data distri- bution of the victim model, i.e., SST-2 and IMDB are both about movie reviews, resulting in a higher data correlation. This observation motivates the Task Relevance Filter in our method, it aims filter texts from the public unannotated corpus that closely resemble the data distribution of the task.
G 117M
RoBERTan GPT-2statte 345M
RoBERTaaage 355M
method
125M
43.7
40,2/43,9
55.6/
64.4 59.8
AL-RS
50.2/43.9
53.6/48.9
50.0/43.7
56.37 54.2
AL-US
40.2/43.9
50.6/45.6
60.7
73.6/ 76.6
5.2
Role of Data Reduction based on Clustering
MeaeQ
75.3 79.5
71.5/75.3
82.4
79.5/ 85.4
Table 3: Model extraction results (Accuracy / Agree- ment, %) on ChatGPT simulated on the Hate Speech dataset at query budget x 0.8 (about 1531 queries). The accuracy of ChatGPT on this task under zero-shot set- ting is 83. 83.26%. The data recorded in the table are the best results of ten retests.
To further explore the role of the clustering- data reduction technique, we qualitatively visualize the query sampled by DRC and RS on subset of WikiText- 103 with t-SNE (Van der Maaten and Hinton, 2008). Figure depicts the distribution of the sampled data. We can see that the data sam- pled by DRC displays a broader distribution with larger inter-point distances, whereas the data sam- pled by RS show some overlapping. This observa- tion demonstrates the capability of DRC sample data based on their information and effectively re- duce redundancy.
5 Analysis
In this section, we provide further analysis of the TRF module and DRC module in our proposed method for model extraction attacks.
5.1 Impact of Task-Relevant Corpora
RS
DRC
We investigate the impact of corpora with varying degrees of relevance to the target task on the per- formance of model extraction attacks. The experi- ments are conducted on a simulated victim model trained on IMDB with different query budgets. To assess the influence of the corpus, we replace the default WikiText- 103 corpus with other datasets that are more closely related to the victim model's dataset, such as the SST-2 and IMDB training sets. For query sampling, we adopt uniform random strategy. The results, shown in Table and Table 7. indicate that using the IMDB training data as the corpus yields the best performance, followed
Figure 5: Comparison of data distribution: DRC Sam- pling vs. RS (Random Sampling). The blue point repre- sents the data pool consisting the text in the subset of WikiText- 103. The selected data points by DRC or RS are marked in red.

---

Corpus
X 0.003
x 0.005
0.008
x 0.01
X 0.02
Wik
53.4 + 3.8 (63.1)
59.8 + 7.8 (71.8)
+ 8.0 (73.5)
63.2 8.3 (75.0)
66.8 11.1 (78.9)
SST-2 training data
70.8 = 6.5 (77.4)
72.3 + 7.0 (81.)
76.5 + 5.5 (817)
78.9 5.6
77.4 + 5.3 (85.3)
IMDB training data
73.5 + 5.9 (84.2)
80.0 + 5.0 (84.8)
84.7 + 2.0 (86.6)
84.8 = 1.3 (85.6)
85.6 + 1.3 (87.2
Table 4:
Model extraction results (Agreement, %) under different query budgets with different corpus IMDB dataset. The results shown in the table are the mean and standard deviation. The max value is shown in green.
Task
RS example
MeaeQ example
Hate Speech
After the song was released in the United Kingdom
Lifton describes Mengele sadistic, lacking empathy, and extremely antisemitic, believing the Jews should eliminated entirely as an inferior and dangerous race.
June 8, 1985, debuted number 25 and peaked number imperial. (not relevant)
SST-2 / IMDB
During World War she was known as Norman El Brazilian in service with the United Stutes Army and USS EI 50th service with the United States 1957. (not relevant)
Roger Ebert, writing in the Chicago Sun- Times, awarded the film four of four portrait of Johnny Marco, saying Coppola fascinating director.
pruised the detail in
AG News
He eventually convinced silence of Powell foreign the Hollywood und this led Pokémon being accepted the Manhattan Project without devices his institutions.
In August 2015, Apple admitted that some iPhone Plus muy have faulty cameras that could be causing photos to look blurry initiated a replacement program.
Table Query examples sampled by and MeaeQ for four tasks, demonstrating that MeaeQ can select samples that semantically more relevant the target task. More query examples can be found in Appendix D.
5.3 Analysis of High-Frequency Words
trast, these words are rarely seen in RS, confirming that MeaeQ effectively filters task-related data and enhances data diversity.
We conduct an analysis of the text content in the query set generated by MeaeQ. Specifically, we seg- ment the text in the query set by word and calculate their frequency of occurrence. Figure presents the top 20 most frequent words in the query sets constructed by RS and MeaeQ for the Hate Speech dataset at query budget 0.5. We observe that the text filtered by MeaeQ contains higher fre- quency of words associated with hate speech, such as "hate", "antisemitic", "nazi", "racist", en con-
In Table 5, we also provide query examples sam- pled by RS and MeacQ, highlighting task-specific words in colors. It is apparent that the content style of the query generated by MeacQ closely aligns with the target task. For hate speech, they fea- ture negative and hateful sentiment words; for SST- 2/IMDB, they contain positive or negative movie reviews; and for AG News, they include key news elements like 'when', 'who', and 'what'. On the contrary, the contents of RS examples are often not relevant to the given task.
RS
MeaeQ
one
hate
new
jews
first
antisemitic
also
nazi
Conclusion
two
hatred
people
In this paper, we propose a straightforward yet ef- fective model extraction attack method MeaeQ. In particular, we initially utilize zero-shot sequence inference classifier in combination with the API service information to filter data with high task relevance from public unannotated text corpus, Subsequently, we employ a clustering- based data reduction technique to obtain representative data as queries for the attack. Extensive experiments demonstrate that our method achieves superior per- formance than basclines with limited query budgets on four benchmark datasets.
would
called
Words (Top 20)
Words (Top 20)
used
racist
hitler
later
would
song
white
may
jewish
including
said
us number released
war
also
first
us holocaust world
many
another
25 50
75
0
40 80
120
Word Frequency
Word Frequency
Figure 6: Top-20 frequent words in the query sets con- structed by RS and MeaeQ respectively at query budget x 0.5 on Hate Speech.

---

Limitations
model extraction attacks with training data inversion. In IJCAI, pages 2439-2444 2447.
Our study focuses on model extraction attacks in text classification tasks, in line with existing base- lines (Pal et al., 2020; Krishna et al., 2020). The applicability of MeacQ to other natural language generation tasks, such as machine translation or text summarization, remains unexplored. We are currently investigating model extraction attacks in these more complex tasks.
Yingzhe He, Guozhu Meng, Kai Chen, Xingbo Hu, and
Jinwen He. 2021. (DRMI): A dataset reduction technology based on mutual information for black- box attacks. In 30th USENIX Security Symposium (USENIX Security 21), pages 1901-1918.
Jeff Johnson, Matthijs Douze, and Hervé Jégou. 2019. Billion-scale similarity search with GPUs. IEEE Transactions on Big Data. 733):335-477.
Mika Juuti, Sehastian Szyller, Samuel Marchal, and N Asokan. 2019. Prada: protecting against dnn model stealing attacks. In 2019 IEEE European Sym- posium on Security and Privacy (EuroS&P), pages 512-527. IEEE
Ethics Statement
The data used in this work including WikiText- 103 (Merity et al.. 2017) and the victim model datascts (de Gibert et al., 2018; Zhang et al., 2015; Socher et al., 2013; Maas et al., 2011) are all open source and do not involve any ethical violations. However, it is essential to note that the method proposed in this paper can be potentially exploited by malicious users to extract real-world black-box models due to its low cost. Therefore, is crucial to raise awareness within the community regarding the significance of model extraction attacks.
Pratik Karmakar and Debabrota Basu. 2023. Marich: A query-efficient distributionally equivalent model extraction attack using public data. ArXiv preprint. abs/2302.084666
Kalpesh Krishna, Gaurav Singh Tomar, Ankur P. Parikh, Nicolas Papernot, and Mohit Iyyer. 2020. Thieves on sesame street! model extraction of bert-based apis. In 8th International Conference on Learning Representations, ICLR 2020, Addis Ababa, Ethiopia. April 26-30, 2020. OpenReview.net.
Mike Lewis. Yinhan Liu, Naman Goyal, Marjan Ghazvininejad, Abdelrahman Mohamed, Omer Levy. Veselin Stoyanov, and Luke Zetllemoyer. 2020. BART: Denoising srquence-to-sqquenee for natural language generation, translation, and com- prehension. In Proceedings of the 58th Annuul Meet- ing of the Association for Computational Linguisties. pages 7871-7880, Online. Association for Computa- tional Linguistics.
References
Varun Chandrasekaran, Kamalika Chaudhuri, Irene Gia- comelli, Somesh Jha, and Songbai Yan. 2020. Explor- ing connections between active learning and model extraction. In 29th USENIX Security Symposium (USENIX Security 20), pages 1309-1326.
pre-training
Jacson Rodrigues Colveia-Silva, Rodrigo F Berriel, Claudine Badue, Alberto F de Souza, and Thiago Oliveira-Santos. 2018. Copycat cmn: Stealing knowl- edge by persuading confession with random non- labeled data. In 2018 International Joint Conference on Neural Networks (IJCNN), pages 1-8. IEEE.
Yinhan Liu, Myle Ott, Naman Goyal, Jingfei Du, Man- dar Joshi, Danqi Chen, Omer Levy, Mike Lewis, Luke Zettlemoyer, and Veselin Stoyanov. 2019. Roberta: A robustly optimized bert pretraining ap- proach. ArXiv preprint, abs/1907.1166
Ona de Gibert, Naiara Perez, Aitor Garefa-Pabll and Montse Cuadros. 2018. Hate speech dataset from white supremacy forum. In Proceedings of the 2nd Workshop on Abusive Language Online (ALW2),
Andrew L. Maas, Raymond E. Daly, Peter T. Pham, Dan Huang, Andrew Y. Ng. and Christopher Potts. 2011. Learning word vectors for sentiment analysis. In Proceedings of the 49th Annual Meeting of the Association for Computational Linguistics: Human Language Technologies, pages 142-150, Portland, Oregon, USA. Association for Computational Lin- guistics.
pages 11 11-20, Brussels, Belgium. Association for Computational Linguistics.
Jacob Devlin, Ming- Ming-Wei Chang, Kenton Lee, and Kristina Toutanova. 2019. BERT: Pre-training of deep bidirectional transformers for language under- standing. In Proceedings of the 2019 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Tech- nologies, Volume I (Long and Short Papers), pages 4171-4186, Minneapolis, Minnesota. Association for Computational Linguistics.
MacQueen. 1967. Classification and analysis of mul- tivariate observations. In 5th Berkeley Symp. Math. Statist. Probability, pages 281-297,
Stephen Merity, Caiming Xiong, James Bradbury, and Richard Socher. 2017. Pointer sentinel mixture mod- els. In 5th International Conference on Learning Representations, ICLR 2017, Toulon, France, April 24-26, 2017. Conference Track Proceedings. Open- Review.net.
Xueluan Gong. Yanjiao Chen, Wenbin Yang. Guanghao Mei, and Qian Wang. 2021. Inversenet: Augmenting

---

Smitha Milli, Ludwig Schmidt, Anca D Dragan, and Moritz Hardt. 2019. Model reconstruction from model explanations. In Proceedings of the Confer- ence on Fairesss, Accountability, and Transparency, pages 1-9.
Dongdong Wang. Yandong Li, Liqiang Wang, and Bo- qing Gong. 2020. Neural networks are more produc- tive teachers than human raters: Active mixup for data-efficient knowledge distillation from a blackbox model. In 2020 JEEE/CVF Conference on Computer Vision and Pattern Recognition, CVPR 2020, Seat- rle, WA, USA, June 13-19, 2020, pages 1495-1504. IEEE.
OpenAI. 2023. Chatgpt (Mar 01 version). https:// chat. openai.peni.... com.
Tribhuvanesh Orekondy, Bernt Schiele, and Mario Fritz. 2019. Knockoff nets: Stealing functionality of black- box models. In IEEE Conference on Computer Vision and Pattern Recognition, CVPR 2019, Long Beach, CA. USA, June 16-20, 2019, pages 4954-4963. Com- puter Vision Foundation/ / IEEE.
Yixu Wang, Jie Li, Hong Liu, Yan Wang. Yongjian Wu, Feiyue Huang, and Rongrong Ji. 2022. Black-box dissector: Towards erasing- based hard-label model stealing attack. In Computer Vision-ECCV 2022: 17th European Conference, Tel Aviv, Israel, October 23-27, 2022, Proceedings, Part V, pages 192-208. Springer.
Soham Pal. Yash Gupta, Aditya Shukla, Aditya Kanade, Shirish K. Shevade, and Vinod Ganapathy. 2020. Ac- tivethief: Model extraction using active learning and unannotated public data. In The Thirty-Fourth AAAI Conference on Artificial Intelligence, AAAI 2020, The Second Innovative Applications of Artificial Intelligence Conference, IAAI 2020, The Temth AAAI Symposium on Educational Advances in Artificial In- lelligence, EAAI 2020, New York, NY, USA, February 7-12, 2020, pages 865 872. AAAI Press.
Adina Williams, Nikita Nangia, and Samuel Bowman. 2018. A broad-coverage challenge corpus for sen- tence understanding through inference. In Proceed- ings of the 2018 Conference of the North American Chapter of the Association for Computational Lin- guistics: Human Language Technologies, Volume (Long Papers), pages 11 12-1122, New Orleans, Louisiana. Association for Computational Linguis- tics.
Qiongkai Xu, Xuanli He, Lingjuan Lyu, Lizhen Qu, and Gholamreza Haffari. 2022. Student surpasses teacher: Imitation attack for black-box NLP APIs. In Proceedings of the 29th International Conference on Computational Linguisties, pages 2849 2260. Gyeongju, Republic of Korea. International Com- mittee Computational Linguistics.
Nicolas Papernot Patrick McDanicl, Ian Goodfellow, Somesh Jha, Z Berkay Celik, and Ananthram Swami. 2017. Practical black-box attacks against machine learning. In Proceedings of the 2017 ACM on Asia conference on computer and communications secu- rity, pages 506-519.
Alec Radford, Jeffrey Wu, Rewon Child, David Luan, Dario Amodei, Ilya Sutskever, et al. 2019. Language models are unsupervised multitask learners. OpenAl blog. 188)9
Zhilin Yang, Zihang Dai, Yiming Yang, Jaime G. Car- bonell, Ruslan Salakhutdinov, and Quac V. Le. 2019. XInet: Generalized autoregressive pretraining for language understanding. In Advances in Neural In- formation Processing Systems 32: Annual Confer- ence on Neural Information Processing Systems 2019, NeurlPS 2019, December 8-14, 2019, Vancouver, BC, Canada, pages 5754-5764
Yi Shi, Yalin Sagduyu, and Alexander Grushin. 2017. How to steal a machine learning classifier with deep learning. In 2017 IEEE International symposium on technologies for homeland security (HST), pages 1-5.
IEEE.
Wenpeng Yin, Jamaal Hay, and Dan Roth. 2019. Bench- marking zero-shot text classification: Datasets, eval- uation and entailment approach. In Proceedings of the 2019 Conference on Empirical Methods in Natu- rul Language Processing and the 9th International Joint Conference on Natural Language Processing (EMNIP--ICCILP)) pages 391 1-33923,, Hong Kong, China. Association for Computational Linguistics.
Richard Socher, Alex Perelygin, Jean Wu, Jason Chuang, Christopher D. Manning, Andrew Ng, and Christopher Potts. 2013. Recursive deep models for semantic compositionality over a sentiment treebank. In Proceedings of the 2013 Conference on Empiri- cal Methods in Natural Language Processing, pages 1631-1642, Seattle, Washington, USA. Association for Computational Linguistics.
Honggang Yu, Kaichen Yang, Teng Zhang, Yun-Yun Tsai, Tsung- Yi Ho, and Yier Jin. 2020. Cloudleak: Large-scale deep learning models stealing through adversarial examples. NDSS.
Florian Tramèr, Fan Zhang, Ari Juels, Michael K Re- iter, and Thomas Ristenpart. 2016. Stealing ma- chine learning models via prediction [APIs). In 25th USENIX security symposium (USENIX Security 16), pages 601-618.
Xiang Zhang, Junbo Jake Zhao, and Yann LeCun. 2015. Character-level convolutional networks for text clas- sification. In Advances in Neural Information Pro- cessing Systems 28: Annual Conference on Neural In- formation Processing Systems 2015, December 7-12, 2015, Montreal, Quebec, Canada, pages 649-657.
Laurens Van der Maaten and Geoffrey Hinton. 2008. Visualizing data using t-sne. Journal of machine learning research, 9(1)).

---

Zhanyuan Zhang, Yizheng Chen, and David Wagner. 2021. Seat: Similarity encoder by adversarial train- ing for detecting model extraction attack queries. In Proceedings of the 14th ACM Workshop Artificial Intelligence and Security, pages 37-48.
MearQ
AL-US
78.20
78.50
79.90
80.10
74.20
76.20
73:50
73.00
Mingyi Zhou, Jing Wu, Yipeng Liu, Shuaicheng Liu, and Ce 7hu. 2020. Dast: Data-free substitute training for adversarial attacks, In 2020 IEEE/CVF Confer- ence on Computer Vision and Puttern Recognition, CVPR 2020, Seattle, WA, USA, June 13-19, 2020, pages 231-240. IEEE.
X
74.90
77:80
77.60
X
63.60
68.60
70.50
B
R
X
R
X
AL-RS
R5
75.90
69.50
53.40
A Results on Accuracy
In this section, we provide the accuracy results of the experimental part in the main text.
78.
71:10
A.1 Main Results
71.63 61.90
68.88
50.00
51.70
61.70
For the main experiment, We list the accuracy re- sults in Table 6. From Table in all settings, MeacQ also better than the base- lines in terms of accuracy. Furthermore, we notice that MeaeQ can achieve similar or higher accuracy performance than the haselines while using approx- imately half of the query budget (as observed in the comparison between Meae of Group A and the baseline method of Group B). Notably, in the best case, for using the query budget set by Group A, the accuracy of the model extracted by MeaeQ can reach 88. 8.8% of the victim model for Hate Speech, 92.8% for SST-2, 86.1 % for IMDB, and 91.1% 11% for AG News, which also demonstrate that MeaeQ is an efficient model extraction attack method.
8
X
8
see that
Figure 7: Cross model extraction results (Accuracy, %) on Hate Speech at query budget 0.5. The horizontal vertical axes represent victim / extracted model architec- ture respectively. 'B', 'R', and "X" represent BERT Base- RoBERTanaee and XLNetBwe Darker colors represent larger values of Accuracy.
A.4 Impact of Task-Relevant Corpora
Table 7 presents the accuracy results obtained us- ing corpora with varying degrees of relevance the target task. The findings align with the corre- sponding chapter in the main text, confirming that task-relevant data significantly enhance the perfor- mance model extraction attacks.
A.2 Cross Model Extraction
We present the results of cross-architecture model extraction in terms of accuracy in Figure 7. It is evi- dent that MeaeQ consistently outperforms the base- line method in extracting models with higher aceu- racy across different model architectures. Further- more, the results on the agreement of MeaeQ, as shown in Figure 2, reveal that the model extracted using BERT cxhibits higher functional consistency than RoBERTa, which contradicts the observation in terms of accuracy in Figure 7. This indicates that accuracy and agreement do not necessarily exhibit complete positive correlation.
B Details of Datasets
The dataset details are listed in Table 8. We split the original train set into new train and validation sets with a 7:1 ratio for SST-2, 9:1 for IMDB and Hate Speech, and 4:1 for AG News. The original validation set serves as the test set for SST-2, while the others remain unchanged.
C Hyperparameter Settings
We list the hyperparameters in the main experi- ment in Tabel 9. Both the victim model and the extracted model the same training hyperparam- eters. Other parameters about the model are default parameters from huggingface
A.3 Ablation Study
The accuracy results of the ablation study are pre- sented in Figure 8. Similar the observation in the main body, we find that MeaeQ exhibits a signifi- cant performance advantage over the two variants at a low query budget.
Phttos://huginngfacc.ooo// http:////aaaceeeoo////////orrrer
index

---

Hate Speech
SST-2
IMDB
AG News
Query Budget
Method
x = 1914)
( [ I 67349)
(
40000)
(x = 120000) )
Victim
83.89
92.55
89.04
92.43
RS
50.0 + 0.0 (50.0)
66.0 + 8.0 (81.1)
52.5 + 3.7 (62.2)
75.0 + 3.0 (78.0)
Group A
AL-RS
49.2 + 2.1 (50.0)
71.9 + 6.6 (83.3)
54.3 + 4.3 (63.6)
79.4 + 2.7 (83.0)
x 0.1 x 0.003
AL-US
56.6 + 6.0 (67.2)
72.2 + 8.4 (83.7)
55.4 -: 4.2 (61.9)
75.0 + 7.3 (83.1)
(191 201/ / /20 360)
MeacQ
69.1 + 5.1 (74.5)
81.0 = 3.7 (85.9)
63.6 = 8.3 (76.7)
80.6 2.8 (84.)
RS
50.0 + 0.1 (50.0)
74.9 + 6.6 (85.1)
58.7 = 7.2 (69.6)
73.0 + 9.8 (84.2)
Group 0.2/ x 0.005 (382/ / 335 / 200/ 600)
AL-RS
50.4 + 2.3 (56.9)
80.5 + 4.2 (85.8)
57.9 7.1 (72.6)
80.7 3.4 (84.5)
AL-US
67.1 + 5.3 (73.2)
76.8 + 7.5 (84.2)
60.8 + 7.7 (74.2)
81.2 4 3.3 (85.4)
MeaeQ
72.9 + 2.4 (75.7)
82.8 + 2.6 (86.9)
71.1 + 4.3 (75.5)
82.7 + 1.6 (85.8)
RS
51.7 + (61.1)
77.4 + 8.3 (86.2)
61.9 + (714)
80.6 + 2.8 (84.2)
Group C
AL-RS
54.4 +: 6.9 (67.0)
82.4 + 4.2 (87.2)
58.6 +: 5.7 (67.2)
78.2 + 2.0 (80.1)
X 0.3 x 0.008 (574/ 536 / 320/ / 960)
AL-US
73.5 1: 4.6 (80.3)
80.5 + 5.0 (86.6)
70.6 1 5.0 (76.0)
82.0 + 0.8 (83.5)
MeacQ
74.9 + 1.2 (77.4)
84.8 + 1.7 (87.6)
76.7 + 1.6 (78.6)
83.4 1.5 (85.7)
Table 6: Model extraction result (Accuracy, %) under different query budgets. x 0.1 / x 0.2/ x 0.3 query budgets are set for Hate Speech dataset. x 0.003 / 0.005 / 0.008 query budgets are set for others. The specific number of queries for different datasets is shown in parentheses below the query rates. The results shown in the table are the mean and standard deviation while the max value is shown in green.
Corpus
x 0.003
x 0.005
0.008
x 0.01
X 0.02
WikiText-103
52.5 1: 3.7 (62.2)
58.7 1: 7.2 (69.6)
61.9 + 7.4 (7144)
61.7 + 7.6 (72.0)
65.0 1: 10.2 (75.8)
SST-2 training data
68.5 1 6.1 (74.9)
70.9 1 5.8 (77.8)
74.2 4.5 (77.9)
76.2 4.4 (80.3)
75.0 + 4.4 (81.3)
IMDB training data
73.3 + 5.2 (80.7)
77.0 + 4.4 (81.1)
81.1 + 1.7 (82.)
81.2 1.0 (82.5)
81.6 + 1.0 (82.6)
Table 7: Model extraction results (Accuracy, %) under different query budgets with different corpus on IMDB dataset. The results shown in the table are the mean and standard deviation. The max value is shown in green.
Datasets
Hate Speech
SST-2
IMDB
AG News
95.00
# of class
2
2
2
4
MAS TRLF
DRC
train size
1722
58930
36000
96000
Meann
validation size
192
8419
4000
24000
88.75
test size
478
872
10000
7600
total
2392
68221
50000
127600
Acorecy
62.50
Table 8: The statistics of the datasets.
76.25
Dataset
Hate Speech
SST-2
IMDB
AG News
adam weight decay
1c-4
lc-4
1c-4
lc-4
batch size
32
32
32
16
learning rate 3e-5
3e-5
3e-5
5e-5
Query
iruncute max length
128
128
128
256
Figure 8: Ablation study results (Accuracy, %) on SST- 2 dataset. The horizontal axes represent different query budgets. Error bars represent standard deviation.
Table 9: Details of Hyperparameters in the main experi- ment only using BERT Base as the model architecture.
Model Architectree
adam weight decay
batch size
learning
truncato max length
XLNelaaa
lc-4
20-5
128
D Query Examples
GPT-2aut
lc-5
8e-5
128
GPT-2waass
1e.5
1e-5
128
More query examples are presented in Table 11.
RoBERTans
Ie-4
26-5
128
RoBERTausge
Ic-4
8c-6
128
Table 10: Details of Hyperparameters in the experiments using different model architectures on Hate Speech.

---

Task
MeaeQ example
Hate Speech
Lloyd Paul Stryker, who wrole a highly favorable 1929 biography of Johnson, labeled Stevens as a horrible old man ... craftily preparing strangle the bleeding, broken body of the South " and who thought it would be a beautiful thing to see the white men, especially the white women of the South, writhing under negro domination""
Run by the Abbé Norbert Wallez, the paper described itself as a Catholic Newspaper for Doctrine and Information and disseminated far-right, fascist viewpoint.
One dismissed customer even yells at him, [ w ] hy don't you go back to your own country returning the spotlight of racial prejudice on him.
Houston Stewart Chamberlain's work The Foundations of Nineteenth Century (1900), one the first to combine Secial Darwinism with antisemitism, describes history struggle for survival between the Germanic peoples and the Jews, whom he characterized as an inferior and dangerous group.
talks about permitting arson and destroying Jewish businesses and synagogues, and orders the confiscation of all archival material out of Jewish community centres and synagogues.
SST-2 IMDB
Roger Ebert of the Chicago Sun-Times praised the film while comparing it to the original.
Ebert and colleague, Gene Siskel, gave the film Two Thumbs Up rating on their syndicated television program, Siskel and Ebert and the Movics.
AMC's Chris Cabin criticized the movie, arguing that its director seems not have the faintest idea of how to properly approach the subject ", because the film is, in Cabin's view, unabashedly pro-Palestine..
BBC film critic Nev Pierce believed the film had spectacular set-picces, but felt there was no strong narrative are keep the viewer interested.
Pittsburgh Post- Gazette critic Ron Weiskind called the film incompetent and criticized the film's ss acting, lack of suspense, and production values.
AG News
By 1904, however, Brazil began to seriously consider upgrading its navy to compete with Argentina and Chile.
In March of that year the News of the World released video footage of Mosley engaged in with five consenting women in scenario that the paper alleged involved Nazi role- role-playing (an allegation that, though dismissed in court as no genuine basis allegedly" "ruined Mosley's reputation).
On April 4, 2016, Ming was elected into the Naismith Memorial Basketball Hall of Fame.
On 11 November 2011, Iommi, Butler, Osbourne, and Ward announced that they were reuniting to record a new album with a full tour in support beginning in 2012.
In the Hinthada District, torrential rains caused flash flooding that killed 18 people and left 14 others missing.
Table 1: More examples sampled by MeaeQ from WikiText-103 for four tasks.


Reviews:
[{"review_id": "25vQHT7528", "reviewer": "Reviewer_rQZ7", "summary": "", "questions": "", "limitations": "", "rating": 4, "confidence": 3, "soundness": 4, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "### To Reviewer rQZ7:\n\n**Q1:** Employing the method on various NLP tasks or classification tasks (e.g. offensive detection) would be better to confirm the efficiency of the method.\n\n**A1:** Thank you for your suggestion. We fully recognize the importance of evaluating our method across a broader spectrum of NLP tasks. Therefore, in our experiments, we have already simulated a range of detection tasks, such as SST-2 for short text sentiment analysis, IMDB for sentiment classification in long texts, AG News for news text classification, and Hate Speech for hate content detection on social media platforms. The results highlight the effectiveness of our approach over baselines, reinforcing its notable impact. Also, in the final version of the paper, we will add more classification tasks (e.g. OffensEval 2019 [1] for offensive language identification and Enron Spam [2] for spam email detection ) to confirm the efficiency of our method.\n\n[1] Zampieri, Marcos, et al. \"Semeval-2019 Task 6: Identifying and Categorizing Offensive Language in Social Media (OffensEval).\" *arXiv preprint arXiv:1903.08983* (2019).\n\n[2] Metsis, Vangelis, Ion Androutsopoulos, and Georgios Paliouras. \"Spam Filtering with Naive Bayes-Which Naive Bayes?.\" *CEAS*. Vol. 17. 2006.\n\n\n\n**Q2:** Even though MLM is important and used in many applications, the method should also be employed in the autoregressive language models, which are used more in real-world applications.\n\n**A2:** We fully agree with your point. In fact, we have systematically conducted model extraction experiments on autoregressive language models. Due to the page limit of the paper, we did not include this experiment. In the final version of the paper, we will supplement this experiment. Now we report the experiment as follows.\n\n#### \n\n**Additional Experiment**\n\nWe conduct model extraction attacks on the ChatGPT-based victim model simulated on the hate speech detection task using manual instructions. The instruction template is shown in Table 1. We set up several types of extracted models with different parameters and architectures, including GPT-2 and RoBERTa. The results of the zero-shot ChatGPT model extraction on the hate speech detection task are reported in Tables 2 and 3. We can find that our method still performs well even though the victim model is ChatGPT-based and the extracted model is autoregressive-based language model, e.g. GPT-2. For instance, GPT-2-small achieves **79.5%** functional similarity (Agreement) and 75.3% test accuracy (**90.4%** of the victim model). We also observe that the RoBERTa series of models outperforms the GPT-2 series in this detection task. Just as MLM can be used to extract classification models, it is sensible to use autoregressive models to extract generative models.\n\n\n\n**Table 1: Instruction Template**\n\n| Task        | Instruction                                                  |\n| ----------- | ------------------------------------------------------------ |\n| Hate Speech | You are now working as a hate speech detector. Next, I will give you [batch\\_size] sentences. You need to indicate whether it contains hate speech. If it contains hate speech, output \"Hate\", otherwise output \"Nohate\". The required format is one output per line and the beginning of each output is numbered sequentially. Sentences:\\n1. [sentence\\_1]\\n2. [sentence\\_2]\\n... |\n\n###### Detail: The [batch_size] in the instruction represents the number of queries, and [sentence_1], [sentence_2], etc. represent the specific query content.\n\n\n\n**Table 2: Model Extraction Results (Agreement, %)**\n\n| Method       | GPT-2 (small,117M) | GPT-2 (medium,345M) | RoBERTa (Base, 125M) | RoBERTa (Large, 355M) |\n| ------------ | ------------------ | ------------------- | -------------------- | --------------------- |\n| RS           | 43.7               | 43.9                | 51.9                 | 59.8                  |\n| AL-RS        | 43.9               | 48.9                | 43.7                 | 54.2                  |\n| AL-US        | 43.9               | 45.6                | 57.3                 | 76.6                  |\n| MeaeQ (ours) | **79.5**           | **75.3**            | **82.4**             | **85.4**              |\n\n###### Detail: Table 2 shows the model extraction results (Agreement, %) on ChatGPT simulated on the Hate Speech dataset at query budget × 0.8 (about 1531 queries). The data recorded in the table are the best results of ten retests.\n\n\n\n**Table 3: Model Extraction Results (Accuracy, %)**\n\n| Method       | GPT-2 (small,117M) | GPT-2 (medium,345M) | RoBERTa (Base, 125M) | RoBERTa (Large, 355M) |\n| ------------ | ------------------ | ------------------- | -------------------- | --------------------- |\n| RS           | 50.0               | 50.2                | 55.6                 | 64.4                  |\n| AL-RS        | 50.2               | 53.6                | 50.0                 | 56.3                  |\n| AL-US        | 50.2               | 50.6                | 60.7                 | 73.6                  |\n| MeaeQ (ours) | **75.3**           | **71.5**            | **77.4**             | **79.5**              |\n\n###### Detail: Table 3 shows the model extraction results (Accuracy, %) on ChatGPT simulated on the Hate Speech dataset at query budget × 0.8 (about 1531 queries). The accuracy of ChatGPT on this task under the zero-shot setting is 83.26%. The data recorded in the table are the best results of ten retests.", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "- A targetted method for extraction, not similar to the previous method, is based on random sampling, which increases the success rate and lowers the required number of queries\n\n- Outperforms the previous baselines.\n\n- Well-written, and the experiments and ablations are informative.", "reasons_to_reject": "- Employing the method on various nlp tasks or classification tasks(e.g. offensive detection) would be better to confirm the efficiency of the method. \n- Even though MLM is important and used in many applications, the method should also be employed in the autoregressive language models, which are used more in real-world applications", "questions_to_authors": "", "paper_topic_and_main_contributions": "The paper provides an effective extraction method based on two filtering processes: selecting the highly relevant samples and reducing the number of samples to be queried.", "excitement": 4, "reproducibility": 3, "ethical_concerns": "No", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {}, "extra_text": {}}, {"review_id": "rK7big39bK", "reviewer": "Reviewer_ymJe", "summary": "", "questions": "Why not try LLMs like GPT and LLAMA instead of BERT?", "limitations": "", "rating": 3, "confidence": 3, "soundness": 3, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "### To Reviewer ymJe:\n\n**Q0:** It would be better if some punctuation can be added to equations.\n\n**A0:** Thank you for your feedback regarding punctuation in equations. We appreciate your suggestion and will make sure to review and enhance the punctuation in equations to improve their clarity and presentation.\n\n**Q1:** The technical quality is relatively limited in my opinion, just some clustering and filtering methods.\n\n**A1:** We acknowledge the limitations of the technical aspects presented in our method. However, it's important to note that the crux of our contribution lies in optimizing query sampling from two angles: query-task relevance and inter-query information redundancy. Our experimental results show that the attacker can extract a replica model with more than 80% functional consistency with the victim model with only a few queries. Despite its apparent simplicity, our method's empirical performance speaks to its effectiveness, aligning with the empirical spirit of EMNLP.\n\n**Q2:** Why not try LLMs like GPT and LLAMA instead of BERT?\n\n**A2:** We value your input on this matter. Although our experiments focused on BERT as a representative example, we acknowledge that LLMs like GPT and LLAMA are significant in the context of model extraction. Therefore, we conducted additional model extraction experiments using GPT series of models as the primary setting to validate the effectiveness of our method. We will include the results of this experiment in the revised version of the paper. Currently, we present the experiment as follows.\n\n**Additional Experiment**\n\nWe conduct model extraction attacks on the ChatGPT-based victim model simulated on the hate speech detection task using manual instructions. The instruction template is shown in Table 1. We set up several types of extracted models with different parameters and architectures, including GPT-2 and RoBERTa. The results of the zero-shot ChatGPT model extraction on the Hate Speech task are reported in Tables 2 and 3. We can find that even though the victim model is ChatGPT-based, our method still performs well for both GPT-like and BERT-like models. For instance, RoBERTa-large achieves **85.4%** functional similarity (Agreement) and 79.5% test accuracy (**95.5%** of the victim model); GPT-2-small achieves **79.5%** functional similarity (Agreement) and 75.3% test accuracy (**90.4%** of the victim model). We also observe that the RoBERTa series of models outperforms the GPT-2 series in this detection task. Just as masked language models can be used to extract classification models, it is sensible to use autoregressive models to extract generative models.\n\n**Table 1: Instruction Template**\n\n| Task | Instruction |\n| --- | --- |\n| Hate Speech | You are now working as a hate speech detector. Next, I will give you [batch\\_size] sentences. You need to indicate whether it contains hate speech. If it contains hate speech, output \"Hate\", otherwise output \"Nohate\". The required format is one output per line and the beginning of each output is numbered sequentially. Sentences:\\n 1. [sentence\\_1]\\n 2. [sentence\\_2]\\n... |\n\n###### Detail: The [batch_size] in the instruction represents the number of queries, and [sentence_1], [sentence_2], etc. represent the specific query content.\n\n**Table 2: Model Extraction Results (Agreement, %)**\n\n| Method | GPT-2 (small,117M) | GPT-2 (medium,345M) | RoBERTa (Base, 125M) | RoBERTa (Large, 355M) |\n| --- | --- | --- | --- | --- |\n| RS  | 43.7 | 43.9 | 51.9 | 59.8 |\n| AL-RS | 43.9 | 48.9 | 43.7 | 54.2 |\n| AL-US | 43.9 | 45.6 | 57.3 | 76.6 |\n| MeaeQ (ours) | **79.5** | **75.3** | **82.4** | **85.4** |\n\n###### Detail: Table 2 shows the model extraction results (Agreement, %) on ChatGPT simulated on the Hate Speech dataset at query budget × 0.8 (about 1531 queries). The data recorded in the table are the best results of ten retests.\n\n**Table 3: Model Extraction Results (Accuracy, %)**\n\n| Method | GPT-2 (small,117M) | GPT-2 (medium,345M) | RoBERTa (Base, 125M) | RoBERTa (Large, 355M) |\n| --- | --- | --- | --- | --- |\n| RS  | 50.0 | 50.2 | 55.6 | 64.4 |\n| AL-RS | 50.2 | 53.6 | 50.0 | 56.3 |\n| AL-US | 50.2 | 50.6 | 60.7 | 73.6 |\n| MeaeQ (ours) | **75.3** | **71.5** | **77.4** | **79.5** |\n\n###### Detail: Table 3 shows the model extraction results (Accuracy, %) on ChatGPT simulated on the Hate Speech dataset at query budget × 0.8 (about 1531 queries). The accuracy of ChatGPT on this task under the zero-shot setting is 83.26%. The data recorded in the table are the best results of ten retests.\n\n**Q3:** The proposed method relies on predefined tasks, which cannot be applied to AGI models.\n\n**A3:** We acknowledge your concern regarding the applicability of our method to AGI models. Our paper primarily focuses on extracting black-box models for classification tasks rather than generative tasks. While AGI models like ChatGPT demonstrate impressive performance across a range of open-ended tasks, masked language models like BERT still remain state-of-the-art representatives for specific classification tasks [1, 2]. Thus, our method retains much value in extracting black-box models powering classification task's APIs.\n\nFurthermore, we do not believe that our method is inherently incompatible with AGI models or LLMs. In the era of such models, numerous efforts have emerged, utilizing the extraction data from ChatGPT to train the LLaMa by instruction tuning (e.g., Alpaca , Vicuna). As Reviewer GBYG commented, \"The clustering and filtering approach introduced in this paper could be an interesting way to create instruction diversity, and could potentially lead to better instruction tuned models.\" Our method holds potential value in filtering high-quality instruction data, which is also part of our future work. \n\nFor instance, we can leverage the self-instruct [3] technique to acquire an extensive and diverse set of instructions. Subsequently, we can apply filters to obtain task-relevant subsets of instructions, e.g., using finance-related instructions for tuning financial LLM[4]. Additionally, the application of clustering methods can be employed to effectively reduce redundant instruction information, thereby further enhancing the quality of instructions. It's important to note that more instructions are not necessarily better [5]; the quality of instructions is of paramount importance [6].\n\n[1] Kocoń, Jan, et al. \"ChatGPT: Jack of all trades, master of none.\" *Information Fusion* (2023): 101861.\n\n[2] Zhong, Qihuang, et al. \"Can ChatGPT Understand Too? A Comparative Study on ChatGPT and Fine-tuned BERT.\" *arXiv preprint arXiv:2302.10198* (2023).\n\n[3] Wang, Yizhong, et al. \"SELF-INSTRUCT: Aligning Language Model with Self-Generated Instructions.\" _arXiv preprint arXiv:2212.10560_ (2022).\n\n[4] Wu, Shijie, et al. \"BloombergGPT: A Large Language Model for Finance.\" _arXiv preprint arXiv:2303.17564_ (2023).\n\n[5] Zhou, Chunting, et al. \"LIMA: Less is More for Alignment.\" _arXiv preprint arXiv:2305.11206_ (2023).\n\n[6] Wei, Lai, et al. \"InstructionGPT-4: A 200-Instruction Paradigm for Fine-Tuning MiniGPT-4.\" _arXiv preprint arXiv:2308.12067_ (2023).\n\n**Q4:** Only BERT model is used in experiments.\n\n**A4:** In fact, we thoroughly explore our method's applicability across model architectures in Section 4.4, where we consider BERT, RoBERTa, and XLNet for both victim and attacker models. The experiment results consistently support its effectiveness across diverse model architectures. In the revised version, we will pay closer attention to detailed descriptions to ensure a more comprehensive presentation of our experimental setup.", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "1. The risk of model extraction attack is important, so the research reported in this paper is practical.\n\n2. This paper is easy to read and understand.\n\n3. The experimental results show the effectiveness of the proposed method.", "reasons_to_reject": "1. The technical quality is relatively limited in my opinion, just some clustering and filtering methods.\n\n2. The models to be extracted are PLMs like BERT, why not try LLMs like GPT and LLAMA?\n\n3. The proposed method relies on predefined tasks, which cannot be applied to AGI models.\n\n4. Only BERT model is used in experiments.", "questions_to_authors": "Why not try LLMs like GPT and LLAMA instead of BERT?", "paper_topic_and_main_contributions": "The paper introduces MeaeQ (Model extraction attack with efficient Queries), a straightforward and effective method to address certain issues. The method involves using a zero-shot sequence inference classifier, along with API service information, to filter task-relevant data from a public text corpus instead of using a problem domain-specific dataset. Additionally, they utilize a clustering-based data reduction technique to obtain representative data as queries for the attack. Extensive experiments conducted on four benchmark datasets demonstrate that MeaeQ achieves higher functional similarity to the victim model compared to baselines while requiring fewer queries.", "excitement": 3, "reproducibility": 3, "ethical_concerns": "No", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {"Missing_References": "None", "Typos_Grammar_Style_and_Presentation_Improvements": "It would be better if some punctuation can be added to equations."}, "extra_text": {}}, {"review_id": "wdq1LuIXNa", "reviewer": "Reviewer_GBYG", "summary": "", "questions": "", "limitations": "", "rating": 3, "confidence": 4, "soundness": 4, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "### To Reviewer GBYG:\n\n**Q1:** This paper could be more timely and interesting if the proposed model extraction methods are applied to instruction tuning settings. However, the paper currently focuses on BERT fine-tuning settings which was the paradigm in 2019, and not used much anymore in state-of-the-art language modeling.\n\n**A1:** We highly appreciate your insight. In our future work, we aim to explore the transferability of our approach to constructing instruction data and attempt to perform model extraction on LLMs like ChatGPT (or GPT4) for specific tasks or general tasks. However, our method is tailored for detection tasks such as hate speech detection on social media platforms, where state-of-the-art models are often bidirectional encoders rather than decoder-only models or LLMs [1, 2]. In order to scrutinize the applicability of our method to the prevalent paradigm of language modeling (autoregressive model), we conducted additional model extraction experiments. Refer to the \"Additional Experiment\" section below for further details.\n\n       Moreover, stringent requirements for model size and inference speed exist in both online and offline detection scenarios. Despite BERT's smaller size compared to LLMs, it performs on par or even better in classification tasks. We believe the traditional BERT fine-tuning paradigm remains relevant in these contexts, and our method still poses a threat to model privacy, enabling the theft of well-performing models at minimal cost.\n\n[1] Kocoń, Jan, et al. \"ChatGPT: Jack of all trades, master of none.\" *Information Fusion* (2023): 101861.\n\n[2] Zhong, Qihuang, et al. \"Can ChatGPT Understand Too? A Comparative Study on ChatGPT and Fine-tuned BERT.\" *arXiv preprint arXiv:2302.10198* (2023).\n\n**Additional Experiment**\n\nWe conduct model extraction attacks on the ChatGPT-based victim model simulated on the hate speech detection task using manual instructions. The instruction template is shown in Table 1. We set up several types of extracted models with different parameters and architectures, including GPT-2 and RoBERTa. The results of the zero-shot ChatGPT model extraction on the Hate Speech task are reported in Tables 2 and 3. We can find that our method still performs well even though the victim model is ChatGPT-based and the extracted model is autoregressive-based language model, e.g. GPT-2. For instance, GPT-2-small achieves **79.5%** functional similarity (Agreement) and 75.3% test accuracy (**90.4%** of the victim model). We also observe that the RoBERTa series of models outperforms the GPT-2 series in this detection task. Just as the masked language model can be used to extract classification models, it is sensible to use autoregressive models to extract generative models.\n\n**Table 1: Instruction Template**\n\n| Task | Instruction |\n| --- | --- |\n| Hate Speech | You are now working as a hate speech detector. Next, I will give you [batch\\_size] sentences. You need to indicate whether it contains hate speech. If it contains hate speech, output \"Hate\", otherwise output \"Nohate\". The required format is one output per line and the beginning of each output is numbered sequentially. Sentences:\\n 1. [sentence\\_1]\\n 2. [sentence\\_2]\\n... |\n\n###### Detail: The [batch_size] in the instruction represents the number of queries, and [sentence_1], [sentence_2], etc. represent the specific query content.\n\n**Table 2: Model Extraction Results (Agreement, %)**\n\n| Method | GPT-2 (small,117M) | GPT-2 (medium,345M) | RoBERTa (Base, 125M) | RoBERTa (Large, 355M) |\n| --- | --- | --- | --- | --- |\n| RS  | 43.7 | 43.9 | 51.9 | 59.8 |\n| AL-RS | 43.9 | 48.9 | 43.7 | 54.2 |\n| AL-US | 43.9 | 45.6 | 57.3 | 76.6 |\n| MeaeQ (ours) | **79.5** | **75.3** | **82.4** | **85.4** |\n\n###### Detail: Table 2 shows the model extraction results (Agreement, %) on ChatGPT simulated on the Hate Speech dataset at query budget × 0.8 (about 1531 queries). The data recorded in the table are the best results of ten retests.\n\n**Table 3: Model Extraction Results (Accuracy, %)**\n\n| Method | GPT-2 (small,117M) | GPT-2 (medium,345M) | RoBERTa (Base, 125M) | RoBERTa (Large, 355M) |\n| --- | --- | --- | --- | --- |\n| RS  | 50.0 | 50.2 | 55.6 | 64.4 |\n| AL-RS | 50.2 | 53.6 | 50.0 | 56.3 |\n| AL-US | 50.2 | 50.6 | 60.7 | 73.6 |\n| MeaeQ (ours) | **75.3** | **71.5** | **77.4** | **79.5** |\n\n###### Detail: Table 3 shows the model extraction results (Accuracy, %) on ChatGPT simulated on the Hate Speech dataset at query budget × 0.8 (about 1531 queries). The accuracy of ChatGPT on this task under the zero-shot setting is 83.26%. The data recorded in the table are the best results of ten retests.\n\n**Q2:** In Table 1 (appendix version), the victim models perform much better than the extracted models, and I am guessing this is because of how strict the query budget for the extracted models are. Is such a strict query budget (<1K queries) necessary? API access is quite cheap these days --- for instance, 10K queries to ChatGPT (each with 500 tokens), would cost just 10$ according to the OpenAI API [5]. The paper should justify the strict query budgets by presenting cost estimates from different APIs. This is critical since the main premise of the paper is that it is a \"query-efficient\" method.\n\n**A2:** Your insightful question has prompted us to reflect deeply on this matter. Allow us to elaborate on why strict query budgets are indeed necessary.\n\n1. The performance gain brought to the extracted model by more queries is limited. From the first row of Figure 3 (main text), we can find that beyond 1K queries (e.g., query budget > 0.008 on AG News), the performance of the extracted model plateaus and maintains a certain performance gap with the victim model. We believe this performance gap does not solely stem from the strict query budget, but rather originates from the inherent differences in training data distributions between the attacker and victim models.\n  \n2. Excessive API calls could trigger defense mechanisms, diminishing the success rate of model extraction. Notably, prior work [1, 2, 3] has shown the effectiveness of distinguishing legitimate users from malicious actors through the detection of poorly crafted or excessive queries. Given that, our paper strives to craft high-quality queries within limited budgets to extract a high-performance model successfully.\n  \n\nThanks again for your insightful inquiry. We will delve deeper into justifying the strict query budgets in the revised paper.\n\n[1] Zhang, Zhanyuan, Yizheng Chen, and David Wagner. \"SEAT: Similarity Encoder by Adversarial Training for Detecting Model Extraction Attack Queries.\" *Proceedings of the 14th ACM Workshop on Artificial Intelligence and Security*. 2021.\n\n[2] Chen, Steven, Nicholas Carlini, and David Wagner. \"Stateful Detection of Black-Box Adversarial Attacks.\" _Proceedings of the 1st ACM Workshop on Security and Privacy on Artificial Intelligence_. 2020.\n\n[3] Kariyappa, Sanjay, and Moinuddin K. Qureshi. \"Defending Against Model Stealing Attacks with Adaptive Misinformation.\" _Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition_. 2020.\n\n**Q3:** I have some concerns about the sequence inference classifier that is used to filter data. It's unclear to me what kind of data passes this filter --- the task of filtering data using the task description seems quite different from the original MNLI distribution. Some examples of the filtered data would be helpful to look at.\n\n**A3:** Thank you for your constructive suggestion. In Section 5.3, we provide examples of high-frequency words that demonstrate the filter's alignment with the task description. For instance, in the hate speech detection task, the filter selects data containing words like \"hate\", \"racist\", \"war\", etc. This analysis illustrates that the filtered data aligns with the task description, validating the effectiveness of the sequence inference classifier, even when the MNLI dataset distribution differs from the corpus. We also provide illustrative examples of filtered data in Table 3 below this response, to offer a clearer understanding of the filter’s functionality, which will be added to our revised paper. Thanks again sincerely for your useful suggestion.\n\n**Table 3: Some data examples filtered by the sequence inference classifier**\n\n| Examples for Hate Speech |\n| --- |\n| 1. Lifton describes Mengele as sadistic, lacking empathy, and extremely antisemitic, believing the Jews should be eliminated entirely as an inferior and dangerous race. |\n| 2. Lloyd Paul Stryker, who wrote a highly favorable 1929 biography of Johnson, labeled Stevens as a \" horrible old man ... craftily preparing to strangle the bleeding, broken body of the South \" and who thought it would be \" a beautiful thing \" to see \" the white men, especially the white women of the South, writhing under negro domination \". |\n| 3. Run by the Abbé Norbert Wallez, the paper described itself as a \" Catholic Newspaper for Doctrine and Information \" and disseminated a far-right, fascist viewpoint. |\n\n| Examples for SST-2 / IMDB |\n| --- |\n| 1. Roger Ebert of the Chicago Sun-Times praised the film while comparing it to the original. |\n| 2. Ebert and his colleague, Gene Siskel, gave the film a \" Two Thumbs Up \" rating on their syndicated television program, Siskel and Ebert and the Movies. |\n| 3. Roger Ebert, writing in the Chicago Sun-Times, awarded the film four out of four stars and praised the detail in the portrait of Johnny Marco, saying \" Coppola is a fascinating director. |\n\n| Examples for AG News |\n| --- |\n| 1. By 1904, however, Brazil began to seriously consider upgrading its navy to compete with Argentina and Chile. |\n| 2. In March of that year the News of the World released video footage of Mosley engaged in acts with five consenting women in a scenario that the paper alleged involved Nazi role-playing (an allegation that, though dismissed in court as \" no genuine basis \", allegedly \" ruined \" Mosley's reputation). |\n| 3. On April 4, 2016, Ming was elected into the Naismith Memorial Basketball Hall of Fame. |\n\n**Q4:** A discussion on possible defenses against model extraction would be useful.\n\n**A4:** We appreciate your insightful suggestion. Exploring possible defenses against model extraction is indeed a valuable avenue for discussion. Here are several potential defense strategies that we have considered:\n\n1. One potential defense strategy could involve training a membership classifier to discern whether a user's input originates from genuine data distribution or from out-of-distribution. While the Task Relevance Filter identifies task-related data, it might capture only partial word associations. The membership classifier would need to grasp more specific patterns related to the task, such as text formatting, logical coherence, or contextual consistency. As attackers use individual sentences as queries, the isolated presentation of a sentence could be perplexing due to the possible absence of context.\n  \n2. Another conceivable defense approach entails introducing perturbations to the API response, possibly altering labels from positive to negative examples; however, this might adversely affect legitimate user inputs.\n  \n3. A third defense strategy could combine the previous two by leveraging the confidence scores (or probabilities) provided by the membership classifier to determine the degree of perturbation to introduce into API responses.\n  \n\nIn the revised paper, we will add a disscussion on possible defense strategies against model extraction attack.", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "1. This paper studies the problem of model extraction, which is a very relevant problem in the age of large language models which are being released behind black-box APIs (like ChatGPT).\n\n2. The paper presents a simple and intuitive algorithm for query-efficient distillation / model extraction.\n\n3. The proposed approach is evaluated on a variety of tasks, and shows strong performance improvements over competing methods like random sampling and active learning on small query budgets. The authors additionally perform a number of ablation experiments to show the effectiveness of the proposed approach.", "reasons_to_reject": "1. Model extraction is a very relevant topic in the age of ChatGPT, where several approaches have been introduced to construct instruction tuning datasets automatically [1, 2, 3, 4], in an attempt to \"extract\" closed-source ChatGPT. This paper would be much more timely and exciting if the methods introduced in this paper are applied to the instrucion tuning setting, and the sample efficiency is compared to techniques like self-instruct [1]. The clustering and filtering approach introduced in this paper could be an interesting way to create instruction diversity, and could potentially lead to better instruction tuned models. However, the paper currently focuses on BERT fine-tuning settings which was the paradigm in 2019, and not used much anymore in state-of-the-art language modeling.\n\n2. In Table 1 (appendix version), the victim models perform much better than the extracted models, and I am guessing this is because of how strict the query budget for the extracted models are. Is such a strict query budget (<1K queries) necessary? API access is quite cheap these days --- for instance, 10K queries to ChatGPT (each with 500 tokens), would cost just 10$ according to the OpenAI API [5]. The paper should justify the strict query budgets by presenting cost estimates from different APIs. This is critical since the main premise of the paper is that it is a \"query-efficient\" method.\n\n3. I have some concerns about the sequence inference classifier that is used to filter data. It's unclear to me what kind of data passes this filter --- the task of filtering data using the task description seems quite different from the original MNLI distribution. Some examples of the filtered data would be helpful to look at.\n\n4. Minor: A discussion on possible defenses against model extraction would be useful.\n\n[1] - https://arxiv.org/abs/2212.10560  \n[2] - https://gpt4all.io/reports/GPT4All_Technical_Report_3.pdf  \n[3] - https://crfm.stanford.edu/2023/03/13/alpaca.html  \n[4] - https://arxiv.org/abs/2305.15717  \n[5] - https://openai.com/pricing", "questions_to_authors": "", "paper_topic_and_main_contributions": "This paper presents a new algorithm for model extraction. Model extraction, like model distillation, is the task of training a student model to mimic a large teacher model (often a black-box API). However unlike model distillation, model extraction does not have access to the teacher model's original training data, and instead has to query the teacher model API for labels. Model extraction is considered a security threat, where the goal of an attacker is to reconstruct a local copy of the teacher model.\n\nThe paper presents a new algorithm for model extraction, which is shown to be more query efficient than random sampling and active learning. The proposed algorithm constructs data for model in two steps: 1) data filtering - a large corpus of unlabeled data is first filtered to keep the most relevant data to a downstream task, using the downstream task description and a natural language inference classifier; 2) data clustering - the filtered data is then clustered and representative examples are selected from each cluster as the query dataset; 3) The selected examples are then fed to the black-box teacher model to obtain labels, and the student model is trained on this data.\n\nThe paper conducts experiments on a variety of tasks like sentiment classification and hate speech classification, and uses BERT-based APIs as the base model. The paper has several interesting ablation studies and analysis of the proposed approach.", "excitement": 3, "reproducibility": 4, "ethical_concerns": "No", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {}, "extra_text": {}}]

Author rebuttal:

Meta-review: The paper presents an innovative and practically relevant approach to model extraction supported by strong experimental evidence. However, it could improve by extending experiments to include larger language models, and a more comprehensive range of tasks would further enhance its quality and applicability. Additionally, considering potential technical depth improvements could strengthen its overall contribution.
Decision: Accept-Main