Dynamic Neural Fortresses: An Adaptive Shield for Model Extraction Defense
Authors: Siyu Luan
Zhenyi Wang
Li Shen
Zonghua Gu
Chao Wu
Dacheng Tao
Venue: ICLR 2025 Poster (2025)
Keywords: Model Extraction Defense
Primary area: alignment, fairness, safety, privacy, and societal considerations

Abstract: Model extraction aims to acquire a pre-trained black-box model concealed behind a black-box API. 
Existing defense strategies against model extraction primarily concentrate on preventing the unauthorized extraction of API functionality. However, two significant challenges still need to be solved: (i) Neural network architecture of the API constitutes a form of intellectual property that also requires protection; (ii) The current practice of allocating the same network architecture to both attack and benign queries results in substantial resource wastage. To address these challenges, we propose a novel \textit{Dynamic Neural Fortresses} (DNF) defense method, employing a dynamic Early-Exit neural network, deviating from the conventional fixed architecture. Firstly, we facilitate the random exit of attack queries from the network at earlier layers. This strategic exit point selection significantly reduces the computational cost for attack queries. Furthermore, the random exit of attack queries from earlier layers introduces increased uncertainty for attackers attempting to discern the exact architecture, thereby enhancing architectural protection. On the contrary, we aim to facilitate benign queries to exit at later layers, preserving model utility, as these layers typically yield meaningful information. 
Extensive experiments on defending against various model extraction scenarios and datasets demonstrate the effectiveness of DNF, achieving a notable 2$\times$ improvement in efficiency and an impressive reduction of up to 12\% in clone model accuracy compared to SOTA defense methods. Additionally, DNF provides strong protection against neural architecture theft, effectively safeguarding network architecture from being stolen.

Published as a conference paper at ICLR 2025
DYNAMIC NEURAL FORTRESSES: AN ADAPTIVE SHIELD FOR MODEL EXTRACTION DEFENSE
Siyu Luan Zhenyi Wang Li Shen?, Zonghua Gu', Chao Wu", Dacheng Tao University of Copenhagen, Denmark UUniversity of Maryland. College Park, USA
3 Shenzhen Campus of Sun Yat-sen University HHofstra University, USA
6 University at Buffalo, USA Nanyang Technological University, Singapore s siyu. luan. sund@gmail com, zwangl 6 69 umd.edu
ABSTRACT
Model extraction aims to acquire a pre-trained black-box model concealed behind a black-box API. Existing defense strategies against model extraction primarily con- centrate on preventing the unauthorized extraction of API functionality. However, two sigmificant challenges still need to be solved: (i) Neural network architecture of the API constitutes form of intellectual property that also requires protection; (ii) The current practice of allocating the same network architecture to both attack and benign queries results in substantial resource wastage. To address these challenges, we propose a novel Dynamic Neural Fortresses (DNF) defense method, employing dynamic Early-Exit neural network, deviating from the conventional fixed archi- tecture. Firstly, we facilitate the random exit of attack queries from the network at earlier layers. This strategic exit point selection significantly reduces the computa- tional cost for attack queries. Furthermore, the random exit of attack queries from earlier layers introduces increased uncertainty for attackers attempting discern the exact architecture, thereby enhancing architectural protection. On the contrary, we aim facilitate benign queries exit later layers, preserving model utility, as these layers typically yield meaningful information. Extensive experiments on defending against various model extraction scenarios and datasets demonstrate the effectiveness of DNF, achieving notable 2x improvement in efficiency and an impressive reduction of up to 12% in clone model accuracy compared SOTA defense methods. Additionally, DNF provides strong protection against neural architecture theft, effectively safeguarding network architecture from being stolen.
INTRODUCTION
Deep learning models have found extensive application in diverse real-world scenarios. Major tech companies such as OpenAl, Google, Meta, and Amazon have developed a range of black-box APIs, offering services to users. This service-oriented paradigm is commonly referred to Machine Learning as a Service (MLaaS) (Ribeiro et al. 2015). The black-box pre-trained models provided by these entities hold significant business value. However, model extraction (Oliynyk al. (023) aims to steal the functionality of the victim model with some query data only. Consequently, safeguarding these APIs from model extraction has become an urgent and critical task.
Existing model extraction defense methods primarily concentrate on safeguarding the functionality of the API from stealing. However, two significant challenges still need be solved. Firstly, network architecture itself can be considered intellectual property (IP) and requires protection against theft. This particularly pertinent determining the optimal architecture often involves substantial computational resources and extensive hyperparameter search (Wistuba et al. [001)) In addition. neural architecture stealing (Oh al. 2018 Rolnick & Kording 2020: Zhu al. 202T: Carlini et al. 2024) is attracting growing interest, driven by the significant commercial value of proprietary model architectures used production systems. Secondly, current defense strategies allocate the same computational power to both benign and attack queries. We contend that this approach is inappropriate, given that attack queries do not demand extensive computational resources.
"Siy Luan and Zhenyi Wang contribute equally. Corresponding author: Zhenyi Wang

---

Published as a conference paper at ICLR 2025
To overcome these limitations, we propose novel defense approach termed Dynamic Neural Fortresses (DNF). This method utilizes a dynamic Early-Exit neural network (EENN) to enhance the efficacy of defenses against model extraction. Our objective is to facilitate the random exit of attack queries at the early initial layers of the network, as these layers typically yield non-semantic and misleading information for attackers. This strategy minimizes the consumption of computational resources for attack queries, leading to a significant improvement in overall running efficiency. Furthermore, by enabling attack queries randomly exit at different layers of the victim model, amplifies the challenge of accurately discerning the exact network architecture of the victim model for attackers. The substantial uncertainty introduced regarding the model architecture serves to safeguard the intellectual property of the model. On the other hand, we aim to enable benign queries to exit at later layers of the network, as these layers typically output meaningful and semantic information, making them suitable for providing valuable insights to legitimate users.
To implement our strategy, we draw inspiration from the deep information bottleneck theory (Alemi et al. 2017). Specifically, when dealing with out-of-distribution (OOD) data, undertake dwo-fold approach. Firstly, we minimize the mutual information between latent features and data labels to deliberately reduce the model's prediction ability, ensuring effective defense. Concurrently, maximize the mutual information between input data and latent features. This action allows the feature extractor to capture non-semantic information in the input data, leading deliberately inaccurate model predictions for enhanced defense. Additionally, this learning objective leads to non-deterministic exits in the classifier to fortify the architecture against potential theft. Conversely, when handling in-distribution (ID) data, our strategy adjusts to maintain model utility. We maximize the mutual information between latent features and data labels, thereby enhancing the association between these features and data labels to improve overall model performance on benign queries. Simultaneously, we minimize the mutual information between input data and latent features. This step enables the feature extractor to capture both compressive and semantic information present in the input data, contributing to a more utility-driven model. We illustrate our approach Figure
To assess the efficacy of the proposed method, we conduct comprehensive experiments aimed at defending against both data-based model extraction (where the attacker uses similar real data to query the victim model) and data-free model extraction (where the attacker uses synthetic data only to query the victim model) in both soft-label and hard-label attack settings. Our method is highly adaptable and can be applied seamlessly various victim model architectures, including both ResNet and large- scale pre-trained Vision Transformer (ViT) models. Its simple design and ease of implementation, using standard línear classifiers without requiring specialized exit classifiers, highlight its exceptional scalability and flexibility. The results demonstrate that our approach consistently surpasses the state-of-the-ue (SOTA) defense method, achieving up a 12% reduction in clone model accuracy. Simultancously, our method significantly enhances running efficiency compared to SOTA defense methods, achieving 2x speedup. Additionally, our method outperforms other defense techniques in terms of overall model utility. Crucially, our defense is notably more effective in model extraction scenarios, regardless of whether attackers utilize OOD data or have access to in-distribution data, highlighting its broad applicability. Furthermore, we conducted additional experiments to evaluate our defense against the SOTA model architecture stealing method (Carlini et al.) 2024). The results demonstrate that our approach effectively protect model architecture from theft.
Our main contributions are summarized as three-fold:
We present the first defense framework that provides three key protective benefits simultane- ously: (1) safeguarding the model's 's functionality while substantially lowering the accuracy of cloned models, (2) securing the model architecture, and (3) greatly improving efficiency in defending against various model extraction scenarios.
We propose novel dynamic Early-Exit network defense method with adaptive deep varia- tional information bottleneck learning objective. To the best of our knowledge, this is the first attempt introduce Early-Exit network into the field of model extraction defense.
Extensive experiments on defending against various model extraction attack settings demon- strate the superiority of the proposed method compared to SOTA defense methods.
2

---

Published as a conference paper at ICLR 2025
2 RELATED WORK
Model Extraction Attack (a) Functionality Stealing: One of the objectives of model extraction is to achieve the same functionality as the victim model or to make their predictions consistent with those of the victim model (Oliynyk et al.) 2023 Nayak et al., 2019 Pal et al. 2020 Jagielski et al.) 2020 Li al. 2023 According the data type used for model extraction, the techniques used in model extraction can be further classified into: (1) Data-based Model Extraction (DBME): DBME is technique that assumes attackers either have knowledge of the training dataset used for the target victim model or possess surrogate dataset to extract knowledge from the victim model (Papernot ctt al.) 2017; Orekondy et al., 2019;; Tang et al. 2024). (2) Data-free Model Extraction (DFME): DFMF is technique that assumes attackers have no prior knowledge of the training dataset and iteratively refine the dataset used to extract knowledge from the victim model based on the model's
output information (Kariyappa et al. 202 Truong et al. 202 Sanyal et al. 2022a Hu et al. 2023).
(b) Architecture Stealing: Architecture stealing is a critical model extraction attack that seeks to uncover the internal architecture of a deployed model (Oh et al. 2018 Rolnick & Kording 2020 Zhu et al., 202I; Carlini et al., 2024). This type of attack is gaining increasing attention due to the high commercial value of proprietary model architectures in production. Attackers target these architectures to gain valuable insights that could be used to replicate or enhance their own models.
Model Extraction Defense Existing model extraction defense can be categorized into two classes.
[1] Model extraction prevention defense. Existing methods consist of: (1) output perturbation
OOreonny et al., 2020; Mazeika et al. 2022); (2) model ensemble (Kariyappa al. 2021b); (3) defensive training (Wang et al. 2023 Hong et al. 2024]; (4) active watermarking [Wang et al.) (222)). However, these methods incur significantly increased computation or memory costs due the need for backpropagation of the victim model during deployment or the storage of multiple models.
[2] Model extraction verification/detection defense (Adi et al. 2018 Jia et al., 2021 Szyller et al., 2021: Dziedzic et al. (022). There are two classes of defense methods: (1) Detection-based defense (Juuti et 2019 Kariyappa & Qureshi 2020; Pal al. (02I)) aims detect whether query is benign or malicious. These defenses become ineffective when attackers change their query data distribution. Furthermore, OOD detectors are vulnerable to attacks (Azizmalayer et al.] (2022). making it easy for adversaries to bypass OOD sreection-baed defenses. (2) Verification-bused defense (Adi et al., 2018 Jia et al. 202T Szyller et al., (02T) can only confirm if a model has been stolen; they do not inherently prevent theft and cannot reduce clone model accuracy. Moreover, when attackers do not make the cloned model public, such methods are incapable of proving model theft.
Our approach falls under the category of model extraction prevention defense since our approach can maximally reduce the clone model accuracy. Existing defense strategies primarily focus on protecting the API functionality from theft. However, two significant challenges still need to be solved. First, the architecture itself holds significant intellectual property value, given the substantial resources invested in searching for the optimal network structure. Therefore, safeguarding the architecture becomes imperative. Second, the current defense methods employ the same architecture to serve both attackers and benign users, resulting in unnecessary resource wastage. In contrast, we introduce the first defense framework that offers three primary protective advantages simultaneously: (1) protecting the model's functionality while considerably decreasing the accuracy of cloned models, (2) shielding the internal model architecture, and (3) significantly boosting defense efficiency against a wide range of model extraction threats.
Early-Exit Neural Networks "Early Exiting" (EE) strategies (Teerapittayanon et al. 2016) incorpo- rate exits into the early layers of DNN to achieve more efficient inference (Matsubara et al. 2022). To the best of our knowledge, this paper represents the first attempt to introduce Early-Exit neural networks (EENN) into model extraction defense. Although EENN can enhance the efficiency of standard DNNs, they are unsuitable for direct application in defending against model extraction since they treat ID and OOD data uniformly, making it challenging to simultaneously maintain model utility and achieve effective defense. In contrast, approach involves different learning objectives tailored for ID and OOD data. This approach plays a pivotal role in preserving model utility while concurrently ensuring effective defense.
3

---

Published as a conference paper at ICLR 2025
API
Couvl
Blaok1
Block2
Blnck3
Block4
Querys
ExIE5
Featurty
Features
Featurre
Peaturee
10
Data
Shield
Erica
Wrong information
0000
,000 0000 0000
Figure 1: Illustration of DNF: ID data is expected to exit at the later layers of the EENN, after more feature extraction operations. Conversely, OOD data (attack queries) is anticipated to exit non-deterministically at earlier layers, undergoing fewer feature extraction operations. This figure shows one scenario that ID data exits from the last layer, while OOD data exits from earlier layers (It is important to note that earlier layers refer to the relatively early exily among all the exits of the network, while the later layers refer to the relatively late exits among all the cxits). This learning goal serves purposes: (I) enhancing uncertainty regarding the model architecture, preventing attackers from precisely determining the victim model's structure; (2) earlier layers may not yield meaningful features, leading the victim model to make incorrect predictions, thereby providing attackers with inaccurate information. Thus, we preserve model utility while achieving effective defense.
PRELIMINARY
Attacker's Knowledge and Goal During model extraction, attacker submits query samples X = {a:):=0 to the victim model Vax, ōv) parameterized by 5y. Through V's API, the attacker obtains the outputs y: V (a:, )v). Subsequently, the attacker constructs the dataset (( )=) The primary objective of the attacker to employ the dataset {(.,, y:) } to train a clone model C parameterized by dc, aiming to endow C' with similar functionality the victim model V. The optimization objective for learning the clone model is: minse Ei= [124, ys, 6c).
When an attacker possesses information about the training data used by the victim model, it is termed as data-based model extraction (DBME). Conversely, attacks operating without any prior knowledge of the victim model's training data are named as data-free model extraction (DFME). Furthermore, two distinct settings of model extraction are identified based on the API return values: soft-label and hard-labed settings, depending on whether the API returns probability values or only the top- label. More details about model extraction can be found Appendix A
Defender's Knowledge and Goal Defender aims to minimize the test accuracy of the clone model on the ID test dataset while preserving the test accuracy of the victim model on the same ID test dataset. The defender operates with limited knowledge, unaware of the attacker's strategy, whether a query is malicious or benign, and details about the attacker's s model architecture. Assumptions widely made that the attack query data OOD in existing model extraction attack and defense works Orekondy et al. (2019); Kariyappa & Qureshi (2020); Kariyappa et al. (2021b); Pal et al. (2020); Truong al. 2222 Wang al. (2023) for several reasons. Firstly, APIs typically offer users limited information, often granting access solely to input-output pairs. This lack of transparency impedes attackers from gaining insights into the specific in-distribution (ID) data used for training Wang (202T). Secondly, the in-distribution data is typically treated as a private asset by the model owner, and is not disclosed to users due concerns related privacy and security. However, it's important to clarify that our assumption regarding the OOD attack query data distribution is made for presentation simplicity. Our defense method is versatile and can be applied to defend against model extraction attacks where the attacker has access to in-distribution data to query the victim model, such as in the JBDA attack (Papernot et al. 2017 Juuti et al. 2019)
:
METHODOLOGY
In this section, we first present the Dynamic Neural Fortresses (DNF) (see Figure [ learning objective in Section 4.1 Then, we discuss the training and deployment process of DNF in Section 4.2
4

---

Published as a conference paper at ICLR 2025
4.1 LEARNING OBJECTIVE OF DNF
The training dataset Dtr utilized to train the victim model is commonly referred to as the ID dataset. The parameters of the victim model, denoted dy, are typically determined by optimizing:
min By
[Co (m,))),, [L y, 5v]]]
(I)
where [. y, dy:) is the cross-entropy loss function for classification task. Traditional victim model utilizes a fixed architecture, to serve both attack and benign queries. This enables the attacker to easily steal the architecture and wastes a lot of computation resources on attack queries. To solve these issues, we propose dynamic network solution. In the following, we first give overview of our DNF defense method. Then, we discuss the learning objectives for ID and OOD data, respectively.
Overview Given training dataset Dr, which contains M classes, the initial step involves training network, as depicted in Eq. (T). Subsequently. the pre-trained parameters dy are frozen, and exit classifiers (V.)EN i=N are added on top of it. V denotes the 1th exit classifier. Following this, fi=0
pre-trained OOD dataset generator Good is employed to generate the OOD dataset Dood along with corresponding labels obtained through the first exit of Early-Exit neural network inference process in V* Appendix Table 20 provides a summary of the symbol notations used throughout the paper for clarity and reference.
Given a pre-trained model V parameterized by dv, to convert it into an Early-Exit neural network V (EENN), as shown in Appendix Figure We first choose the confidence threshold for each intermediate exit classifier during the V inference, and intermediate exit classifiers, the number of Exrly-Exit N (intermediate exit classifiers), and their placements over V. Then, we can freeze the backbone parameters of V, i.e., 8v, and train the intermediate classifiers to optimize the exit classifiers )=0 Each exit classifier Vi-" contains one or two Linear layers for classification, and place the classifier in the appropriate location depending on the structure of the victim model V, we put more details in Appendix E We only add a tiny number of trainable parameters (account for 0.47% of the victim model parameters). It is important to highlight that the exit classifiers use standard architectures without requiring any specialized design, and they can be easily integrated into the victim model in a plug-and-play fashion. This straightforward implementation works seamlessly across various architectures, demonstrating strong adaptability and flexibility.
To obfuscate information gathered by potential attackers, our goal is to maximize the disparity between exit layers for OOD and ID data. We incorporate the entropy value of cach exit layer as an additional regularization measure specific to that exit layer. We denote the victim model outputs at the 7t exit classifier h = V (a, 8..)) (z, 8%-) denotes the jth class probability at 7th exit classifier. 8%, denotes the parameters associated with the 2t classifier, V,-, The entropy on the victim model outputs is defined as the following (where J denotes the number of classes.):
(hh = - - (Z, (x, 8i) log (x,
(2)
The entropy value increases with the model's uncertainty at a given exit layer. Accordingly, we anticipate higher entropy for ID data at earlier exits through (n -0- 1) and a lower entropy value at the later exit. This arrangement ensures that ID benign queries exit at later layers of the victim model to preserve model utility. Conversely, OOD data follows the opposite pattern, with lower entropy values at earlier exits 1 through (n - 1) and a higher entropy value at the later exit. This ensures that OOD attack queries exit at earlier layers, contributing to effective defense (see Figure []. In the following, we will describe the separate learning objectives for II) and OOD data, respectively.
ID Data Learning Objective We present the following learning objectives Eq. for ID queries.
i=N Lid - 2,) I(Z,; Y.a) +a, (()
In Eq. 33, N is the number of the exit classifiers, two distinct loss terms are incorporated: entropy regularization and Information Bottleneck (IB) regularization. Subsequently, we elaborate on each of these loss terms in the following.
(1) Entropy Regularization: H the hyperparameter for the tth exit, and the piecewise function K(i) assigns values based on the
is the entropy regularization defined in Eq. 2 ag is
5

---

Published as a conference paper at ICLR 2025
comparison of the integer i with the constant N. ((1) returns a value of 1 when is equal to the N. If is less than the N. (i) instead returns a value of -1. The objective is to minimize the entropy at the later layer, facilitating the ID query exit at that point, as a lower entropy value indicates a concentration of a probability distribution on a partieular class, as shown in Appendix Figure 5 Simultaneously, we aim maximize entropy the other layers, ensuring that probabilities in those layers are more uniform and flat. This strategy discourages ID queries from exiting in earlier layers.
(2) Information Bottleneck (IB) Regularization: Earlier layers may lack compressive feature infor- mation, rendering those features not suitable for predicting the class label. To tackle this challenge, we introduce the following IB learning objective for ID training data. Let I (Z,; Yia) represent the mutual information between the two random variables, the latent features Z; and data labels Yia- Similarly, T(X id; 7:) denotes the mutual information between the two random variables, input II) data Xia and latent features Z4. In Eq. (3) T(Xid; 7x) T(Z4; Yıa) is the IB loss. We maximize the mutual information between Zj and Yia to enhance the correlation between them to improve generalization. We minimize the mutual information between Xis and Z, to reduce the effect of nuisance information of input Xa on the features Zi.
OOD Data Learning Objective Conversely, we put forth the following objectives for OOD queries:
i N
Good Σ 1=1
[1; - I [(oodd 2.) + a; (x) (2ood))
In Eq. (4), there are two distinct loss terms incorporated: entropy regularization and Information Bottleneck (IB) regularization. It is important to note that the learning objective for OOD data is opposite the case in ID data. Subsequently, we elaborate on each of these loss terms.
(1) Entropy Regularization: (a:ood)) is the entropy regularization defined in Eq. (2). ar is the hyperparameter for the 7th exit, and the piecewise function Jii) assigns values based on the comparison of the integer with the constant N. (i) returns a value of -1 when is equal to the N. If is less than the N, (i) instead returns value of 1. The aim to maximize entropy later layer, promoting flat and uniform probability distribution to prevent OOD data from exiting at later layer. At the same time, we minimize entropy at the other layers, facilitating the output exit at the earlier layers, as lower entropy indicates larger probabilities concentrated on one class.
(2) Information Botteneck (IB) Regularization: Certain earlier layers may retain some informative features crucial for predicting the class probabilities of OOD data. To reduce the correlations between OOD data features and labels and to encourage more random exit of OOD data at earlier layers, we optimize the opposite IB learning objective for OOD data follows: Here, [(Z,; Yood) represents the mutual information between latent features Z; and data labels Y Yood: T(X Zj) denotes the mutual information between OOD data Xood and latent features Z;: In Eq. (4) I(Z;; Youd) I(X (Xood; Zi) is the negative IB loss. This optimization is contrary to the IB optimization applied to the ID data. We minimize the mutual information between Z, and Yood to diminish the correlation between features and predictions, thereby promoting a more random behavior in exit classifiers. Simultaneously, we maximize the mutual information between Xood and Z, to amplify the influence of the nuisance information from input Xood on the features Z,, fostering a stronger defense mechanism.
4.2 TRAINING ALGORITHM OF DNF
While formulate our learning objective in Section 4.
caleulating the mutual information is intractable in general since the mutual information involves calculating an expectation, which often requires integrating over the joint distribution of the variables involved. In many cases, this integration is computationally or analytically intractable. Variational inference provides a framework for approximating these intractable integrals. We derive the variational lower bound as the following:
1(2:Y) - 1(2; 2 (rddyds.((e))PP)))))))) (x) log P(y]z.)
(5)
ff dadz. P(x)) log
P(2,(=)
r(z.)
where r(z;) is a variational marginal approximation of the intractable marginal P(z,) and is set to be r2z,) (O 7). We put the derivation steps in Appendix B

---

Published as a conference paper at ICLR 2025
To efficiently calculate Eq. (5). the P(a,y) = P(x))( is approximated by empirical data distribution P(x, y) 인े4 Em= 62. (z)) (z). dz., the Dirac delta function on nn and yy. is the Dirac delta function on Yn- Assuming the EE network feature extractor has the form P(z|(z:)
(2z,11/ (x), (.)). An MLP/ fc outputs the K-dimensionnl mean μ of 2, and the K x K covariance matrix E. Then can use the reparameterization trick (Kingma & Welling, 2013; Alemi al.) 2017) to rewrite the P(z ((zlz))dz = P(e) dde where 2 = f(x E) represents a deterministic function of a and Gaussian random variable c. This enables the distribution P( x) to be reparameterized as a function of c. We can calculate the KL divergence between the P(z x:) and r(z), and combine all together to minimize the empirical IB loss function the following Eq. (
N
Lid =
Ea(- log q(uaa (æxs, e))] + KKL(P(Z ((2)) + (.) (x.d e))
(6)
6 c ~ (o,, the first term is to maximize the log-likelihood on ID data and the second term minimizes the KL divergence w.r.t to 3(Z) on ID data make the latent features more compressive and extract better representations.
In Eq.
Conversely, the empirical IB loss function for OOD data can be summarized as the following:
N=N
Σ Lood N
E. [log (Hood) Vi (Wood, c)) KIL(P(Z) (B()) + (V) (Wood: e))
(7)
where in Eq. 7 E N~OO,, 1), the first term is to minimize the log likelihood on OOD data and the second term is to maximize the KL divergence w.r.t 3(2) on OOD data make the latent features more diverse so that the their exit points are more diverse, achieving effective defense and protecting the architecture from extracting. The overall learning objective is to minimize the following loss:
La = Lid + Lood
(8)
We summarize the joint training details in Algorithm iin Appendix. In line 3-4, we randomly sample ID and simulated OOD dataset. In line 5-8, we calculate the base DNF training loss by Eq. 66 respectively. Then, we update the Early-Exit neural network V via SGD optimizer with respect to the exit classifiers parameters öy..
Deployment of DNF During testing, given the input query and the array of confidence thresholds r = (ro, 71. Fyy ). where ro represents the first confidence threshold for the first intermediate exit classifier, and so forth. Starting from the input and progressing to the position of the first intermediate exit classifier in the model V*, the first intermediate exit classifier calculates the softmax probabilities Vi*(a) (ar) across all prediction classes. We denote the probability of the jth class in the prediction at the first intermediate exit classifier as (z)) The largest probability value in Vo (æ) is denoted as maz (x) = max, Vū (z). If Vi Vo 2 ro, it signifies that the model V is confident in the current result, enabling the early termination of subsequent calculations. If V maa < To. the inference process continues. We summarize the DNF testing algorithm in Algorithm 2 in Appendix.
5 EXPERIMENTS
5.1 EXPERIMENTAL SETUP
Datasets (1) We evaluate the effectiveness of our method against DFME attack by using MNIST (10 classes) (Deng) 2012), CIFAR-10 (10 classes), 110 datasets (100 classes) (Krizhessky) 2009), and ImageNet- 100 (Vinyals et al. 2016) (100 classes), as these datasets are commonly used in existing DFME research. (2) For evaluating the effectiveness of our method against DBME, following Mazeika et al. 22022, we use Caltech256 (Griffin et al. (007) as the query dataset for both ImageNet200 (200 classes) and CUB200 (Wah et al. 2011) datasets trained victim models.
Baselines We evaluate the superiority of our method against five different strong defense baselines:
(1) Undefended: without any defense. (2) Random Perturb (RandP) (Orekondy et al.) 2020 (3)
P-poison (Oreoody et al. 2020). (4) GRAD (Mazeika et al. 2022) (5) MeCo (Wang et al. 2023)
We put a detailed description of these baselines in Appendix
Implementation Details In our experiments, following (Wang et al. 2023). in DFME attack, the I
perturbation budget is set to 1.0, for the defense baselines in order to mount strong defense against
7

---

Published as a conference paper at ICLR 2025
model extraction. This means that the 4 norm of the difference between y and ÿ. where y represents the original output probabilities and à represents the modified output, does not exceed 1.0. In DFME attack setting, following (Trong et al. 202T, the query budget for different datasets we set is as follows, 2M for MNIST, 20M for CIFARTO 10, 200 for CIFAR 100, M00 for ImageNet- 100. In DBME attack, the query budget is to be 10K for ImageNet200, 2 for CUB200. We report the results with mean and standard deviation with five runs. All experiments are run on a single NVIDIA RTX A6000 GPU. Due space limitations, we put the details of training and hyperparameter in Appendix D exit classifier architecture in Appendix [ and OOD generator in respectively. Exir Threshold Selection Guideline: We propose a sample-efficient Bayesian optimization framework for automatically selecting the exit threshold. The detailed selection process is outlined in Appendix H
Farly-Fxit Victim Model For MNIST dataset, LeNet-5 is the backbone model. For CIFAR TEAARI&1I 100 and ImageNet- 100 datasets, ResNet34-8x (He et al. 2016) is the backbone model. For CUB200 dataset, ResNet50 is the backbone model. For ImageNer200 dataset, Swin ViT (Liu et al. 202T) is the backbone model. During the DNF training, we freeze backbone parameters and only train exit classifiers. We put the victim model training details in Appendix
5.2 DEFENSE AGAINST DATE-FREE MODEL EXTRACTION
DFME attack baselines We adopt the following DFME attack baselines: the soft-label attack method
DFME (Truong et al. 2021), and the hard-label attack method DFMS-HL (Sanyal et al. 2022a).
Results on CIFARIO and CIFARI00 The results of defense against soft-label and hard-label DFME attack on CIFARIO 10 and CIFAR100 are shown in Table Our EENN's backbone is based on ResNet34. We use three distinct model architectures as the clone model architectures, which include R8 (He et al. 2016). MobileNetV2 (Sandler et al.) 2018) and 1e (Huang et al., 2017). Compared to the undefended method, under soft-label attack setting, DNF can significantly reduce clone model accuracy by 27% to 34% the CIFAR 10 dataset and by 17% to 41% on the CIFAR 100 dataset. Under hard-label attack setting, DNF can significantly reduce clone model accuracy by 4% to 8% on CIFARIO 10 and by 14% 19% on CIFAR 100. The clone model accuracy under different defense methods with varying query budgets is shown in Figure 2 in Appendix !|
Table 1: Clone model accuracy on CIFAR-10 and CIFAR- 100 with ResNe134 as the victim model.
Attack
Defense
CIPAR-BBUbo: Medel Architettum
CIPAR-100 Chere Medel Architenne
ResNettS-SX
MoDileNarV2
DesseNett21
ResNettS-SX
MobiieNer22
DenseNert 21
Undelended
57.36
DFMB
KandP
7000
22.7
(Soft label)
Pipesoon
79.38
1.68%
65.82
2.72%
20.71 2.50%
22.05
McCo
51
1.96%
16.55 2.00%5
23.57
12:18 1.05%
10.79
DNF
53.91
2.30%
1.45%
2.15%
18,03
3,03%
10,82
1.23%
UndeSended
64.67 4: 1.00%
79.28 2: 1.57%
68.67 2 2.50%%
72.67 :: 1.28%
02.71 1: 1.68%
63.58 à: 1.79%
RandP
84.02 4: 2.31%
78.71 2: 1.50%
68.16
2.22%%
72.48 2: 1.43%%
62.06 2: 1.82%
63.16 2: 1 1.703
DFMS HL.
goison
$4.06 4: 1.57%
79.02
2: 1.50%
65.05
2.17%
71.83 2: 1.32%
61.83 4: 1.79%
62.78 1.91%
(Had-labl)
GRAD
64.26 1.66%
78 25.85-1
1.91%
68.11
1.98%
71.89 + 1.57%
02:60 + 1.71%
02.57
1.0%
MiCai
76.86 + 2.005
71.22
+ 1.57%
62:33
2.000
$9.30
+ 1.705
$5.32
:: 1.65%
56.80
1.56%
DNF 1
76.51 1: 2.12%
75.01 #: 1.25%
61.02 :: 1.21%
52.95 :: 2.24%
48.41 :: 1.78%
49.72
1.24%
For MNIST and ImageNet-100 dataset, we put detailed results in Appendix [.1
Model Utility Evaluation. We evaluate the victim model utility with various defense strategies in Table We can see the DNF maintains high test accuracy compared with other defense strategies. This proves that our DNF method not only improves the defense capability and computational efficiency, but also improves the victim model utility for legitimate users. We evaluate the accuracy of ID data at each exit on CIFARIO 10 dataset. We use 10,000 test samples and force predictions to occur at a single exit, shown in Figure 3 in the Appendix, the model achieves progressively higher accuracy at later exit points, with corresponding error bars indicating the standard deviation.
5.3 DEFENSE AGAINST DATA-BASHR MODEL EXTRACTION
To evaluate the effectiveness of DNF under DBME attack, we follow the random strategy in Knockoff Nets (Orekondy al. 2019). The attacker randomly samples images from different distribution than that of ID data to extract the victim model. For uncertainty- based sampling as proposed in ActiveThief (Pal et al. 2020), provide the results in Table 19 in the Appendix.
Results on CUB200 and ImageNet200 To verify the effectiveness of DNF on CUB200 and Im- ageNer200 datasets, our EENN's backbone is ResNet50 and Swin Transformer, respectively. We
8

---

Published as a conference paper at ICLR 2025
choose ResNet50, ResNet34, and VGG19 the clone model architectures for CUB200 dataset and Vision Transformer (ViT) (Dosovitskiy et al. 2020) (Large), CaiT (Touvron et al., 2021b). DeiT (Touvron et al. 2021a) and Swin (Liu et al. (022)) (Large) for ImageNet200 dataset. The results are shown in Table 2 Compared to the undefended method, under soft-label attack setting, DNF
can significantly reduce clone model accuracy by 31 % to 40% on the CUB200 dataset and by 11% to 19% on the ImageNet200 dataset. Under hard-label attack setting, DNF can significantly reduce clone model accuracy by 10% to 15% on the CUB200 dataset and by 12% to 16% on ImageNet200.
Table 2: Clone model accuracy on CUB200 and ImageNet200 with 5e as the victim model.
Attack
Defense
CUB200 Clime Model Archisecture
ImageNNe1200 Clune Model Archilecture
RecNet50
ResNet34
VGG19
Large
Call
DelT
Swin Large
Undefonded
56.49
1.50%
38.71
0.79%
39.55 85 1.71%
73.35
0.50
62.26 1.244%
56.59
1.54%
00.67 1: 1.45%
RandP
30.77
0.35%
20.50
1.3.5%
69.68
1.78%
53.07
P-poison
24.94
1.225%
15:33
20:81
1..3%
65.53
1006
08.91
1.745
52.08
0.5.%%
52.67
1.00%
Son-label
GRAD
24.32 0.56%
15.06
1.11%
20.65
+-0.4%%%%
63.32
066885
50.24
0.78%
51.97
1.82%
55.70
0.57%
MeCo
$1.92 0.54%
31,98
0.73%
31.35
0.255%
60:53
0.3255
60.17
1 00.20%
53.78
0.799%
58.85
1. 0.735%
DNF 1
15.32
1.21%
5.27
1.72%
8:21 1 2.23%
$6.03
1.12%
46.21
1 0.91%
45.31
1.09%
42.10
1 1.26%
Undefiende 1
31.29 1.58%
21.57 1 0.62%
23.27
1 0.80%
63.57
1 0.699%
57.73 1. 0.80%
53.16
1.52%
00.26 1 1.23%
RandP
30.80
0.6155
21.68
0.913
23.44 1: 1.34%%
63.18 1 144
67.91 1: 0. 80%
52.85
0.65%
59.58 1: 1.01%
P.
30.09
0.9115
1,985
1.41%%
G3L0
0.6336
67.12 0.515
52.57
1.12%%
1.87%
Hardi label
GRAD
31.23 0.613%
22.38
+-444%
63.21
1494
67:25
52.34
1.54%
MeCo
20.42
0..05%
0.42%
0.54%
63.31
18%
52
0.45%
DNF 4
16.01
1.24%
10.91
1.02%
12.98
1.25%
48.95
2.11%
43.65 3: 0.91
40.98 1.69%
44.01 1: 0.82%
Table 3: Effect of different OOD datasets on de- fense performance during DNF training on Ima- geNet200 with Swin Transformer as the victim model.
Table 4: Victim model utility measured by the
Test accuracy
Mathael
CIFARCI
CIPARII
CLIR300
Imag:Netlt)
Undelended
0.37%
26.71
1.25%
81.88 1.21%
0.52%
RasdP
95
0.96%
75.93 + 13.95-
50.58
4-0-11%
91:574 4-0.5%
Setting
Underfondedt
DNF (Inkour67))
DNF (SVHN) J. DNF (CIAN)
Pipeison
94.53
0.015
1..42
0.334%
01.14
Sof 73.35 = DUB5ES
Hard-label
03.57 - 0.0955
56,00 = 1.12%
48395 + 9.1156
$5.511 181%
4873 + 1.72%
10.71 1.718
19.18 1.91%
GRAD
MeClo
91:65
94.74.00075
0.65%
75:70=0..005
75.00
75.36 = 0.6855
1.45%
50.23
79.42 # 0.53%
40.10
155:
91.38
90.00 0.536
01.00:0:.37.
0.51%
5.4 IN-DISTRIBUTION JBDA DEFENSE AND ARCHITECTURE STEALING DEFENSE
Defense Against JBDA with In-Distribution Data Attack: To assess the effectiveness of DNF when the attacker has access to in-distribution data for queries, we tested its performance under the JBDA (Papernot et al. 2017 Juuti et al., 2019) attack. As shown in Table 17 DNF achieves the lowest clone model accuracy of 40.60% in the loft-label attack. In the label attack, it reduces the clone accuracy to 44,, outperforming other defense methods. Overall, DNF consistently demonstrates robust performance in lowering clone model accuracy across both attack scenarios.
To assess the effectiveness of our defense against model architecture stealing, proposed in (Carlini al., 2024). we use a pre-trained ViT-Large model with 200 output classes on the ImageNet200 dataset as the victim model. The objective is to steal the hidden dimension prior to the class logits output. We set the hidden dimensions for different exit classifiers to be [80, 100, 120, 140, 160]. Using 300 images from the ImageNet200 dataset, query the victim model and obtain output matrix 300xx00 Following the method described in (Carlini et al. 2024), we calculate and sort the singular values of O as X 2 X2 2 ... 2 An- The index that produces the largest difference between consecutive singular values is selected the hidden dimension. The output of the architecture stealing algorithm in (Carlini et al. (2024) identified 137 as the hidden dimension, which is entirely different from the hidden dimensions set in our exit classifiers. This discrepancy demonstrates the effectiveness of our defense in protecting the model architecture from being stolen.
Defense against Model Architecture Stealing:
5.5 ADAPTIVE ATTACKS
We further evaluate the robustness of DNF against potential countermeasures by attackers. We proposed two types of adaptive attacks. Specifically, (1) we consider the DFME soft-label attack, where attackers are aware of our defense strategy, including the early-exit network architecture and loss function, referred to as the Adaptive Entropy Attack. In this scenario, attackers employ adaptive attacks by incorporating the same designed entropy loss our DNF into the clone model's loss function, updating the clone model together to attempt bypassing our defense. (2) We also evaluate an EE Adaptive Entropy Attack, where attackers have even more information, using the same carly-exit architecture as the victim model. As shown in Appendix Tables 9 DNF is still effective against these two adaptive attacks.
9

---

Published as a conference paper at ICLR 2025
5.6 ABLATION STUDY
Exit Point Threshold Evaluation We evaluate the influence of the threshold choices for each exit point. As shown in Appendix Table[TO| the threshold selection does not affect the defense ability of DNF too much. We also propose a sample-efficient Bayesian optimization framework to automatically select the optimal exit threshold Appendix H
Different OOD Dataset Evaluation We assess the impact of using different OOD datasets for DNF training. For instance, as demonstrated in Table DNF consistently reduces clone model accuracy across different OOD datasets, including Indoor67, SVHN and GAN-generated, with small variation in clone model accuracy. This highlights the stability of DNF regardless the OOD dataset used.
Inference Efficiency Evaluation Compared with Fixed Architecture To assess the inference efficiency of dynamic network architecture as opposed to fixed architecture, we conduct a comparison of their respective efficiency in Appendix Table II which shows that DNF achieves notable 2 improvement in inference efficiency than the SOTA defense method and undefended model.
Fine-tuning Efficiency We measure the training time of DNF. As shown in Appendix Table 13 & 14 the wall-clock time of training is about 20 seconds for the MNIST dataset, the wall-clock time of training is about 7 hours for Swin Transformer, the fine-tuning is highly efficient.
To assess the exit point percentage of input query data on attack query and benign query data, analyze the exit point percentages for attack and benign query data in Appendix Table 12 which reveals that 54.93% of OOD data generated by the attacker is exiting from the initial exit point. Because the initial exit point typically doesn't generate lot of meaningful information, which results in these OOD queries disrupting the training process of the attacker's 's model. Some OOD data exiting from later layers could be attributed when processing the ID data, it is noteworthy that over half of the data exited through the third exit. This significantly reduces the time consumption for model inference. This can be attributed to the advantages of early-exit networks in avoiding overthinking, as highlighted by (Kaya et al., (209)), which also enhance inference efficiency for ID data. Overall, ID data tends to exit at relatively deeper layers compared to OOD data. A more detailed explanation of the ID and OOD exit percentages across different exit classifiers is provided in the Appendix above Table 12
Exit Point Evaluation
shifts in data distribution. Conversely,
The impact of different loss weights on the results We measure if the different weights of ID and OOD losses Li + -ood in Eq. (8) will influence the final results. As shown in Appendix Table 15 the change in clone model accuracy is not large, which illustrates DNF's stability.
The impact of different entropy weights a on the clone model accuracy of different entropy weights in Eq. (6) and (7) on the defense performance. As shown in Appendix T8] the OOD data loss term contribute significantly to the effectiveness of our approach. Despite variations in the entropy weight, the changes in defense performance not large, demonstrating the stability of DNF across different a.
We measure the impact
Effectiveness of DNF protecting the architecture searched by neural architecture search (NAS) In addition to the traditional network architectures, we also verified whether our method can be used on more general architectures searched by NAS (Lu euetal al. 2020p. Our method is still effective as shown in Appendix Table 16
CONCLUSION
We present DNE, a novel defense strategy aimed at efficiently countering model extraction attacks by achieving three key objectives: (1) safeguarding model functionality, (2) protecting the network architecture, and (3) improving defense efficiency. Our method steers attack queries to exit at earlier network layers for effective protection, while allowing benign queries to exit at later layers, thereby preserving model utility. Comprehensive experiments across various model extraction scenarios and datasets confirm the effectiveness and efficiency of our approach.
Limitations Our approach does not protect the entire network architecture from being stolen. Even though certain parts of the network are executed, the attacker cannot determine which inputs corre- spond to which specific exit classifiers. This uncertainty helps preserve the protection of the network architecture.
10

---

Published as a conference paper at ICLR 2025
REFERENCES
Yossi Adi, Carsten Baum, Moustapha Cisse, Benny Pinkas, and Joseph Keshet. Turning your weakness into strength: Watermarking deep neural networks by backdooring. 27th USENIX Security Symposium (USENIX Security 18), 2018.
Alexander A. Alemi, Ian Fischer, Joshua V. Dillon, and Kevin Murphy, Deep variational information bottleneck. In International Conference on Learning Representations, 2017.
Mohammad Azizmalayeri, Arshia Soltani Moakar, Arman Zarei, Reihaneh Zohrabi, Moham- mad Taghi Manzuri, and Mohammad Hossein Rohban. Your out-of-distribution detection method is not robust! In Advances in Neural Information Processing Systems, 2022.
Nicholas Carlini, Daniel Palcka, Krishnamurthy Dj Dvijotham, Thomas Steinke, Jonathan Hayase, A. Feder Cooper, Katherine Lee, Matthew Jagielski, Milad Nasr, Arthur Conmy, Eric Wallace, David Rolnick, and Florian Tramèr. Stealing part of a production language model. In Forty-first International Conference Machine Learning, 2024.
Li Deng. The mnist database of handwritten digit images for machine learning research [best of the web]. IEEE signal processing magazine, 291)):14 2012.
Alexey Dosovitskiy, Lucas Beyer, Alexander Kolesnikov, Dirk Weissenborn, Xiaohua Zhai, Thomas Unterthiner, Mostafa Dehghani, Matthias Minderer, Georg Heigold, Sylvain Gelly, et al. An image is worth 16x16 words: Transformers for image recognition scale. arXiv preprint vrXiv:2010.11929, 2020.
Adam Dziedzic, Muhammad Ahmad Kaleem, Yu Shen Lu, and Nicolas Papernot. Increasing the cost of model extraction with calibrated proof of work. In International Conference on Learning Representations, 2022.
lan J. Goodfellow, Jean Pouget-Abadie, Mehdi Mirza, Bing Xu, David Warde-Farley, Sherjil Ozair, Aaron Courville, and Yoshua Bengio. Generative adversarial networks. In Advances in Neural Information Processing Systems, 2014.
Gregory Griffin, Alex Holub, and Pietro Perona. Caltech-256 object category dataset. 2007.
Kaiming He, Xiangyu Zhang, Shaoqing Ren, and Jian Sun. Deep residual learning for image recognition. In IEEE Conference on Computer Vision and Pattern Recognition, 2016.
Ziming Hong. Zhenyi Wang, Li Shen, Yu Yao, Zhuo Huang, Shiming Chen, Chuanwu Yang. Mingming Gong, and Tongliang Liu. Improving non- non-transferable representation learning by harnessing content and style. In The Twelfth International Conference on Learning Representations, 2024.
Zixuan Hu, Li Shen, Zhenyi Wang, Baoyuan Wu, Chun Yuan, and Dacheng Tao. Learning to learn from apis: black-box data-free meta-learning. In International Conference on Machine Learning, pp. 13610-13627. PMLR, 2023.
Gao Huang, Zhuang Liu, Laurens Van Der Maaten, and Kilian Q. Weinberger. Densely connected convolutional networks. In IEEE Conference on Computer Vision and Pattern Recognition, 2017.
Matthew Jagielski, Nicholas Carlini, David Berthelot, Alex Kurakin, and Nicolas Papernot. High accuracy and high fidelity extraction of neural networks. In USENIX Security 2020, 2020.
Hengrui Jia, Christopher A. Choquette-Choo Varun Chandrasekaran, and Nicolas Papernot. Entan- gled watermarks as a defense against model extraction. In 30th USENIX Security Symposium, 2021.
Mika Juuti, Sebastian Szyller, Samuel Marchal, and N. Asokan. Prada: Protecting against dnn model stealing attacks. In IEEE European Symposium on Security and Privacy, 2019.
Sanjay Kariyappa and Moinuddin K Qureshi. Defending against model stealing attacks with adaptive misinformation. In IEEF/CVF Conference on Computer Vision and Pattern Recognition, 2020.
11

---

Published as a conference paper at ICLR 2025
Sanjay Kariyappa, Atul Prakash, and Moinuddin K Qureshi. Maze: Data-free model stealing attack using zeroth-order gradient estimation. In IEEE/CVF Conference on Computer Vision and Pattern Recognition, 2021a.
Sanjay Kariyappa, Atul Prakash, and Moinuddin K Qureshi. Protecting {dnn]s from theft using an ensemble of diverse models. In International Conference on Learming Representations, 2021 b.
Yigitcan Kaya, Sanghyun Hong, and Tudor Dumitras. Shallow-deep networks: mitigating network overthinking. International conference on machine learning, pp. 2301-3310. PMLR, 2019.
Understanding and
Diederik P Kingma and Max Welling. Auto-encoding variational bayes. arXiv:1312.6114, 2013.
arXiv preprint
Alex Krizhevsky. Learning multiple layers of features from tiny images, 2009.
Guanlin Li, Guowen Xu, Shangwei Guo, Han Qiu, Jiwei Li, and Tianwei Zhang. Extracting robust models with uncertain examples. In International Conference on Learning Representations, 2023.
Marius Lindauer, Katharina Eggensperger, Matthias Feurer, André Biedenkapp, Difan Deng, Carolin Benjamins, Tim Ruhkopf, René Sass, and Frank Hutter. Smac3: A versatile bayesian optimization package for hyperparameter optimization. Journal of Machine Learning Research, 23(54):1 2022.
Ze Liu, Yutong Lin, Yue Cao, Han Hu, Yixuan Wei, Zheng Zhang, Stephen Lin, and Baining Guo. Swin transformer: Hierarchical vision transformer using shifted windows. In Proceedings of the IEEE/CVF international conference computer vision, pp. (0012-10022) 2021.
Iyya Loshchilov and Frank Hutter. Decoupled weight decay regularization. arXiv preprint
arXiv:171.05100, 2017.
Zhichao Lu, Kalyanmoy Deb. Erik Goodman, Wolfgang Banzhaf, and Vishnu Naresh Boddeti. Nsganetv2: Evolutionary multi-objective surrogate-assisted neural architecture search. In Computer Vision-ECCV 2020: 16th European Conference, Glasgow; UK, August 23-28, 2020, Proceedings, Part 16, pp. 35-51. Springer, 2020.
Yoshitomo Matsubara, Marco Levorato, and Francesco Restuccia. Split computing and early exiting for deep learning upplications: Survey and research challenges. ACM Computing Surveys, 55(5):
1-30, 2022.
Mantas Mazeika, Bo Li, and David Forsyth. How to steer your adversary: Targeted and efficient model stealing defenses with gradient redirection. In International Conference on Machine Learning. 2022.
Gaurav Kumar Nayak, Konda Reddy Mopuri, Vaisakh Shaj. R. Venkatesh Babu, and Anirban Chakraborty. Zero-shot knowledge distillation deep networks. International Conference on Machine Learning, 2019.
Seong Joon Oh, Max Augustin, Mario Fritz, and Bernt Schiele. Towards reverse-engineerring black-box neural networks. In International Conference on Learning Representations, 2018.
Daryna Oliynyk, Rudolf Mayer, and Andreas Rauber. know what you trained last summer: A survey on stealing machine learning models and defences. ACM Computing Surveys, 55(14s): 2023.
Tribhuvanesh Orekondy, Bernt Schiele, and Mario Fritz. Knockoff nets: Stealing functionality of black-box models. In IEEE/CVF Conference on Computer Vision and Pattern Recognition, 2019,
Tribhuvanesh Orekondy, Bernt Schiele, and Mario Fritz. Prediction poisoning: Towards defenses against den model stealing attacks. In International Conference on Learning Representations, 2020.
Soham Pal, Yash Gupta, Aditya Shukla, Aditya Kanade, Shirish Shevade, and Vinod Ganapathy. Ac- tivethief: Model extraction using active learning and unannotated public data. In AAAI Conference Artificial Intelligence, volume 34, 2020.
12

---

Published as a conference paper at ICLR 2025
Soham Pal, Yash Gupta, Aditya Kanade, and Shirish Shevade. Stateful detection of model extraction attacks, 2021.
Nicolas Papernot, Patrick McDaniel, Ian Goodfellow, Somesh Jha, Z. Berkay Celik, and Ananthram Swami. Practical black-box attacks against machine learning. In ACM Asia Conference on Computer and Communications Security, 2017.
Alec Radford, Luke Metz, and Soumith Chintala. Unsupervised representation learning with deep convolutional generative adversarial networks. In International Conference on Learning Represen- tations, 2016.
Mauro Ribeiro, Katarina Grolinger, and Miriam AM Capretz. Mlaas: Machine learning as a service. In 2015 IEEE 14th international conference on machine learning and applications (ICMLA), pp. 896-902. IEEE, 2015.
Maria Rigaki and Sebastian Garcia. A survey of privacy attacks in machine learning. ACM Computing Surveys, 56(4):1 1-34, 2023.
David Rolnick and Konrad Kording. Reverse-engineering deep ReLU networks. In International conference on machine learning, pp. 8178 8187. PMLR, 2020.
Mark Sandler, Andrew Howard, Menglong Zhu, Andrey Zhmoginov, and Liang-Chieh Chen. Mo- bilenetv2: Inverted residuals and linear bottlenecks. In IEEE/CVF Conference on Computer Vision and Puttern Recognition, 2018.
Sunandini Sanyal, Sravanti Addepalli, and R. Venkatesh Babu. Towards data-free model stealing in hard label setting. In JEEE/CVF Conference on Computer Vision and Pattern Recognition, 2022a.
Sunandini Sanyal, Sravanti Addepalli, and R Venkatesh Babu. Towards data-free model stealing in hard label setting. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 15284-15293, 2022b.
Scbastian Szyller, Buse Gul Atli, Samuel Marchal, and N. Asokan. Dawn: Dynamic adversarial watermarking of neural networks. ACM Multimedia, 2021.
Minxue Tang, Anna Dai, Louis DiValentin, Aolin Ding, Amin Hass, Neil Zhenqiang Gong, and Yiran Chen. Modelguard: information-theorett defense against model extraction attacks. In 33rd USENIX Security Symposium (Security 2024), 2024.
Surat Teerapittayanon, Bradley McDanel, and Hsiang-Tsune -suun Kung. Branchynet: Fast inference via early exiting from deep neural networks. In 2016 23rd international conference pattern recognition, pp. 2464-2469. IEEE, 2016.
Hugo Touvron, Matthieu Cord, Matthijs Douze, Francisco Massa, Alexandre Sublayrolles, and Hervé Jégou. Training data-efficient image transformers distillation through attention. International conference on machine learning, pp. 10347-10357. PMLR, 2021a.
Hugo Touvron, Matthieu Cord, Alexandre Sablayrolles, Gabriel Synnaeve, and Hervé Jégou. Going deeper with image transformers. In Proceedings of the IEEE/CVF international conference on computer vision, pp. 32-42, 2021b.
Jcan-Baptiste Truong, Pratyush Maini, Robert J. Walls, and Nicolas Papernot. Data-free model extraction. In IEEE/CVF Conference Computer Vision and Pattern Recognition, 2021.
Oriol Vinyals, Charles Blundell, Timothy Lillicrap, Daan Wierstra, al. Matching networks for one shot learning. Advances in neural information processing systems, 2016.
Catherine Wah, Steve Branson, Peter Welinder, Pietro Perona, and Serge Belongie. The caltech-uesd birds-200-2011 dataset. 2011.
Zhenyi Wang, Li Shen, Tongliang Liu, Tichang Duan, Yanjun Zhu, Donglin Zhan, David Doermann, and Mingehen Gao. Defending against data-free model extraction by distributionally robust defensive training. Advances in Neural Information Processing Systems, 36, 2023.
13

---

Published as a conference paper at ICLR 2025
Zhenyi Wang, Yihan Wu, and Heng Huang. Defense against model extraction attack by bayesian active watermarking. In Forty-first International Conference on Machine Learning, 2024.
Zi Wang. Zero-shot knowledge distillation from a decision-based black-box model. In International Conference Machine Learning, 2021.
Martin Wistuba, Ambrish Rawat, and Tejaswini Pedapati. survey on neural architecture search. arXiv preprint arXiv: 2019.
Yuankun Zhu, Yueqiang Cheng, Husheng Zhou, and Yantao Lu. Hermes attack: Steal DNN models with lossless inference accuracy. In 30th USENIX Security Symposium (USENIX Security 21), pp. 1973-1985 USENIX Association, 2021.
14

---

Published as a conference paper at ICLR 2025
In this Appendix, provide more details of Model Extraction Details in A Derivations of the Information Bottleneck Lower Bound in the details our experiment bases 11 hyperparameter selection in Section D exit classifier architecture in Section OOD Generator in Section F the Victim Model Backbone Training in G and the more the Performance evaluation of defense methods budget increases in Section and the results of experiments in Section []
the training and
A MODEL EXTRACTION DETAILS
Soft-label Attack In soft-label DFME attack, generative model G is employed to generate input data. Initially, attacker samples random noise vector (V, T), which is then input into the generative model G to produce an image represented model C is trained to align its predictions with those of the victim model V on the input data æ, expressed the following:
G((e, 0c). Subsequently, clone
min KL(C((,, 6V)
6c
Hard-label Attack In a hard-label DFME attack (Sanyal et al. (202bb, a generative adversarial network (GAN) Goodfellow et al. (2014 based architecture is trained generate query data for querying the victim model and training the clone model C. Initially, DCGAN Radford et al. (2016) trained by imposing an image prior using synthetic or unrelated proxy data, serving as the initial training phase for the generator G. Subsequently, the clone model C' and the generator G undergo alternating training. The generator synthesizes data æ through The victim model V processes æ and delivers the hard-label j(x:) to the attacker. The attacker constructs input-label pairs (ax (xx)) using this information, which are then utilized train the clone model C. Additionally, the generated data æ is used to train the generator G' through adversarial and diversity losses.
((c, 6g), where e NO, 7).
B DERIVATIONS OF THE INFORMATION BOTTLENECK LOWER BOUND
Given ((2;))) Y) and I(Z; X), we have the following derivation.
I(,,))) /
dx dy dz, P(s))
(10)
I(Z,, X) / dz; dx P(x,, zi) log
P(z((z)) P(z:)
diz dx P(x, zi) log P(=,(x)) - / dz; P(zs) log P(zi)
(11)
We use ((Z) be a variational approximation to
P(x:) = 1 f P(z7(æ) P(x)
(12)
Based on Eq. 12). Eq. can be rewritten as 13
X): S do dz, ((x) log P(2[2)
(13)
r(z:)
Based on 10 and
we have that:
I(Z,Y) - I(;; dıdydz, P(x))(()=))P(2)))) log loggly)z) g(vlz))
(14)
dzdz, P(z) log P(2(())
L.
(15)
r(zi)
15

---

Published as a conference paper at ICLR 2025
C BASELINE
: Rigaki & Garcia (023): the paper proposes to perturb the prediction vectors of the victim model, the goal is poisoning the training objective of the attacker.
RandP Orekondy et al. (2020);
Prediction Poisoning Orekondy al. (2020): the paper introduces a perturbation objective termed 'Maximizing Angular Deviation (MAD). The core objective of MAD is to alter the prediction probabilitiiss of the vietim model. This alteration strategically designed to produce an adversarially perturbed gradient, which diverges maximally from the original gradient of the victim model.
GRAD Mazeika et al. (2022): the paper represents gradient redirection method, designed to enable the adversarial gradient update in any arbitrary direction. The purpose of this method is to control the direction of the gradient update.
MeCo Wang et al. (2023): the paper represents both memory and computation efficient defense method through distributionally robust defensive training by adding a data- data-dependent random perturbation generator to perturb the input data. The attacker cannot steal useful information from the black-box model, the same time, MeCo can keep the target model utility.
D THE TRAINING AND HYPERPARAMETER DETAILS
When training the ResNet34-8s 8x backbone for CIFARI0/100 datasets, we set 200 epochs in total, the initial learning rate is 0.1, optimizer is SGD. The learning rate will be adjusted as epochs grow. If the number of epochs is smaller than 80, the learning rate is 0.1, if the number of epochs is between 80 with 120, the learning rate is 0.01. If the number of epochs is bigger than 120, the learning rate is 0.001.
When training the DNE, optimizer is SGD, the learning rate is 0.001. For MNIST dataset, we set confidence thresholds array as [0.92, 0.94, 0.96] by default. For CIFAR10/100, TinyImageNer CUB200 and ImageNet200 datasets, we set confidence thresholds aray as [0.90, 0.92, 0.94, 0.96] by default. In our design, the default confidence thresholds are progressively increasing. Due to our loss function, this ensures that decisions at earlier exits are made with relatively lower confidence, allowing OOD data to exit earlier, while ID data progresses to later exits where decisions are made with higher confidence. Additionally, opt for different thresholds across different exit layers for two reasons. First, this approach enhances defense against adaptive attacks. It makes it more challenging for attackers to accurately predict the correct threshold for all exit layers simultaneously. Second, using different threshold values for each exit increases the separation between the exit distributions of ID and OOD data, further improving defense performance.
the 0.1 by default, the number of epochs is 10, and the
E THE ARCHITECTURE DETAILS OF EXIT CLASSIFIER
If is ResNet34-8x 8x which contains 20 million trainable parameters, based on its structure features, we add the exit classifier after each Residual Blocks of victim model V. in total, excluding the original V exit, we add four more exits {V.- = The architecture details for ResNet34-8x based exit classifier as shown Table and all exit classifiers (V: contains 100440 trainable parameters, in total, we add 0.47% trainable parameters, which is acceptable.
Table 5: The architecture details of exit classifiers, M is the numbher of classes.
Exit Index
First layer
Second layer
Parumeters
Exit1
Lineur(64, 100)
Linear(100, M)
7510
Exit2
Linear 128, 100)
Linear(100, M)
13910
Linear(255 100)
Linear(100, M)
26710
Exit4
Linear(512, 100)
Linear(100, M) 52310
If V is LeNet-5 which contains 61K trainable parameters, based on its structure features, we add the exit classifier after each convolution layer of victim model V, in total, excluding the original V exit,
16

---

Published as a conference paper at ICLR 2025
add three more exits {V;")= The architecture details for LeNet-5 based exit classifier as shown in @ and all exit classifiers {= contains 16990 trainable parameters, in total, we add 27% trainable parameters, which is acceptable.
Table 6: The architecture details of exit classifiers, M is the numbher of classes.
Exit Index
Hirst layer
Parameters
Exitl
Lincar (1176, M)
11770
Exir2
(400, Linear (400, M)
Exit3
Lincor (12c, M)
1210
The architecture details of exit classifiers for other networks, such as transformer-series are similar to this, the difference is we will use LayerNorm.
F OOD GENERATOR
For generating outto--distribution (OOD) data, we use a Deep Convolutional Generative Adversarial Network (DCGAN) Radford et al. (0016). The training dataset for this DCGAN is the TinyImageNet (if the TinyImageNet is the ID dataset, we choose other datasets as the training dataset, such as SVHN), widely recognized subset in the field of computer vision. We set the learning rate (Ir) to 0.0002, and 30 epochs in total. Furthermore, we choose the Adam optimizer model training.
G VICTIM MODEL BACKBONE TRAINING
For LeNet-5 as the victim model for MNIST datasets, we run 20 epochs in total and set the initial learning rate 0.001, optimizer as Adam. For ResNer34-8x He et al. (2016) as the victim model for CIFAR10G 100 and ImageNet- 100 datasets, we run 200 epochs in total and set the initial learning rate as 0.1, optimizer as SGD. For Swin Transformer (Swin) Liu et al. (202T) as the victim model for ImageNet200 datasets, we run 30 epochs in total and set the initial learning rate as 0.01, optimizer as Adam with decoupled weight decay (Adam W) Loshchilov & Hutter (2077). Furthermore, we follow Tang et al. (2024) for the clone model training.
the procedure in Truong al. (021 :
H BAYESIAN HYPERPARAMETER OPTIMIZATION FOR EXIT THRESHOLDS
We provide the following step-by-step guide to Bayesian Hyperparameter Optimization for Exit Thresholds Selection
Step 1: Early exit networks have multiple exits where predictions can be made. Each exit has an associated threshold (confidence level) that determines whether to exit or proceed to the next stage. Let's denote these thresholds the optimal set of thresholds r = {ro, Ti..., Fx } that minimize the clone model's 's accuracy on the validation set of the target dataset. To ensure generalization, we select the exit thresholds using different query or attack dataset than the one employed by the attacker. For example, if the victim model trained on CIFARIO 10, the attacker may use ImageNet for extraction. To select the exit thresholds, we query the victim model with third dataset, such as CIFAR 100, ensuring no prior knowledge of the attack query data distribution. We then assess the clone model's accuracy on CIFAR10's validation set to evaluate the performance of the chosen thresholds. We employ the Bayesian Optimization framework. specifically SMAC3 (Lindauer et al., 2022), which leverages surrogate model to approximate the objective function and efficiently guide the search for optimal exit thresholds.
(ro. /i, FN } for N-exit network. The goal is to determine
Step 2: Set the search space for each threshold r; to be within the range [0.85,1]. Start with an initial set of samples to the surrogate model. The optimizer of SMAC3 will then iteratively select the next set of thresholds to evaluate by balancing exploration (trying new values) and exploitation (refining around known good values). For each set of thresholds r = (ro, ri, rx proposed by the optimizer, train the early exit network and measure the clone model accuracy dataset. Return the result to the Bayesian optimizer of SMAC3 and update the surrogate model. Once the optimization process is complete, select the set of exit thresholds {ro, r1, TN) that
the validation set of the target
17

---

Published as a conference paper at ICLR 2025
yielded the best performance according the objective function. Furthermore, opt for different thresholds across different exit layers for two reasons. First, as this approach enhances defense against adaptive attacks. It makes more challenging for attackers to accurately predict the correct threshold for all exit layers simultaneously. Second, using different threshold values for each exit increases the separation between the exit distributions of ID and OOD data, further improving defense performance.
I PERFORMANCE EVALUATION OF DEFENSE METHODS AS BUDGET INCREASES
As shown in Figure 2 as budgets grow, the performance of clone models continues to improve, but DNF's defense performance always better than other methods.
Clone model accuracy varies with increasing query budget
90
Clone Model Accuracy
60
50
40
30
10
12
14
16
18
20
Query Budget (million)
Upperbound
-X
Pppiison
-
MeCo
- RandP
* GRAD
- DNF
Figure 2: Performance of cloning models as budget increases for different defense methods. The plot shows the relationship between clone model accuracy and increasing query budget (in millions) under different defense methods. The x-axis represents the query budget, ranging from 6 to 20 million, while the y-axis shows the elone model accuracy, which increases the query budget grows. The graph compares several defense strategies, including RandP, P-poison, GRAD, McCo, and DNF, DNF consistently yields the lowest clone model accuracy across all query budgets.
J MORE EXPERIMENT RESULTS
J.1 RESULTS ON MNIST AND IMAGENET-100
Results on MNIST For MNIST dataset, under soft-label attack setting, compared to the undefended method, the results shown Table 7 that DNF can significantly reduce the effectiveness of existing soft-label DFME attack by up to 42% when the clone model is LeNet5, 50% when the clone model LeNet-Half.
Results on ImageNet-100 For ImageNet- 100 dataset, under soft-label attack setting, compared to the undefended method, the results shown in Table @ that DNF can significantly reduce the effectiveness of existing soft-label DFME attack by up to 25% when the clone model is ResNet- 22% when the clone model is MobileNetV2, and 17% when the clone model is 1
18

---

Published as a conference paper at ICLR 2025
Table 7: Clone model accuracy on MNIST with LeNer5 as the victim model.
Clone Model Architecture
Attack
Defense
LeNet5
LeNet5 Half
Undefanded
98.76 0.27%
96.65 0.43%
DFME
RandP 4
92.25 + 0.32%
91.56 0.40%
P-goison
88.31 0.78%
86.09
0.90%
(Soft (Shel-
GRAD
$7.22 + 0.70%
85.38
0.91%
MeCo
85.07 1. 0.87%
82.93
1.27%
DNF
55.97 + 3.19%
46.26
5.78%
Table 8: Clone model accuracy on ImageNet-100 with ResNet34-8r as the victim model.
Clone Model Architecture
Attack
Defense
ResNet18-8X
MobileNerV2
DenseNet121
Undefended
35,89 1: 3.97%
26.71
3,25%
25.05 4: 3.68%
RandP
30.76 1: 1.09%
22.06 1: 3.83%
20.23 :: 3.97%
P-poison
29.36 4.25%
21.83 3.77%
20.01
3.89%
(Soft-label)
GRAI
29.87 3.76%
21 21.60 3.70%
3.77%
MrCo
23.29 1: 3.83%
17.83
3.67%
16.73
3,88%
DNF (Ours)
10.21
0.65%
6.78 0.92%
7.87 1.23%
J.2
ABLATION STUDY
Table 9: Clone model accuracy on CIFAR-10 with ResNet34-8r as the victim model under Adaptive Attack.
Attack
Defense
Accuracy
Original Soft-label Attack
Undefended
87.36 0.78%
Adaptive Entropy Allack
DNF 1
57.62 :: 2.24%
EE Adaptive Entropy Attack
DNP 1
59.07 2.03%
Table 10: Exit Point Threshold Effect on clone model accuracy on CIFAR-10 with ResNet34-8x as the victim model under Original Soft-label Attack.
Method
Clone model accuracy
Victim model accuracy
[0.90, 0.90, 0.90, 0.90]
58.62
1.25%
94.50
0.19%
10.91, 0.91, 0.91. 0.91]
57.31
12%
94.78 0.22%
[0,90, 0.92, 0.94 0.96
53.91 3.30%
91.34 0.07%
[0.95, 0.95, 0.95, 0.95]
1.2%%
Table 11: Running time (seconds) during deployment
Algorithm
CIFARIO
CIFAR100
Undefended
10.12 2: 1.83
10.96
2.09
poissn
382.49
2.78
GRAD
183.47 : 1.66
543.19
1.29
McCo
10.36 0.80
11.17
DNFJID
6.42 2.54
6.89
2.51
DNF.OOD
2.21
593
2.32
we observe that the majority of ID data exits at the third exit, with some exiting at earlier points. Similarly, most OOD data exits at the first exit, and some OOD data also proceed to later exits. We provide an explanation for this in the following:
In Table 12
Why some ID data exit in earlier classifiers: Most ID samples exit at the third exit, while the majority of OOD samples exit at the first exit classifier. The pre-trained victim model is optimized and trained on the ID training dataset, resulting in higher prediction confidence when predicting for ID data. This enables some test ID samples to be predicted with high confidence at early exits, such the third exit. This results in some ID data exits earlier.
19

---

Published as a conference paper at ICLR 2025
Why some OOD data exit in later exits: A significant portion of OOD samples exit at the first exit, providing noisy and misleading information that disrupts attacker's 's training. When the generated attack OOD data more challenging for the victim model to classify. resulting in lower prediction confidence, some OO1) data may exit through later exits. However, these
OOD data may differ significantly from the IDD training data, and using such OOD data to extract the victim model could result in a lower-qualite clone model. Additionally, substantial portion of OOD samples still exit the first stage, further ensuring that our method remains effective in defending against such attacks.
Why our method can still effectively defend when some OOD exit later classifiers: Even for OOD samples that exit at the fourth stage, our optimization objective encourages the model to memroize the data representations by maximize the mutual information between the input data and latent representations, this worse data representations reduce the model's generalization ability to OOD data. Furthermore, we minimize the mutual information between features and labels, thus the correlation between data and labels is weakened, lowering the prediction accuracy for OOD data and achieving the desired defensive effect.
Table 12: Exit point evaluation during deployment
Exit points
Exi10
Exitl
Exit2
Exi13
Exit4
ID Percent
33.78
6.39
1.51
0OD Percent (%)
54.90
0.84
9.10
26.32
8.84
Table 13: Victim model fine-tune time for ResNet model (seconds)
Method
MNIST
CIFARIO
CIFARI00
ImageNet-100
DNF
20.52
65.52
125.51
780..9
Table 14: Victim model fine-tune time for VIT model (hours)
Method
ImageNet200 (GAN)
ImageNer220 (Indoor67)
ImageNet200 (SVHN)
DNF
6.51
7,26
6,23
Table 15: Clone model accuracy CIFAR-10 with ResNe134-8r as the victim model under different loss weight.
Weight
Clone model accuracy
Victim accuracy
0.8
58.21 # 5.5%
91.79 1: 2.1%
2.30% 56.31 1.69%
94.34 0.07%
91.78 1.19%
Table 16: Clone model accuracy on CIFAR-10 with NAS model as the victim model.
Method
Clone model sccurscy
Victim model accuracy
Undefended
83 0.16% 48.11 1 1.23%
96.58 2.00% 96..2 1.95%
DNF
Table 17: Clone model accuracy under JBDA attack on CIFAR-10 with VGG as the victim model.
Defense
Accumcy
Undefended
56.63 1.24%
RandP 1
48.70 7.3%
JBDA
P-poison
12.51 = 2 02%
GRAD
43.31 1.70%
(Sof--lbel)
McCo 1
50.02
1.28%
DNF L
40.60 1.22%
Undefended
19.88 2.03%
RandP
19.12 - 0.83%
JBDA
P-poison
48.51 + 1.97%
(Hard-Label)
GRAD
18.75 = 13%
McCo 1
48.24 = 2.96%
20

---

Published as a conference paper at ICLR 2025
Table 18:
Clone model and victim model accuracy on CIFAR-10 with Re as the victim model under different Entropy loss weight
Weight a
Clone model sccuracy
Vietim model accuracy (uiility)
Without OOD
78. # 1.23%
91.79 9.9%%
Entropy Weight
3.20%
0.07%
Entropy Weight = 0.2 58.71 = 2.02%
95.12 = 0.98%
J.3 ACTIVETHIEF SAMPLING EXTRACTION DEFENSE RESULTS
Table 19: Victim and clone model accuracy with and without ActiveThicf sampling. Methods
Teacher
Student
Random sampling (no defense)
81.18 + 1.21%
55.49 + 1.50%
Random sampling (DNF, ours)
80.13 + 0.86%
15.32 + 1.21%
ActiveThief uncertainty-sampling (no defense)
81. + 1.21%
57.85 # 0.37%
ThiiveThief uncertainty-sampling (DNF, ours)
80.13 + 0.86%
17.01 # 2.18%
J.4 NOTATION TABLE AND ALGORITHM
Table 20: Table of symbol notations.
Symbol
Description
Input data
Data's label
I
Loss function
V
Victim model
Clone model
Dtr
The training dataset
M
The number of classes in the dataset
The exit classifiers
Good
Pre-trained OOD dataset generator
V
The Early-Exit neural network
gth
The j class probability
H
The entropy regularization
K
The piecewise function
The piecewise function mutual information
Z;
The latent features
Gaussian random variable Entropy loss weight Network weights
8
04 ((%) (ro. [I.. [x]
The hyperparameter for the exit variational approximation
Confidence thresholds
As shown in Figure We show the experiment results about the accuracy of ID data at each exit. We use 10,000 samples and force predictions to occur a single exit, reporting the accuracy of the early-exit network. Since OOD data lack true labels, their accuracy cannot be computed.
As shown in Figure it is a diagram of neural network with an early exit mechanism, where multiple intermediate classification exits (Exitl to Exit4) are placed at different feature extraction stages, allowing the model to exit early when certain conditions are met to reduce computation, while retaining the main exit (Exit5) for final classification.
The figure Iillustraaes the relationship between different probability distributions and their corre- sponding entropy values. The distributions range from a uniform distribution with maximum entropy (2.3026) to a deterministic distribution with zero entropy. Each bar chart represents the probability associated with ten classes, highlighting how entropy decreases the distribution becomes more concentrated on specific classes. These visualizations demonstrate the fundamental principle that entropy reflects the level of uncertainty randomness in probability distribution.
21

---

Published as a conference paper at ICLR 2025
Algorithm DNF Training.
Algorithm DNF Testing.
1: Require:
Pre-trained victim model V with parame- ters dy; M is the number of training iterations; the in- distribution (ID) training dataset, Dor; pre-trained OOD dataset generator.
1: Require: EENN V*; x is the input data; the array of confidence thresholds r =
{ro, 71, ... TNN..
2: for k = to M do
2: for i = to N do
3: randomly sample new mini-batch data (ax y) from
Vimax
max, (x, c).
'Ddd
where ()
4: randomly sample a mini-batch OOD data from the pre-trained OOD dataset generator.
4: if V L.max 2 ri then
The inference stops.
5:
calculate empirical IB loss for ID data (Eq. (6
6: Return V (x, else
6: calculate empirical IB loss for OOD data (Eq. (7))
7:
7: calculate the overall DNP loss function (Eq.
8: The inference process continues.
8: update the exit classifiers {VM of Early-Exit model V* by gradient decent with Eq. (
9: end if
9: end for
10: end for
Model Accuracy at Different Points ICIFA9-10)
1.00
Figure 3: Model accuracy on the CIFAR-10 dataset at different exit points, utilizing ResNet34-88, as the backbone.
Conv1 Block1
Block2
Block3
Block4
ExitS
Querys
Featurea
Feature,
Fenture,
Feature
Exit
Exit2
Exit3
Exite
Is conf idnfidece greater than 0.90?
Is conf idence greater than 0.92?
Is confidee fidecce greater than 0.94?
Enirapy
Prosabiityy Divttatton and Entroy
0.14
Next exit!
Next exit!
Exit Now!
Figure 4: Neural Network with Early Exit Mechanism.
22

---

Published as a conference paper at ICLR 2025
Entropy 2.3026
Entropy 2.2647
Entropy 1.5316
Entropy 0.2547
Entropy 0.000
Figure 5: Visualization of Probability Distributions and Corresponding Entropy Values.
23


Reviews:
[{"review_id": "dHvqGcF4MN", "reviewer": "Reviewer_dSR3", "summary": "The paper presents “Dynamic Neural Fortress” or DNF framework as a defense against Model Extraction Attacks. These attacks allow an adversary to create a copy of a pre-trained model accessible via black-box APIs, posing risks to proprietary models. The authors identify two main challenges in current defenses: (1) Neural Network architecture protection, a thing that is taken for granted in previously proposed attacks by using the same model architecture for victim and clone models, and (2) optimizing computational resources by avoiding allocation of equal resources to both benign and attack queries. \n\nThe authors implement an Early-Exit neural network wrapper (EENN) on top of a trained model. This wrapper facilitates random exits at earlier layers for attack queries while preserving model utility by making benign queries exit at later layers. The authors assume the usage of out-of-distribution (OOD) data by attackers in most cases, but there are some experiments conducted for in-distribution (ID) data as well. Using concepts from deep information bottleneck theory, the authors optimize mutual information between input data, latent features, and output labels for training the EENN model. \n\nThe proposed method has been evaluated via testing on various architectures and datasets, and compared against other state of the art defenses.", "questions": "- The authors claim their approach falls under the model extraction prevention defense category. Still, it works like a detection approach where the OOD detector is built into the model itself and thus relies heavily on the OOD data used for classification. The results shared by authors, to argue otherwise, are insufficient. I would ask the authors to include more experiments for this argument. \n- If the model is trained to early exit in the case of OOD samples, but the labels used are from the original neural network (essentially the last possible exit), what is the accuracy of the model on OOD data used for training the model? I suspect that the early exit model misclassifies OOD data with high confidence. If it were learning the original network’s output labels for OOD data, then the defense would not work for the hard-label setting as the attacker would still receive a large portion of the original network’s labels as output with some erroneous ones.\n- Regarding the exit point evaluation ablation study, I would like to know the accuracy at each exit and the exact number of ID and OOD samples passing through each exit instead of terms such as “over half,” etc.", "limitations": "", "rating": 8, "confidence": 5, "soundness": 2, "presentation": 2, "contribution": 2, "strengths": "- The proposed idea of implementing early exits as a defense against model extraction is novel and sound.\n- The method is easily adaptable to different architectures like ResNets and ViTs. \n- The use of entropy and information bottleneck theory is sound and well-suited to the goal of reducing extractable information for the attacker.\n- The experiments conducted cover various scenarios, models and datasets validating its generalizability. The performance comparisons with state-of-the-art defenses further strengthen its credibility. \n- The ablation study is thorough and captures various scenarios that highlight the effectiveness of the proposed method and its components.", "weaknesses": "The paper presents a technically sound idea, but the presentation is poor and needs major revisions. I am listing the weaknesses sectionwise. \n### Related work:\n- The related work is not organized properly, and some works are not cited in their appropriate sections, although they are cited later in the paper. For example, ActiveThief by Pal et al. (2020) [1] should be present under functionality stealing. \n- When a model extraction attack is data-based, the data might be natural or synthetic. For E.g., I can generate a dataset of 10,000 images from a pretrained generative network and use that for model extraction. This would still fall under the category of data-based model extraction. Data-free model extraction means that the data used for stealing is generated based on some information received from the victim. \n- Therefore, restructuring the related work section is necessary here. \n\n### Methodology:\n- The steps followed to convert a pre-trained victim model into an EENN are not easily followed. A network is trained on the ID data first. Then exit classifiers are added on top of it. Then, an OOD generator is used to generate OOD data, which is then passed through the original network without the exit networks for inference. The steps followed after this are not written in a coherent manner. One has to go through Algorithm 1 to get a clear picture of the training process.\n- Overuse of the term specific to start two consecutive paragraphs (224-235 and 236-241) and even inside the paragraphs when the sentences contained in both paragraphs are not specific at all. \n\n### Experimentation:\n- The authors should differentiate between the DFME and DBME settings in more detail. In line 387, it is assumed that the reader will know that they are talking about the DFME setting instead of the soft-label setting. This also invites confusion regarding the budget difference between the soft and hard label settings, where the budget should be the same for valid comparison. \n- For the DFME setting, one clone model architecture should be the same as the victim model for valid comparison (Resnet-34 in this case). Previous methods, like the prediction poisoning [2] method used by authors for comparison, have conducted experiments that keep the victim architecture for the stolen model. Moreover, the proposed method is not better than MeCo for the CIFAR-10 dataset. This should be analyzed and discussed.\n- For the DBME setting, using the random strategy for sampling images is not ideal. It has been shown in the ActiveThief [1] paper that using an uncertainty-based sampling method is more effective. \n- To showcase the effectiveness of the in-distribution defense, using JBDA as the attack strategy is fairly obsolete, and the paper cited needs to be corrected. The paper that proposed the attack is  [3]. The authors should use either ActiveThief or Knockoff nets attack for evaluation as they are more recent and utilize intelligent sampling-based strategies for attack. If an actual attacker has access to in-distribution data, they will try to use the best strategy possible. \n- To demonstrate the defense’s effectiveness against model architecture stealing, the authors pick the latest attack by Carlini et al. but fail to show effectiveness against previously cited work, specifically “Towards reverse-engineering black-box neural networks. In International Conference on Learning Representations, 2018.” that perform attack on imagenet models. Considering that this was one of the major claims made by the authors, they should evaluate this aspect thoroughly. \n\n\n### Grammar:\nThe paper has incoherent paragraphs, spelling mistakes, and redundant sentences. Some of them are listed below:\n- Line 225, it should be “convert” instead of “covert.”\n- In Table 1 and Table 2, the spelling of label is incorrect. \n- Appendix D, Lines 778-779, same line repeated twice. \n\nCitations:\n- [1] Pal, Soham, et al. “Activethief: Model extraction using active learning and unannotated public data.” Proceedings of the AAAI Conference on Artificial Intelligence. Vol. 34. No. 01. 2020.\n- [2] Orekondy, Tribhuvanesh, Bernt Schiele, and Mario Fritz. “Prediction poisoning: Towards defenses against dnn model stealing attacks.” arXiv preprint arXiv:1906.10908 (2019).\n- [3] Papernot, Nicolas, et al. “Practical black-box attacks against machine learning.” Proceedings of the 2017 ACM on Asia conference on computer and communications security. 2017.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "5SBOcCjypX", "reviewer": "Reviewer_xham", "summary": "This paper introduces a new defense against model extraction attack for model architecture and model utility. The key idea is to use multi-exit neural network architecture and its random exit mechanism to protect the network's architecture while ensuring the efficiency. For benign queries, the authors trains the early-exit model to distinguish OOD data (attack queries) and in-distribution data to ensure the model utility.\nFinally, the authors show that DNF outperforms previous defenses and evaluate the adaptive attacks.", "questions": "Can you provide a formal definition or description of in-distribution and out-distribution data in this paper's setting? How to distinguish the normal user data (OOD) and attack data (OOD)?", "limitations": "", "rating": 6, "confidence": 3, "soundness": 3, "presentation": 3, "contribution": 3, "strengths": "+ Good motivation. The authors adopt multi-exit architecture to defend architecture extraction attack, which is a well motivated and interesting idea.\n+ Extensive evaluation. The authors not only evaluate the defense effectiveness but also adaptive attacks.", "weaknesses": "- The assumption of attack data are OOD data, although widely adopted in prior work, should be more carefully justified. Meanwhile, as the model's training data are unknown to the user, benign queries may also be OOD data. DNF might decrease the model utility in this case.\n- The main part of paper (Section 4) is somehow hard to follow. I would suggest the author to simplify the notations or subscripts. Moreover, I also suggest the authors to provide an overview figure to replace some descriptions.\n- Although the authors investigate the adaptive attacks, the adversary can still design more powerful attack by exploiting the multi-exit model.  Please discuss more about the potential vulnerability of multi-exit architecture and compare with prior attacks on multi-exit networks.\n\n[1] Auditing Membership Leakages of Multi-Exit Networks. ACM CCS 2022.\n\n[2] Model Stealing Attack against Multi-Exit Networks. arXiv:2305.13584.\n\n[3] Mind your heart: Stealthy backdoor attack on dynamic deep neural network in edge computing. IEEE INFOCOM 2023.\n\n[4] Aegis: Mitigating Targeted Bit-flip Attacks against Deep Neural Networks. Usenix Security 2023.\n\n[5] Prediction Privacy in Distributed Multi-Exit Neural Networks: Vulnerabilities and Solutions. ACM CCS 2023.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "CXqIEtrkoQ", "reviewer": "Reviewer_zh4c", "summary": "The dynamic neural fortress (DNF) defense method introduced in this paper employs a dynamic early exit neural network to defend model extraction attacks. This approach effectively provides simultaneous protection for model functionality, network architecture, and enhanced defense efficiency against these threats. Extensive experiments demonstrate that the proposed defense method outperforms SOTA model extraction defenses in terms of both effectiveness and efficiency.", "questions": "* Can the proposed defense be easily extended to other tasks and domains, such as object detection and NLP applications?\n\n* Does the number of exit points impact the performance of the proposed defense?\n\n* According to the design, earlier blocks are intended to reduce the model's predictive capability. However, it is notable that the ID dataset maintains high accuracy even after exiting at Exit2. This raises questions about the effectiveness of the defense mechanism. Moreover, the OOD dataset still retains 35% of its data after passing through the last two blocks. What is the observed defense effect in this case?", "limitations": "", "rating": 6, "confidence": 3, "soundness": 3, "presentation": 2, "contribution": 2, "strengths": "* The first defense framework simultaneously offers three key protective benefits: protecting the functionality, and model architecture, while improving the efficiency of the inference.\n\n* An innovative design of the loss function is achieved by incorporating the Information Bottleneck (IB) theory.\n\n* The experimental design is well-structured and covers various scenarios, effectively validating the method's effectiveness.", "weaknesses": "* The claims regarding the protection of model architecture are overstated. Early Exit (EE) mechanisms indeed prevent attackers from executing the entire pipeline of DNN, therefore protecting the entire model architecture information from being leaked. However, the authors fail to provide how attackers might exploit this vulnerability to steal the model architecture when executing the entire network. Furthermore, EE mechanisms typically occur in the last few layers of DNNs; therefore, while the proposed approach may protect certain layers, it only works those that are unexecuted, leaving the majority of the neural network vulnerable (if there are effective attacks that can steal the model architecture). The authors should consider discussing these limitations in a dedicated section titled \"Limitations.\"\n\n* The definitions of out-of-distribution (OOD) and in-distribution (ID) data lack clarity. It is unclear why the authors consider OOD data to be \"illegal\" while ID data is deemed \"legal,\" and the rationale behind the corresponding loss term needs further explanation. Additionally, the authors aim to minimize the mutual information between $X_{id}$ and $Z_{id}$ in Eq. (3). However, this approach could potentially compromise the overall performance of deep neural networks (DNNs). The authors should provide additional clarification on why a reduced mutual information between $X_{id}$ and $Z_{id}$ does not impact the prediction accuracy.\n\n* Table 12 indicates that queries drawn from ID dataset exit at Exit2 over 90%, while the OOD queries only exit at about 75% at the same stage. This discrepancy seems inconsistent with the motivation behind two loss terms in Eq. (3) and Eq. (4). The authors should explain this discrepancy and discuss how it impacts the effectiveness of the proposed defense mechanism. I would like to suggest the authors provide a more detailed analysis of the exit patterns for ID vs OOD data.\n\n* The explanation for choosing a specific mutual information optimization method to achieve the defense objectives lacks a deeper theoretical explanation and intuitive justification, making it challenging to fully follow the principles behind the proposed method.\n\n* The experiments conducted to protect the model architecture appear limited, which does not sufficiently demonstrate the contribution related to model architecture protection mentioned in the paper. Consider adding additional experiments and evaluation metrics specifically designed to assess the robustness of the model architecture against potential theft. \n\n* It would be advantageous to include experiments that investigate the correlation between accuracy and exit points, providing a clearer visualization of the early exit mechanism's impact. I would like to suggest a graph showing accuracy vs. exit points for both ID and OOD data or report a statistical analysis of this relationship.\n\n* It seems that all datasets utilized are classification datasets, which makes it difficult to validate the effectiveness of the proposed method in other tasks and domains.\n\n* The notations in this article have been used repetitively, e.g., $r$.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "eZyr33wMG6", "reviewer": "Reviewer_FTna", "summary": "Model extraction is a type of attack where an attacker tries to replicate a victim model to either:\n1. Estimate the model’s parameters to emulate the model’s performance\n2. Copy the model’s architecture, to recreate the model as-is.\n3. Get protected knowledge of the training data of the victim model, to better understand the data distribution it was trained on, so that other type of adversarial attacks can be done.\n\nExisting defense strategies are costly – they do not differentiate between benign and malicious queries from an attacker and this form of defense allocates the same computational power to both. This paper provides a novel way to tackle model extraction attacks – Dynamic Neural Fortresses. \n\nThey propose an early-exit strategy wherein the victim model has built-in early exits routes that the model can take and provide outputs that are OOD from it’s expected input-output combination. If an input query matches an early-exits threshold, the model inference exits with the output at that stage.", "questions": "1. Concepts related to entropy and IB regularization are presented with some mathematical rigor and learning objectives for both ID and OOD data are presented with entropy and IB regularization contratints; However some additional insights into potential limitations are necessary – How would the strategy perform under adaptive attacks with a much varied and increasingly sophisticated OOD spectrum? And how it would impact models that aim for domain generalizability and to incorporate that OOD spectrum into their model’s capabilities?\n2. How does this defensive method translate to multi-modal architectures like VLMs. Or multi-pipeline methods where each branch operates on different modalities? Or ML methods where different models are trained for different modalities and their outputs are combined (via some aggregation)?", "limitations": "", "rating": 6, "confidence": 3, "soundness": 3, "presentation": 2, "contribution": 3, "strengths": "1. The paper presents an interesting defenseive method to counter model extraction attacks. The paper’s novelty lies in the core idea of using a dynamic exit strategy based on the input query. While early exit strategies have been explored in the context of neural networks, their application to defensive methods is novel.\n2. The paper is well written, and the core idea is simple to understand. The language is lucid but see weakness 2, 3.\n3. The paper is well organized with a clear progression between sections. Figure 1 greatly aids clarity in trying to understand the pipeline, however see weakness 2.\n4. Experimental evaluation is robust and does seem to support the author’s claims that DNF achieve substantial reduction is successful model cloning.\n5. This paper addresses a growing concern in the space of AI/ML model deployment – protecting against model cloning and privacy and intellectual rights protection. This work does have the potential to help drive forward work in defense space for these attack types.", "weaknesses": "1. Despite strength 5, this method can be adapted widely only after these weaknesses are addressed and questions explored.\n2. Should make better use to visual elements – probably, atleast in the appendix, add an example of what an attack query would look like, why the victim system would classify the query as attack and what the victim model’s behaviour would be on it, how early would it exit?\n3. Math is useful and helps to aid the reader’s understanding but at times also hampers readability. Especially in textual sections it breaks the flow of readers. Something that may help is condense the math and limit them to equations that can be repeatedly referenced or have a table of symbol notations that readers can refer to.\n4. Some sections could use with clearer explanations - OOD Data Learning Objective, underlying theory for Entropy and IB regularization. Maybe providing examples around mutual information or ER could help.\n5. The paper does provide some explanation about Entropy and IB regularization but could expand a little more on how mutual information reduction leads to lower predictability and can be leveraged  for distinguishing between benign and malignant queries.\n6. Maybe a comparison with other information-theory based approaches such as standard adversarial training would help drive home the imminent advantages on DNF. Another set of comparisons that could  strengthen the paper’s results are against other dynamic architectures (example ‘BranchyNet’).\n7. The paper uses ER to determine optimal exits from the model’s inference. However the choice of thresholds is only briefly discussed. Maybe an ablation study of various hyperparameters, exit thresholds and entropy weights could help explain the choice a certain threshold or explain the assumptions that the authors may have made.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "1VP9nuyC4G", "reviewer": "Reviewer_CXR5", "summary": "In this paper, a defense against model stealing attacks (targeting either the model architecture or its functionality) based on a multi-exit neural network is proposed. The main idea is to output accurate prediction scores for ID data from the later network exits, as well as uninformative scores for OOD data from the earlier exits. To do so, for each network exit, a thresholded classifier is trained on the respective intermediate layer representation with a specifically designed loss, which maximizes the aforementioned objective using concepts from information theory. During the deployment, an exit is chosen for a sample when the maximum score of an exit classifier exceeds the respective threshold.", "questions": "- Could you please estimate the impact of early exiting for IID samples? For instance, you might compute the misalignment in model outputs for IID samples when they exit early with respect to being forwarded into the entire network.\n- Could you please evaluate the defense against a worst-case attacker, enhancing the already implemented adaptive attacks with (partial) knowledge of the training data distribution?", "limitations": "", "rating": 8, "confidence": 4, "soundness": 2, "presentation": 3, "contribution": 4, "strengths": "- The paper presents a clearly novel idea to address a very relevant issue. Indeed, to the best of my knowledge, this is the first application of a multi-exit neural network to defend against model extraction attacks.\n- The proposed network architecture can also reduce the inference time during deployment.\n- The approach is very intuitive and well-justified.\n- The reported results are promising.", "weaknesses": "- 90% of IID samples exit in the first 3 exits. Although this can be viewed as a benefit (it reduces the inference time), on the other side, the defense mechanism will produce less informative outputs for those samples. The impacts of these effects should be clearly understood.\n- I appreciate the fact that the authors consider different types of attacks and try to implement adaptive ones. However, a best practice when dealing with security is to simulate a worst-case scenario against the strongest attack. This helps understand the limitations of the defense and estimate lower bounds of robustness in these settings - even if, in practice, they are unlikely to occur. In this case, the adaptive attacks should be implemented using model extraction techniques that rely on some knowledge about the training data distribution. This assumption is not too unrealistic, as it might happen that the attacker (who knows the domain on which the model is applied) is able to gather in-distribution data from public domains - for instance, if the model is a malware detector, it should be very easy to collect samples and also very likely to have some overlap between them and the training data used by the victim. In other cases, the attacker might possess a subset of or all the training data, and she could easily train its own model, but she is rather interested in reproducing the exact model functionality and reproducing its decision boundaries to build a surrogate model and use it for other attacks (like evasion ones, aka adversarial examples).", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}]

Author rebuttal:

Decision: Accept (Poster)